Filter date is in UTC format

Posted by: greg.frazer on 11 December 2017, 3:14 am EST

    • Post Options:
    • Link

    Posted 11 December 2017, 3:14 am EST

    When I filter on a date column, by condition, and then examine the filterDefinition conditions, the date is in UTC format.

    I am in Melbourne, Australia ie GMT +11 for summer time

    So when I selected a date in the filter dialog, of 1/4/2017 it is in the filterDefinition as ‘2017/03/31T13:00:00.000Z’

    That makes it hard to do filtering, impossible really. as the date in the flexgrid is ‘2017/04/01T00:00:00.000Z’

    The date in my Database is 01/04/2017 as a SQL date field. which is THE correct date.

    What is wrong here is the date emitted by the Filter filterDefinition.

    It seems that the filter Dialog is calling date.toISOSting() method.

    BTW the filter is excluding too many rows.

    How can we change the filter format so that it has the date that is in the grid?

  • Posted 12 December 2017, 1:53 am EST

    Hi Greg,

    Probably, the service used for fetching data from server returning the date with string data type. This will be reason for displaying date in UTC format. To overcome this, we need to parse JSON data at client to convert string type date to Date object. Please refer to the following thread for the same:

    https://www.grapecity.com/en/forums/wijmo/wijmo-5-flexgrid-column-fo_1#wijmoarchivehello-allthe-f

    ~Manish

  • Posted 13 December 2017, 11:51 pm EST

    I understand what Bernardo is saying, but the FlexgridFilter must do some “date Math” to do the comparisons.

    I agree with @rmiller2.

    Wijmo ought to allow UTC dates and then internally use “date Math” for comparison on filter/sorting/grouping.

    I have 5000 rows and 439, 000 cells in what is a small-ish grid (later to be 200,000 rows).

    If I have to examine every field in the itemsSource to insert data objects, I fear the performance will make the grid unworkable.

    If the cells date is ‘2017-12-14T00:00:00Z’ ,

    then FlexGrid.getCellData(row,col,false) will return ‘2017-12-14T00:00:00Z’

    and FlexGrid.getCellData(row,col,true) will return ‘14/12/2017’ (for GB and AU) and ‘12/14/2017’ (for US)

    so how is that not parsing the date object.

    All we ask is that you allow UTC dates and do ‘date Math’ based on the browser settings

  • Posted 14 December 2017, 5:10 am EST

    Hi Greg,

    We forwarded this issue to the concerned team for further investigation.

    ~Manish

  • Posted 14 December 2017, 8:00 pm EST

    bernardo

    That is fine, but I knew all that; and I suppose most experienced javascript devs do too.

    But still worth repeating.

    The issue here is not how the date is formatted but how to is stored in the grid and how calculations are done.

    1. How it is stored in the grid. We, that is the other posters in the link Manish supplied, would like to store it as a string. And I suppose that the UTC format is the best for server and client components.
    2. how calculations are done. Seriously, why do I care. Given that the format is string UTC, the component author, that’s you Bernardo, picks up the date, performs the calculation and outputs the string UTC format again. It doesn’t matter what time zone the browser is in, because the calculation is done using that time zones string UTC. It is up to the Server developer to supply the correct date in the string UTC format.

    And for filtering/sorting/grouping at the server end, that string UTC will need to be handled by the server dev.

    In short string UTC in the grid is better that a Date object; because we don’t need to examine/change every cell in all the date columns in our itemsSource before we load the grid

    I hope this makes the issue and requirement clearer.

  • Posted 15 December 2017, 12:24 am EST

    Sorry. I don’t follow…

    The grid does not control how dates are stored. It shows and edits the data in the format it received it. If you give it strings, it will show and edit those strings (without conversions). If you give it Date objects, it will show and edit those (using our Globalize methods to format and parse the dates).

    UTC strings are definitely not “better than Date objects”. They cannot be formatted, or used directly in calculations (add a month, get the difference between two dates, get the quarter, etc). You would have to convert back and forth constantly. Not only on the grid, but on your entire UI!

    I think the core of this discussion is the fact that JavaScript dates are really UTC at heart (milliseconds since 1970 in Greenwich) but are formatted as local values. So the same date looks different on machines in different time zones. This has nothing to do with the grid, and I don’t think it’s a good reason to abandon Date objects.

    The ODataCollectionView allows you to perform this conversion once, when you load the data, so you can use it with grids, charts, or any other controls. The dates will look the same (UTC/GMT) on all machines regardless of time zone. I believe this is the right place to perform this conversion since this only matters when you persist the dates.

    I apologize if I am missing your point. If that is the case, please set me straight with a simple example that shows exactly what the problem is. You could post it here or send it to me by mail:

            bernardo.castilho@grapecity.com
    

    Thanks!

  • Posted 15 December 2017, 11:04 am EST

    Hello all

    OK, here’s a pretty long discussion of this issue. I hope you will find it interesting:

    JavaScript Date objects are based on a value that represents the number of milliseconds since 1/1/1970 GMT. This value is independent of time zone.

    However, most methods in the Date class do take the time zone into account. They do this by adding a number of minutes to the date. This value depends on the machine and can be retrieved using the date’s getTimezoneOffset() method. This can be confusing because the value returned by this method depends on the location and also on whether on daylight savings apply to the date in the current location.

    This can lead to strange results when serializing dates as strings and de-serializing in different time zones.

    For example, assume you have serialized a date as “1948-12-08T00:00:00Z”. This means Dec 8, 1948, in Greenwich Mean Time (GMT).

    If you de-serialize this date by calling “new Date(s)”, you will get back the same date. But depending on where you are, it may look different. For example:

    var dt = new Date("1948-12-08T00:00:00Z");
    console.log(dt.toString())
       "Tue Dec 07 1948 22:00:00 GMT-0200 (E. South America Daylight Time)"
    
    

    Dec 8 at midnight in Greenwich is Dec 7 at 10pm in South America. There’s a two-hour difference taking into account daylight savings time.

    The dates are stored correctly, but JavaScript will format them using local time. If you are in a time zone with a negative offset (West of Grennwich), all dates formatted this way will look like the previous day (subtract the time zone offset from midnight and you will get the previous day).

    Also, values stored as pure dates (midnight-based) will now have a time part.

    Summarizing, all JavaScript dates are based on a GMT offset value. But they are converted to appear like local dates which means they will look like different values depending on the time zone settings on your machine.

    We rencently added a showDatesAsGmt property to the ODataCollectionView. This property causes the ODataCollectionView to convert dates read from the database so they look like GMT dates.

    Specifically, setting this new property to true causes the ODataCollectionView to add the local time zone offset to dates being read from the database and to subtract it when writing. This way, when dates are formatted to the local time zone they look like the original GMT values.

    For example, the employee list stored in the public OData service at

    'http://services.odata.org/V4/Northwind/Northwind.svc'
    

    Looks like this when loaded on a machine in South America:

    Name		Birth Date		Hire Date
    Davolio		1948/12/07 22:00:00	1992/04/30 21:00:00
    Fuller		1952/02/18 21:00:00	1992/08/13 21:00:00
    Leverling	1963/08/29 21:00:00	1992/03/31 21:00:00
    Peacock		1937/09/18 21:00:00	1993/05/02 21:00:00
    Buchanan	1955/03/03 21:00:00	1993/10/16 21:00:00
    Suyama		1963/07/01 21:00:00	1993/10/16 21:00:00
    King		1960/05/28 21:00:00	1994/01/01 22:00:00
    Callahan	1958/01/08 22:00:00	1994/03/04 21:00:00
    Dodsworth	1966/01/26 22:00:00	1994/11/14 22:00:00
    
    

    Notice how the dates also have time parts, and they are not all the same since some of the dates falls into a daylight savings period.

    If you try to filter on these dates by date alone (e.g. look for ‘1948/12/07’) you will not find anything, because the time part will be missing from the filter.

    Loading the same data on a machine in the US or in Japan would show different date values.

    If you set the showDatesAsGmt property to true, the data will look like this:

    Name		Birth Date		Hire Date
    Davolio		1948/12/08 00:00:00	1992/05/01 00:00:00
    Fuller		1952/02/19 00:00:00	1992/08/14 00:00:00
    Leverling	1963/08/30 00:00:00	1992/04/01 00:00:00
    Peacock		1937/09/19 00:00:00	1993/05/03 00:00:00
    Buchanan	1955/03/04 00:00:00	1993/10/17 00:00:00
    Suyama		1963/07/02 00:00:00	1993/10/17 00:00:00
    King		1960/05/29 00:00:00	1994/01/02 00:00:00
    Callahan	1958/01/09 00:00:00	1994/03/05 00:00:00
    Dodsworth	1966/01/27 00:00:00	1994/11/15 00:00:00
    
    

    The dates are now displayed as GMT, regardless of location.

    Presumably, these values are more meaningful. Nancy Davolio’s birthday is on December 8, regardless of where you are.

    I hope this makes sense, and that you find the new showDatesAsGmt property useful.

  • Posted 19 December 2017, 12:01 am EST

    Bernardo

    As you have pointed out in previous posts, not here, that JSON serializing does a poor job with dates. Devs, like me, must use a date format that is consistent. As I don’t do coding for the USA the date format is not ‘mm/dd/yyyy’. So we must use another JSON compatible date format that is also compatible with UI tools. UTC format fits that requirement.

    What you are asking use to do is convert that date string into a Date object for every Date cell.

    It is likely that we will have millions of Date cells.

    This is more so if we wish to get a “free lunch” with Filtering/Sorting/Grouping by downloading all the rows. My largest rowset is 250k+, with 20-30 columns; of which 10+ can be date columns

    The fear/dread hear is performance drag by updating all those cells.

    We have to do it twice: when loading the data FROM the server AND when sending rows TO the server.

    If we choose paging then we don’t get that “free Lunch” with Filtering/Sorting/Grouping that I mentioned. We have to convert Flexgrid’s Filtering/Sorting/Grouping JSON objects into our models, before sending that JSON to the server as well.

    This is not easily done as there is no Docs on the values used in the FlexGrid JSON properties. And Wjimo could change how these values work or what their purpose is; ie no interface contract for downstream developers

    So it is the cost of lagging performance because of LARGE conversion of dates, which the user will notice; or the cost of converting Flexgrid’s Filtering/Sorting/Grouping JSON objects into our models so they are part of the paged query at the server.

    Or

    Wijmo picks up the string in a Date Column when it needs to do a calculation for Flexgrid’s Filtering/Sorting/Grouping, converts it to a date; does the calculation, emits the new set.

    No need to change the data.

    BTW:

    the primitive value of the Date object IS a date since midnight of January 1, 1970 GMT expressed in milliseconds and can be obtained with the “valueOf()” method.

  • Posted 19 December 2017, 9:36 am EST

    Hello Greg!

    As you have pointed out in previous posts, not here, that JSON serializing does a poor job with dates. Devs, like me, must use a date format that is consistent. As I don’t do coding for the USA the date format is not ‘mm/dd/yyyy’. So we must use another JSON compatible date format that is also compatible with UI tools. UTC format fits that requirement.

    Right. The Date.toJSON native method does serialize the date in UTC format. It is entirely equivalent to saving the number of milliseconds since Jan 1, 1970 UTC as a string. And the JSON string can be converted back into a date by passing it as an argument to the date constructor:

    var jsonDate = (new Date()).toJSON();
    var backToDate = new Date(jsonDate);
    console.log(jsonDate); //2015-10-26T07:46:36.611Z ('Z' means UTC)
    
    

    What you are asking us to do is convert that date string into a Date object for every Date cell. It is likely that we will have millions of Date cells.

    Actually, I am suggesting rather than asking ;-). You could keep the dates as strings if you wanted to. But if you want the grid (or any other element of your app) to interpret those strings as dates (and sort/format/etc), that’s the only way to do it as far as I know.

    The conversion is quite fast compared to the time required to download the strings from the server or to upload them later. I really wouldn’t worry about it. Plus, dates take up less storage than the equivalent strings.

    This is more so if we wish to get a “free lunch” with Filtering/Sorting/Grouping by downloading all the rows. My largest rowset is 250k+, with 20-30 columns; of which 10+ can be date columns. The fear/dread hear is performance drag by updating all those cells.

    Fear not! As I said, it will take far longer to download those 250k items than to convert their dates from strings to actual dates. We have tons of customers who deal with large data sets, and the time required to revive JSON objects (including dates) has never been an issue.

    If we choose paging then we don’t get that “free Lunch” with Filtering/Sorting/Grouping that I mentioned. We have to convert Flexgrid’s Filtering/Sorting/Grouping JSON objects into our models, before sending that JSON to the server as well. This is not easily done as there is no Docs on the values used in the FlexGrid JSON properties. And Wjimo could change how these values work or what their purpose is; ie no interface contract for downstream developers

    This is really a separate issue. Each server has its own format for expressing filters. The Wijmo format is documented in the wijmo.grid.filter class topic. Each column filter may have a value or a condition filter. Each condition filter may have up to two conditions, specified as an operator and a value. It’s all fully documented. For example, the condition operators are listed here:

    http://demos.wijmo.com/5/Angular/WijmoHelp/WijmoHelp/topic/wijmo.grid.filter.Operator.Enum.html

    You can see examples of how to convert the Wijmo filter format into server filter definitions in our ODataCollectionView source as well as in our ServerCollectionView sample.

    OR Wijmo picks up the string in a Date Column when it needs to do a calculation for Flexgrid’s Filtering/Sorting/Grouping, converts it to a date; does the calculation, emits the new set. No need to change the data.

    You wrote: ‘Wijmo picks up the string in a DateColumn, …, converts it to a date…’

    That is essentially what we do now, except we convert it do a date only once, when the data is read. What you are suggesting would require a LOT of conversions, including every time we wanted to show a date in a grid cell. Believe me, it would not be a good trade-off.

  • Posted 25 October 2018, 10:01 am EST

    Is there a bug in the way that the MVC component is rendering dates with daylight savings? If I use the MVC code to create a Flex Grid where I bind the model, then the day is one day off. If I create the grid clientside js with wijmo and parse the date, it shows correctly. What to do? This is on 2018v3, days after 11/4 show day prior.

  • Posted 26 October 2018, 8:08 am EST

    Hi Matthew,

    Your original issue reported at MVC forum has been replied:

    https://www.grapecity.com/en/forums/mvc-edition/daylight-savings-bug-in-mv

    ~Manish

Need extra support?

Upgrade your support plan and get personal unlimited phone support with our customer engagement team

Learn More

Forum Channels