Better handling of timezones for relative default date + times

Created on 9 September 2020, over 4 years ago
Updated 3 May 2024, 8 months ago

Problem/Motivation

Since support was added to core for relative default dates, core seems to support relative dates + times as well. There is just a problem with the timezone handling.

Steps to reproduce

Set your site to a another time zone as GMT (for example GMT+8) and create a date field on a content type with a relative default date and time (+1 Saturday 13:00). When adding content, the default date might be correct, but the time has "updated" to the timezone (21:00).

Proposed resolution

Looks like there might be some timezone overcompensation. Patch attached

Remaining tasks

User interface changes

API changes

Changes to datetime & datetime_range might be necessary.

Data model changes

Release notes snippet

🐛 Bug report
Status

Active

Version

11.0 🔥

Component
Datetime 

Last updated 6 days ago

Created by

🇨🇳China splash112

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • First commit to issue fork.
  • Pipeline finished with Success
    8 months ago
    Total: 501s
    #163001
  • 🇳🇿New Zealand ericgsmith

    Converted patch to MR - curious to see what the tests will show.

    I'm not sure the possible side effects of the change, but for me it doesn't make sense to use the storage timezone for relative dates in the UI - its confusing and limits some good meaning defaults we can have.

    Found this issue after what seemed like a very simple request from a client "we want the default value to be the next day at 2pm" - can't do it as even doing hours relative to UTC is impacted by timezone changes.

    A relative value of "tomorrow 2pm" works with this patch.

  • Status changed to Needs work 8 months ago
  • 🇳🇿New Zealand ericgsmith

    Not tests fail with this change - interesting!

    Looks like DateTimeFieldTest::testDefaultValue only tests for date only fields which have not changed here - it will need some test coverage for datetime.

  • Pipeline finished with Failed
    4 months ago
    Total: 227s
    #290885
  • Pipeline finished with Success
    4 months ago
    Total: 1147s
    #290888
  • 🇳🇿New Zealand ericgsmith

    I have expanded DateTimeFieldTest::testDefaultValue to also test using a datetime field.

    I note that there is currently no coverage in the daterange test for timezones or using the datetime type - I am assuming this will need to also get coverage to change - but setting to needs review to get feedback on this change in general.

  • Pipeline finished with Failed
    4 months ago
    Total: 325s
    #292826
  • 🇳🇿New Zealand ericgsmith

    Adding related issue #2778083: Default value for Date-Only fields is broken when UTC date is different than user current date - this is where the bug was fixed for date only fields - it is not clear in that issue why datetime was excluded from this fix

  • Pipeline finished with Failed
    4 months ago
    #292851
  • Pipeline finished with Canceled
    4 months ago
    Total: 68s
    #292852
  • Pipeline finished with Success
    4 months ago
    #292853
  • 🇳🇿New Zealand ericgsmith

    Addressed MR feedback and expanded range test to match date time field.

  • 🇺🇸United States smustgrave

    believe feedback for this one has been addressed.

  • The Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

    This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

    Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

  • First commit to issue fork.
  • 🇯🇵Japan ptmkenny

    I added the missing return type as suggested in #20, but I can't determine why the unit tests are now failing.

  • 🇯🇵Japan ptmkenny

    It looks like the test failure was temporary, as the tests are now passing again. I think it's also highly unlikely that adding a return type of array to a function that always returned an array will break anything, so I'm setting this back to RTBC.

  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    This bug feels tricky... it you set the default relative date to +1 Saturday 13:00 whose timezone does the 13:00 apply to? The user who is creating the content - the site? I agree that there is a bug here because UTC is not the correct choice for default values. But I'm not sure that user's timezone is either. I think I would expect the site's timezone to be respected. So no matter by whom the content is created the default time is the same. Tricky. Note that if user timezones are not configurable I think this is exactly what this MR delivers.

  • 🇩🇪Germany rkoller Nürnberg, Germany

    From a users perspective this is super confusing. i've tested after reading #25 🐛 Better handling of timezones for relative default date + times Active . My setup:

    admin/config/regional/settings has New York as the default time zone
    user/1/edit has Berlin as the time zone in the local settings
    user/2/Edit has New York as the default time in the local settings
    +1 Saturday 13:00 is the relative default value set on the date field

    Without the MR on the node edit form (creator user 1) :
    11/16/2024 2pm (user 1 with berlin)
    11/16/2024 8am (user 2 with new york)

    with the MR on the node edit form (creator user 1)
    11/16/2024 1pm (user 1 with berlin)
    11/16/2024 7am (user 2 with new york)

    I receive the same set of dates and times if i create the node with user 2 same as if i create the field and set the relative date with user 2. that leaves me as the user puzzled and completely in lack of situational awareness about what the actual point of reference is when i am setting the relative default value on the field? my main assumption would have been that the point of reference date and time would be new york, the sites timezone. but with +1 saturday 13:00 against new york i wouldnt expected to get 11/16/2024 7am (with MR) 11/16/2024 8am (without MR); for user 2 i would have expected those time and for user 1 i would have expected 7pm (with MR) and 8pm (without MR) but not the other way around?

    I think one of the most important things here is providing the actual point of reference for the calculations on the description of the "relative default value" field on the field settings to provide some situational awareness to the user. at the moment the description only provides

    Describe a time by reference to the current day, like '+90 days' (90 days from the day the field is created) or '+1 Saturday' (the next Saturday). See strtotime for more details.

    there is no clue about the point of reference the calculations are based on? personally, as a user, i am completely lost here and feel highly confused.

  • 🇩🇪Germany rkoller Nürnberg, Germany

    might be also a more than suitable issue to discuss on fridays ux meetings imho. shall i add it to the shortlist?

  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    @rkoller thanks for the fantastic testing and yes I think discussing as a group is the best way forward.

  • 🇩🇪Germany rkoller Nürnberg, Germany

    i've added the issue to the shortlist yesterday and maybe we find time already today discussing it. But I did some more testing and the setup i've did yesterday was missing an important aspect - it matters who creates the issue. The matter is still puzzling, well time zones in general, but at least things became a bit more clear now. The revised setup:

    admin/config/regional/settings has New York as the default time zone
    user/1/edit (rkoller) has Berlin as the time zone in the local settings
    user/2/Edit (admin) has New York as the default time in the local settings
    +2 Saturday 13:00 is the relative default value set on the date field

    No MR applied

    Node creator rkoller - berlin

    Node edit form:
    11/23/2024 02:00:00pm

    Node:
    Sat, 23 Nov 2024 - 14:00 (rkoller berlin)
    Sat, 23 Nov 2024 - 08:00 (admin new york)

    Node creator admin - new york

    Node edit form:
    11/23/2024 08:00:00am

    Node:
    Sat, 23 Nov 2024 - 14:00 (rkoller berlin)
    Sat, 23 Nov 2024 - 08:00 (admin new york)

    MR applied

    node creator rkoller - berlin

    Node edit form:
    11/23/2024 01:00:00pm

    Node:
    Sat, 23 Nov 2024 - 13:00 (rkoller berlin)
    Sat, 23 Nov 2024 - 07:00 (admin new york)

    node creator admin - new york

    Node edit form:
    11/23/2024 01:00:00pm

    Node:
    Sat, 23 Nov 2024 - 19:00 (rkoller berlin)
    Sat, 23 Nov 2024 - 13:00 (admin new york)

  • 🇩🇪Germany rkoller Nürnberg, Germany

    i've added the issue to the shortlist yesterday and maybe we find time already today discussing it. But I did some more testing and the setup i've did yesterday was missing an important aspect - it matters who creates the issue. The matter is still puzzling, well time zones in general, but at least things became a bit more clear now. The revised setup:

    admin/config/regional/settings has New York as the default time zone
    user/1/edit (rkoller) has Berlin as the time zone in the local settings
    user/2/Edit (admin) has New York as the default time in the local settings
    +2 Saturday 13:00 is the relative default value set on the date field

    No MR applied

    Node creator rkoller - berlin

    Node edit form:
    11/23/2024 02:00:00pm

    Node:
    Sat, 23 Nov 2024 - 14:00 (rkoller berlin)
    Sat, 23 Nov 2024 - 08:00 (admin new york)

    Node creator admin - new york

    Node edit form:
    11/23/2024 08:00:00am

    Node:
    Sat, 23 Nov 2024 - 14:00 (rkoller berlin)
    Sat, 23 Nov 2024 - 08:00 (admin new york)

    MR applied

    node creator rkoller - berlin

    Node edit form:
    11/23/2024 01:00:00pm

    Node:
    Sat, 23 Nov 2024 - 13:00 (rkoller berlin)
    Sat, 23 Nov 2024 - 07:00 (admin new york)

    node creator admin - new york

    Node edit form:
    11/23/2024 01:00:00pm

    Node:
    Sat, 23 Nov 2024 - 19:00 (rkoller berlin)
    Sat, 23 Nov 2024 - 13:00 (admin new york)

  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    I think the MR behaviour is way better than HEAD because it is consistent. On the node edit form the time being saved is in the users timezone which makes sense. And then when a user views it they see the correct time. I think the major UX issue is that we're not showing the timezone on the node edit form as that would be very very helpful for people because maybe the node creator rkoller is very aware the site's timezone is new york and would expect node edit forms to be in the site timezone and not theirs.

  • 🇮🇳India sagarmohite0031

    Hello,
    MR applied successfully attaching before and after screenshots.
    Please check attachments

  • Status changed to Needs review about 1 month ago
  • 🇩🇪Germany rkoller Nürnberg, Germany

    Usability review

    We discussed this issue at 📌 Drupal Usability Meeting 2024-11-15 Active . The direct link to the recording of the meeting is https://www.youtube.com/watch?v=Yy18FL7GuvE. For the record, the attendees at the usability meeting were @AaronMcHale, @benjifisher, @rkoller, and @simohell.

    First we went through the tests outlined in #29 , comparing the behavior of the relative default value field with and without the MR applied. We asked ourselves if the root cause for the odd behavior is the default value or how the field is getting interpreted, so we went ahead and did a few more experiments. First we tried adding a timezone to the relative default value field (which is not directly apparent) and noticed a few problems along the way:

    Entering +2 Saturday 13:00 America/New York lead to the following error: The relative date value entered is invalid. Although the error message explains what went wrong, but at the moment it does not provide any solution how to resolve the invalid date value error - we have tried different variants of lower and upper case lettering for “New York” but no luck. In our explorations following up after the meeting we’ve realized that the timezone does not allow any spaces, you have to use an underscore, like +2 Saturday 13:00 America/New York, to be recognized as a valid date value.

    We’ve noticed another detail about the capitalization of timezones. +2 Saturday 13:00 GMT+5, +2 Saturday 13:00 gmt+5, +2 Saturday 13:00 Gmt+5, +2 Saturday 13:00 UTC+5, +2 Saturday 13:00 utc+5, or +2 Saturday 13:00 Utc+5 are all recognized as valid when then field settings are saved. Problem is if you go to the corresponding node edit forms that are using the field with those timezones on the relative default values, the set default times are only shown for the upper case variants UTC and GMT, all other cases like gmt+5, Gmt+5, utc+5,and Utc+5, lead to empty fields only showing a placeholder value instead:

    We then followed the strtotime-link in the description for the Relative default value field which lead to the documentation page https://www.php.net/manual/en/function.strtotime.php for that PHP-function. The page contains an explicit warning that the timestamp that this function returns does not contain any information about timezones.

    Now that time zones are implicitly and explicitly used in the Relative default value field, we wondered how date and time is actually stored in the database. Turns out the default relative date and time value is stored as a string (for example +1 Saturday 23:00 or +2 Saturday 13:00 UTC+1), and on the particular node it is also stored as a string (for example 2024-11-16T23:00:00). So for the field settings the timezone is being stored if added by the user, while the string stored on the node is not containing any information about the timezone.

    Due to subsequent discussions to the meeting in the #ux channel on the Drupal Slack, I’ve further expanded the test setup from #29 to better understand the actual behavior: https://gist.github.com/rpkoller/9b4c93c28d1d97ccec404194e277f404. The first markdown file is the scenario from #29 , for the second markdown file an explicit timezone (America/Los_Angeles) dissimilar to the site's and the user's was used, and the last three scenarios explicitly applied UTC and the timezones of the two users to the default relative date.
    It turns out that if no timezone is explicitly set for the default relative date, Drupal is using an implicit timezone. Without the MR applied, UTC is used, while with the MR applied, the timezone of the current user is used. So the relative default value is using a timezone all the time, either implicit and not shown to the user or explicit if set and therefore changed by the user.

    After getting an overview of the entire problem space, we came to the following initial conclusions:

    • The currently implicit timezone for the relative default value should be made explicit and actually shown (*we haven't discussed the details about how the field should behave and be presented yet, but one thought that came to my mind during finalizing this comment- if the user isn't entering and altering the default timezone the default timezone should be added to the relative default value on safe). The site builder needs to be aware of the point of reference, no matter if the MR will use in the end the site’s timezone or the user’s timezone - each of the two options has a reasonable use case. The user simply has to be aware THAT a timezone is used and which it actually is.
    • In the context of the timezone the problems listed above in regard of case and syntax should be tackled, to avoid those form errors as well as empty date fields on node edit forms that are missing the set relative default date and time.
    • The description for the relative default value field should also be extended, perhaps providing an example based on the timezone for the relative default value and providing one or two example about the actual values shown on the node edit form to illustrate the implications.

    In addition to that @benjifisher tried to get some advice from @mandlcu, as the maintainer of the smart date module, at Nedcamp and will add the outcome in a comment on this issue.

    If you want more feedback from the usability team, a good way to reach out is in the #ux channel in Slack.

  • 🇩🇪Germany rkoller Nürnberg, Germany

    Due to the additional research and discussion, and the complexity of the topic the write up of the comment took a little bit longer. But I agree with the point @alexpott meanwhile made in #31 , it is not only necessary to add the timezone to the default value on the field settings page, but also sort of required to add the point of reference aka the timezone to the node edit form as well. Otherwise the entry the user makes on the node edit form is just based on an assumption. That is also sort of in line with an article i’ve stumbled across a few days ago: https://simonwillison.net/2024/Nov/27/storing-times-for-human-events/. In the recommendation section the authors suggests to store the user’s intent time and the location/timezone. I suppose that suggestion is out of the scope for this issue but I consider it a more than reasonable one and would suggestion to open up a followup for it?

  • 🇺🇸United States smustgrave

    @alexpott based on #31? Is this a net gain enough to move forward?

Production build 0.71.5 2024