ics import problems due to malformed VTIMEZONE definition

Created on 6 December 2021, almost 3 years ago
Updated 30 June 2023, over 1 year ago

Problem/Motivation

NOTE: Both of the following problems have been confirmed to originate from the same root cause:

  • .ics files generated from events cannot be imported into the latest version of Outlook for Mac
  • .ics files generated from events and imported into Outlook for Windows are imported with a start and end time one hour later than the actual event

The iCalendar spec describes the required VTIMEZONE calendar component.

From https://icalendar.org/iCalendar-RFC-5545/3-6-5-time-zone-component.html:

The "VTIMEZONE" calendar component MUST include the "TZID" property and at least one definition of a "STANDARD" or "DAYLIGHT" sub-component. The "STANDARD" or "DAYLIGHT" sub-component MUST include the "DTSTART", "TZOFFSETFROM", and "TZOFFSETTO" properties.

An individual "VTIMEZONE" calendar component MUST be specified for each unique "TZID" parameter value specified in the iCalendar object. In addition, a "VTIMEZONE" calendar component, referred to by a recurring calendar component, MUST provide valid time zone information for all recurrence instances.

Per that spec, the VTIMEZONE calendar component currently being generated for .ics downloads by the addtocal_augment contrib module is missing the required DTSTART attribute on the STANDARD sub-component, as shown in this example:

BEGIN:VTIMEZONE
TZID:America/Chicago
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0500
END:STANDARD
END:VTIMEZONE

Different calendar clients respond differently when attempting to import an ics file with this malformed VTIMEZONE component.

  • Some import with no problem (Google Calendar, Outlook Web Access)
  • Some import with a one hour offset on the time, due to missing daylight savings time definition (Outlook for Windows)
  • Some fail to import, with an undefined error (Outlook for Mac)

Steps to reproduce

  1. Using a Mac, click "Add to Calendar" on a event in a UTDK site and select the "Outlook" format
  2. Drag the download event.ics file onto the Outlook icon in the Dock
  3. Confirm "Import" in the "Add Event" dialog
  4. Should receive an error message that says "Sorry we couldn’t import some of your file(s). Please try again later."
  1. Using a Windows computer, click "Add to Calendar" on a event in a UTDK site and select the "Outlook" format
  2. Confirm the import into Outlook for Windows
  3. The event should display in Outlook with a start and end time one hour later than the time displayed on the website

Proposed resolution

Remove the VTIMEZONE component from the generated ics file altogether, and express the DTSTART and DTEND components in UTC.

Example:

BEGIN:VCALENDAR
PRODID:Example site name
VERSION:2.0
BEGIN:VEVENT
UID:2adaf962-ab23-4e97-b38d-9984edd1036b
SUMMARY:Example event title
DTSTAMP:20230503T081300Z
DTSTART:20230510T180000Z
DTEND:20230510T190000Z
DESCRIPTION:Example event description
LOCATION:Example event location
END:VEVENT
END:VCALENDAR

This removes the responsibility for constructing a valid VTIMEZONE component from the module, produces a valid ics file, and allows the calendar client importing the ics file to display the event in the selected time zone of the user.

This approach is similar to that used by all of the major calendar programs -- Google Calendar, Microsoft Outlook, and Apple Calendar -- in their ics exports. Regardless of the user's timezone, events from those calendar systems are exported with the start and end time expressed in UTC.

Remaining tasks

User interface changes

API changes

Data model changes

πŸ› Bug report
Status

Fixed

Version

1.1

Component

Code

Created by

πŸ‡ΊπŸ‡ΈUnited States texas_tater Austin, TX

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • πŸ‡ΊπŸ‡ΈUnited States gravelpot

    @mandclu Happy new year! Just bumping this to see if you've had time to consider whether you would accept another developer from our team here at the University of Texas at Austin testing and marking this issue as RTBC, as long as we affirm that the review is being done independently?

  • πŸ‡¨πŸ‡¦Canada mandclu

    Allow me to recount a personal anecdote that hopefully will shed light on the revised approach I'm proposing here. In early November, I joined a local photography club, and used an add-to-calendar link on their website to add the weekly meetings to my Google Calendar. Since daylight savings ended, every single meeting is off by an hour, so each week I have to manually update the meeting in my own calendar. I relate this experience because it illustrates that it isn't sufficient to publish times in UTC and trust the calendar software to get it right. Particularly for recurring events that can move in and out of daylight savings time, knowing the proper timezone is invaluable, when this can be communicated without introducing other problems.

    As such, I have decided that since the problem reported in this issue is really with Outlook, it isn't advisable to remove the timezone information for Google Calendars, which do support this information. I have provided an updated patch (sorry no interdiff today) that creates a separate set of data for Outlook, specifically excluding the timezone and rendering the data in UTC.

    Apologies also for including some code standards fixes in with the changes. I know this isn't best practice but wanted to get them included while I'm editing the files.

    Let me know if this updated patch still resolves the original issue, and I'll get it merged in.

  • πŸ‡ΊπŸ‡ΈUnited States gravelpot

    @mandclu Thanks for the update, I'll try to get some time to look at the new patch this week.

    Would you indulge me in sharing the URL to the calendar for your photography club? Assuming they haven't updated it since you encountered that problem, I would like to look at their ICS file structure to better understand why your calendar behaved the way it did.

  • πŸ‡¨πŸ‡¦Canada mandclu

    I think I originally added their event to my calendar on my phone, so it would be tricky to find the original file. Here's a event export I created today. I had to add a txt extension so drupal.org would let me upload it.

  • Status changed to Needs work over 1 year ago
  • πŸ‡ΊπŸ‡ΈUnited States mark_fullmer Tucson

    it isn't advisable to remove the timezone information for non-Outlook clients, which do support this information. I have provided an updated patch that creates a separate set of data for Outlook, specifically excluding the timezone and rendering the data in UTC.

    I'm in support in principle of the approach above. However, I'm not getting the expected output with patch #24.

    Additionally, when I started to set about validating the output through the existing Unit test, I got an error:

    1) Drupal\Tests\addtocal_augment\Unit\BasicLinkTest::testCal
    Drupal\Core\DependencyInjection\ContainerNotInitializedException: \Drupal::$container is not initialized yet. \Drupal::setContainer() must be called with a real container.
    

    Setting this to "Needs work", with the following remaining work to be done:

    1. Establish that the Outlook variant does in fact excluding the timezone information and the iCal variant has no change.
    2. Remediate the failing test above.
    3. Add test coverage for the Outlook variant.

  • Status changed to Needs review over 1 year ago
  • πŸ‡ΊπŸ‡ΈUnited States mark_fullmer Tucson

    The attached patch restores the Unit test functioning and adds test coverage for the Outlook variants. Only a small change was made to the business logic, setting the $separator as a class variable so that it could be used in the test.

    Setting to "Needs Review"!

  • πŸ‡ΊπŸ‡ΈUnited States gravelpot

    Hi @mandclu, just checking in to see if you would be willing to look at the newer patches above from @mark_fullmer? Thanks!

  • πŸ‡¨πŸ‡¦Canada mandclu

    Thanks for everyone's work (and patience) here.

    Generally the patch in #28 looks good, but there is one thing I'm not sure I understand about the updated code. In the updated BasicLinkTest :: testCal for both the ical and outlook tests, it does an implodeRecursive() using the addtocal separator and then does an explode on the result for the comparison. Is that to flatten the generated array?

  • πŸ‡ΊπŸ‡ΈUnited States mark_fullmer Tucson

    Yes, that scansion of the code intent in the new test coverage is correct. The reason I did this was in order to as faithfully reproduce the way that the actual render proceeds (by doing the implodeRecursive() with its separator) while being able to have a test expectation compare against an actual output as a simple array.

    In my testing, comparing two arrays in the test made it much simpler to see what, if anything, was dissimilar about the output.

    • mandclu β†’ committed 61dc19ed on 1.1.x
      Issue #3252918 by mmarler, mandclu, mark_fullmer, sanduhrs, BeauTownsend...
  • πŸ‡¨πŸ‡¦Canada mandclu

    Merged in and will roll a new release shortly. Thanks again!

  • Status changed to Fixed over 1 year ago
  • Automatically closed - issue fixed for 2 weeks with no activity.

  • I tried the 1.1.0-beta1 version and the issue is persisting for me where the Outlook link does not open in my Outlook desktop app. The link for Google works and the iCal link opens in the default mail app for Mac.

    I am on a Mac Apple M1, on Ventura and Outlook version 16.73.

  • πŸ‡ΊπŸ‡ΈUnited States mark_fullmer Tucson

    I tried the 1.1.0-beta1 version and the issue is persisting for me where the Outlook link does not open in my Outlook desktop app.

    I can confirm that this is the case. It's due to missing colons on the DSTART and DTEND values. I'll create a new issue.

  • πŸ‡ΊπŸ‡ΈUnited States mark_fullmer Tucson

    I've created a new issue and provided a patch at πŸ› [Outlook] DTSTART and DTEND values missing colon Fixed

Production build 0.71.5 2024