Provide authors with tools to manage transcripts and captions/subtitles for local video and audio

Background

One thing I'd like to look at soon is addressing WCAG and ATAG for audio and video media types. Basically, authors should be able to provide captions and transcripts. The ATAG principle is that authoring tools should help to produce accessible alternatives for non-text content.

When core didn't have audio/video, it meant this wasn't in scope to address the WCAG and ATAG criteria.

Back in Vienna Drupalcon, at the media initative update, I asked about how to manage transcripts and captions. The answer was something like "media is just a framework, so it's up to site builders to figure out how audio/video transcripts will be managed and presented". In the case of 3rd party hosted media platforms, some already provide means to manage captions on the hosting platform, and individual media type plugins would need to address the individual platform APIs when rendering embedded content in Drupal.

That's fine so long as we just see media as a bunch of framework modules, and leave the accessibility plan up to site builders.

Current problem

But since then, the standard install profile has added pre-configured media entity bundles for local audio, and local video, in #2934962: Ship local audio and video media types in Standard → . This brings the WCAG and ATAG criteria into scope for the Drupal core accessibility gate. There's currently no way for an author to provide text alternatives for local media, and that's failing at level-A.

Proposed Resolution

We should figure out how to manage transcripts, captions, and subtitles, for LOCAL audio + video media entities.

A minimum viable product for managing and publishing captions and transcripts could be this:

Support Transcripts

1. ATAG: Add a transcript field to local audio and video media bundles in standard profile. A long text, formatted, permitting HTML.
2. WCAG: Render the transcript field in manage display. Transcripts can be long, so it's probably worth using a HTML summary/details element to wrap this, and place it after the the audio/video element itself. This will be fairly unintrusive design-wise. We'll need to create a field formatter plugin for this, which can be a general purpose formatter suitable for any text field.
   ✨ Create a details/summary FieldFormatter plugin for text fields. Needs work
3. WCAG: An additional approach to making the transcript available is to render the word "transcript" as a link, pointing to the /media/% page where the full transcript can be viewed. This suits some situations where videos and audio entities are embedded in a broader narrative article, say.
4. ATAG: add transcripts advice for authors in hook_help() and/or the new help_topics module.

Support Captions, Subtitles (and other track kinds)

1. ATAG: Add a Media Track field type. This will be a compound field type, comprising a file upload, and some additional properties for the HTML <track> attributes.
   • For this MVP plan, we will only support WebVTT files, and assume authors know how to export (or convert) this format with their caption editor tools. That's because core only supports the native HTML <video> and <audio> elements, which in turn only support WebVTT format for tracks.
   • The expectation is that authors will prepare tracks using a caption-editor application, then upload a WebVTT format text file. (Clarified in comments #13-15.)
   • Media track field instances will usually be multi-value, so authors can provide several caption tracks in different languages.
   • Bonus: it should be trivial to support the other <track> attribute values specified by the HTML5 recommendation. This just means ensuring that authors can specify all of the srclang, kind, label, default attributes. Browsers don't support all kind values yet (like audio descriptions and chapters) but some 3rd-party JS video players do.
   • Task issue - 📌 Provide a FieldType and FieldWidget to represent a WebVTT file for the HTML track element Needs work .
2. WCAG: Render the captions using a HTML <track> element nested inside the <audio> and <video> elements.
   • We're delegating it to browsers to display the tracks, including the means of turning them on/off. See comment #12 for an overview of browser support.
   • Update the existing video field formatter, so it has nested track elements. This part sounds is slightly awkward, because one field formatter is including information from another field. The proposed way to do this is to let a video formatter settings point to a media track field. Task issue - 📌 Update audio and video file field formatters to include track elements Needs work .
   • Likewise, update the existing audio field formatter to support tracks the same way. Much code can be shared in FileMediaFormatterBase, so we can do it in the same task issue as the video formatter.
3. ATAG: Keep in mind that supporting other caption file formats (like SRT) is a highly desirable follow-up for sites which use various JS-based video players, via other field formatter plugins from contrib. So if we can keep the track field type flexible enough for different caption mime-types, so much the better. Assuming the new Media Track FieldType extends the default file type, then site builders can adjust the acceptable file extensions; .vtt will just be the default.
4. ATAG: add advice about track support for authors,
   • in hook_help()
   • via the new help_topics module.
   • in the drupal.org handbook

Remaining tasks

See child issues for the actual implementation details.

Useful resources

• DrupalCon Vienna 2017: State of the media initiative. Captions and transcript discussed in the audience questions near the end.
• TODO: links to WCAG and ATAG criteria + techniques
• Authoring Tool Accessibility Guidelines (ATAG) Overview
• Implementing Principle B.2: Authors are supported in producing accessible content
• W3C Accessible Media tutorials...
• Transcripts
• Producing Captions
• Differences between subtitles and captions - Subtitles are just the speech, captions are speech + info about other sounds like the fact music is playing.
• https://www.w3.org/2008/06/video-notes
• Able Player is 3rd party JS media player plugin. It's the state of the art for accessibility, with extensive support for displaying tracks in a variety of ways.