← 4.8.9 The track elementTable of contents4.8.11 The canvas element →
      1. 4.8.10 Media elements
        1. 4.8.10.1 Error codes
        2. 4.8.10.2 Location of the media resource
        3. 4.8.10.3 MIME types
        4. 4.8.10.4 Network states
        5. 4.8.10.5 Loading the media resource
        6. 4.8.10.6 Offsets into the media resource
        7. 4.8.10.7 Ready states
        8. 4.8.10.8 Playing the media resource
        9. 4.8.10.9 Seeking
        10. 4.8.10.10 Media resources with multiple media tracks
          1. 4.8.10.10.1 AudioTrackList and VideoTrackList objects
          2. 4.8.10.10.2 Selecting specific audio and video tracks declaratively
        11. 4.8.10.11 Synchronising multiple media elements
          1. 4.8.10.11.1 Introduction
          2. 4.8.10.11.2 Media controllers
          3. 4.8.10.11.3 Assigning a media controller declaratively
        12. 4.8.10.12 Timed text tracks
          1. 4.8.10.12.1 Text track model
          2. 4.8.10.12.2 Sourcing in-band text tracks
          3. 4.8.10.12.3 Sourcing out-of-band text tracks
          4. 4.8.10.12.4 Guidelines for exposing cues in various formats as text track cues
          5. 4.8.10.12.5 Text track API
          6. 4.8.10.12.6 Text tracks describing chapters
          7. 4.8.10.12.7 Event definitions
        13. 4.8.10.13 User interface
        14. 4.8.10.14 Time ranges
        15. 4.8.10.15 Event definitions
        16. 4.8.10.16 Event summary
        17. 4.8.10.17 Security and privacy considerations
        18. 4.8.10.18 Best practices for authors using media elements
        19. 4.8.10.19 Best practices for implementors of media elements

4.8.10 Media elements

Media elements (audio and video, in this specification) implement the following interface:

interface HTMLMediaElement : HTMLElement {

  // error state
  readonly attribute MediaError? error;

  // network state
           attribute DOMString src;
  readonly attribute DOMString currentSrc;
           attribute DOMString crossOrigin;
  const unsigned short NETWORK_EMPTY = 0;
  const unsigned short NETWORK_IDLE = 1;
  const unsigned short NETWORK_LOADING = 2;
  const unsigned short NETWORK_NO_SOURCE = 3;
  readonly attribute unsigned short networkState;
           attribute DOMString preload;
  readonly attribute TimeRanges buffered;
  void load();
  DOMString canPlayType(DOMString type);

  // ready state
  const unsigned short HAVE_NOTHING = 0;
  const unsigned short HAVE_METADATA = 1;
  const unsigned short HAVE_CURRENT_DATA = 2;
  const unsigned short HAVE_FUTURE_DATA = 3;
  const unsigned short HAVE_ENOUGH_DATA = 4;
  readonly attribute unsigned short readyState;
  readonly attribute boolean seeking;

  // playback state
           attribute double currentTime;
  readonly attribute unrestricted double duration;
  readonly attribute Date startDate;
  readonly attribute boolean paused;
           attribute double defaultPlaybackRate;
           attribute double playbackRate;
  readonly attribute TimeRanges played;
  readonly attribute TimeRanges seekable;
  readonly attribute boolean ended;
           attribute boolean autoplay;
           attribute boolean loop;
  void play();
  void pause();

  // media controller
           attribute DOMString mediaGroup;
           attribute MediaController? controller;

  // controls
           attribute boolean controls;
           attribute double volume;
           attribute boolean muted;
           attribute boolean defaultMuted;

  // tracks
  readonly attribute AudioTrackList audioTracks;
  readonly attribute VideoTrackList videoTracks;
  readonly attribute TextTrackList textTracks;
  TextTrack addTextTrack(DOMString kind, optional DOMString label, optional DOMString language);
};

The media element attributes, src, crossorigin, preload, autoplay, mediagroup, loop, muted, and controls, apply to all media elements. They are defined in this section.

Media elements are used to present audio data, or video and audio data, to the user. This is referred to as media data in this section, since this section applies equally to media elements for audio or for video. The term media resource is used to refer to the complete set of media data, e.g. the complete video file, or complete audio file.

A media resource can have multiple audio and video tracks. For the purposes of a media element, the video data of the media resource is only that of the currently selected track (if any) given by the element's videoTracks attribute, and the audio data of the media resource is the result of mixing all the currently enabled tracks (if any) given by the element's audioTracks attribute.

Both audio and video elements can be used for both audio and video. The main difference between the two is simply that the audio element has no playback area for visual content (such as video or captions), whereas the video element does.

Except where otherwise specified, the task source for all the tasks queued in this section and its subsections is the media element event task source.

4.8.10.1 Error codes
media . error

Returns a MediaError object representing the current error state of the element.

Returns null if there is no error.

All media elements have an associated error status, which records the last error the element encountered since its resource selection algorithm was last invoked. The error attribute, on getting, must return the MediaError object created for this last error, or null if there has not been an error.

interface MediaError {
  const unsigned short MEDIA_ERR_ABORTED = 1;
  const unsigned short MEDIA_ERR_NETWORK = 2;
  const unsigned short MEDIA_ERR_DECODE = 3;
  const unsigned short MEDIA_ERR_SRC_NOT_SUPPORTED = 4;
  readonly attribute unsigned short code;
};
media . error . code

Returns the current error's error code, from the list below.

The code attribute of a MediaError object must return the code for the error, which must be one of the following:

MEDIA_ERR_ABORTED (numeric value 1)
The fetching process for the media resource was aborted by the user agent at the user's request.
MEDIA_ERR_NETWORK (numeric value 2)
A network error of some description caused the user agent to stop fetching the media resource, after the resource was established to be usable.
MEDIA_ERR_DECODE (numeric value 3)
An error of some description occurred while decoding the media resource, after the resource was established to be usable.
MEDIA_ERR_SRC_NOT_SUPPORTED (numeric value 4)
The media resource indicated by the src attribute was not suitable.
4.8.10.2 Location of the media resource

The src content attribute on media elements gives the address of the media resource (video, audio) to show. The attribute, if present, must contain a valid non-empty URL potentially surrounded by spaces.

The crossorigin content attribute on media elements is a CORS settings attribute.

If a src attribute of a media element is set or changed, the user agent must invoke the media element's media element load algorithm. (Removing the src attribute does not do this, even if there are source elements present.)

The src IDL attribute on media elements must reflect the content attribute of the same name.

The crossOrigin IDL attribute must reflect the crossorigin content attribute.

media . currentSrc

Returns the address of the current media resource.

Returns the empty string when there is no media resource.

The currentSrc IDL attribute is initially the empty string. Its value is changed by the resource selection algorithm defined below.

There are two ways to specify a media resource, the src attribute, or source elements. The attribute overrides the elements.

4.8.10.3 MIME types

A media resource can be described in terms of its type, specifically a MIME type, in some cases with a codecs parameter. (Whether the codecs parameter is allowed or not depends on the MIME type.) [RFC4281]

Types are usually somewhat incomplete descriptions; for example "video/mpeg" doesn't say anything except what the container type is, and even a type like "video/mp4; codecs="avc1.42E01E, mp4a.40.2"" doesn't include information like the actual bitrate (only the maximum bitrate). Thus, given a type, a user agent can often only know whether it might be able to play media of that type (with varying levels of confidence), or whether it definitely cannot play media of that type.

A type that the user agent knows it cannot render is one that describes a resource that the user agent definitely does not support, for example because it doesn't recognize the container type, or it doesn't support the listed codecs.

The MIME type "application/octet-stream" with no parameters is never a type that the user agent knows it cannot render. User agents must treat that type as equivalent to the lack of any explicit Content-Type metadata when it is used to label a potential media resource.

"application/octet-stream" is special-cased here; if any parameter appears with it, it should be treated just like any other MIME type. This is a deviation from the rule that unknown MIME type parameters should be ignored.

media . canPlayType(type)

Returns the empty string (a negative response), "maybe", or "probably" based on how confident the user agent is that it can play media resources of the given type.

The canPlayType(type) method must return the empty string if type is a type that the user agent knows it cannot render or is the type "application/octet-stream"; it must return "probably" if the user agent is confident that the type represents a media resource that it can render if used in with this audio or video element; and it must return "maybe" otherwise. Implementors are encouraged to return "maybe" unless the type can be confidently established as being supported or not. Generally, a user agent should never return "probably" for a type that allows the codecs parameter if that parameter is not present.

This script tests to see if the user agent supports a (fictional) new format to dynamically decide whether to use a video element or a plugin:

<section id="video">
 <p><a href="playing-cats.nfv">Download video</a></p>
</section>
<script>
 var videoSection = document.getElementById('video');
 var videoElement = document.createElement('video');
 var support = videoElement.canPlayType('video/x-new-fictional-format;codecs="kittens,bunnies"');
 if (support != "probably" && "New Fictional Video Plugin" in navigator.plugins) {
   // not confident of browser support
   // but we have a plugin
   // so use plugin instead
   videoElement = document.createElement("embed");
 } else if (support == "") {
   // no support from browser and no plugin
   // do nothing
   videoElement = null;
 }
 if (videoElement) {
   while (videoSection.hasChildNodes())
     videoSection.removeChild(videoSection.firstChild);
   videoElement.setAttribute("src", "playing-cats.nfv");
   videoSection.appendChild(videoElement);
 }
</script>

The type attribute of the source element allows the user agent to avoid downloading resources that use formats it cannot render.

4.8.10.4 Network states
media . networkState

Returns the current state of network activity for the element, from the codes in the list below.

As media elements interact with the network, their current network activity is represented by the networkState attribute. On getting, it must return the current network state of the element, which must be one of the following values:

NETWORK_EMPTY (numeric value 0)
The element has not yet been initialized. All attributes are in their initial states.
NETWORK_IDLE (numeric value 1)
The element's resource selection algorithm is active and has selected a resource, but it is not actually using the network at this time.
NETWORK_LOADING (numeric value 2)
The user agent is actively trying to download data.
NETWORK_NO_SOURCE (numeric value 3)
The element's resource selection algorithm is active, but it has not yet found a resource to use.

The resource selection algorithm defined below describes exactly when the networkState attribute changes value and what events fire to indicate changes in this state.

4.8.10.5 Loading the media resource
media . load()

Causes the element to reset and start selecting and loading a new media resource from scratch.

All media elements have an autoplaying flag, which must begin in the true state, and a delaying-the-load-event flag, which must begin in the false state. While the delaying-the-load-event flag is true, the element must delay the load event of its document.

When the load() method on a media element is invoked, the user agent must run the media element load algorithm.

The media element load algorithm consists of the following steps.

  1. Abort any already-running instance of the resource selection algorithm for this element.

  2. If there are any tasks from the media element's media element event task source in one of the task queues, then remove those tasks.

    Basically, pending events and callbacks for the media element are discarded when the media element starts loading a new resource.

  3. If the media element's networkState is set to NETWORK_LOADING or NETWORK_IDLE, queue a task to fire a simple event named abort at the media element.

  4. If the media element's networkState is not set to NETWORK_EMPTY, then run these substeps:

    1. Queue a task to fire a simple event named emptied at the media element.

    2. If a fetching process is in progress for the media element, the user agent should stop it.

    3. Set the networkState attribute to NETWORK_EMPTY.

    4. Forget the media element's media-resource-specific text tracks.

    5. If readyState is not set to HAVE_NOTHING, then set it to that state.

    6. If the paused attribute is false, then set it to true.

    7. If seeking is true, set it to false.

    8. Set the current playback position to 0.

      Set the official playback position to 0.

      If this changed the official playback position, then queue a task to fire a simple event named timeupdate at the media element.

    9. Set the initial playback position to 0.

    10. Set the timeline offset to Not-a-Number (NaN).

    11. Update the duration attribute to Not-a-Number (NaN).

      The user agent will not fire a durationchange event for this particular change of the duration.

  5. Set the playbackRate attribute to the value of the defaultPlaybackRate attribute.

  6. Set the error attribute to null and the autoplaying flag to true.

  7. Invoke the media element's resource selection algorithm.

  8. Playback of any previously playing media resource for this element stops.

The resource selection algorithm for a media element is as follows. This algorithm is always invoked synchronously, but one of the first steps in the algorithm is to return and continue running the remaining steps asynchronously, meaning that it runs in the background with scripts and other tasks running in parallel. In addition, this algorithm interacts closely with the event loop mechanism; in particular, it has synchronous sections (which are triggered as part of the event loop algorithm). Steps in such sections are marked with ⌛.

  1. Set the networkState to NETWORK_NO_SOURCE.

  2. Asynchronously await a stable state, allowing the task that invoked this algorithm to continue. The synchronous section consists of all the remaining steps of this algorithm until the algorithm says the synchronous section has ended. (Steps in synchronous sections are marked with ⌛.)

  3. ⌛ If the media element has a src attribute, then let mode be attribute.

    ⌛ Otherwise, if the media element does not have a src attribute but has a source element child, then let mode be children and let candidate be the first such source element child in tree order.

    ⌛ Otherwise the media element has neither a src attribute nor a source element child: set the networkState to NETWORK_EMPTY, and abort these steps; the synchronous section ends.

  4. ⌛ Set the media element's delaying-the-load-event flag to true (this delays the load event), and set its networkState to NETWORK_LOADING.

  5. Queue a task to fire a simple event named loadstart at the media element.

  6. If mode is attribute, then run these substeps:

    1. Process candidate: If the src attribute's value is the empty string, then end the synchronous section, and jump down to the failed step below.

    2. ⌛ Let absolute URL be the absolute URL that would have resulted from resolving the URL specified by the src attribute's value relative to the media element when the src attribute was last changed.

    3. ⌛ If absolute URL was obtained successfully, set the currentSrc attribute to absolute URL.

    4. End the synchronous section, continuing the remaining steps asynchronously.

    5. If absolute URL was obtained successfully, run the resource fetch algorithm with absolute URL. If that algorithm returns without aborting this one, then the load failed.

    6. Failed: Reaching this step indicates that the media resource failed to load or that the given URL could not be resolved. In one atomic operation, run the following steps:

      1. Set the error attribute to a new MediaError object whose code attribute is set to MEDIA_ERR_SRC_NOT_SUPPORTED.

      2. Forget the media element's media-resource-specific text tracks.

      3. Set the element's networkState attribute to the NETWORK_NO_SOURCE value.

    7. Queue a task to fire a simple event named error at the media element.

    8. Set the element's delaying-the-load-event flag to false. This stops delaying the load event.

    9. Abort these steps. Until the load() method is invoked or the src attribute is changed, the element won't attempt to load another resource.

    Otherwise, the source elements will be used; run these substeps:

    1. ⌛ Let pointer be a position defined by two adjacent nodes in the media element's child list, treating the start of the list (before the first child in the list, if any) and end of the list (after the last child in the list, if any) as nodes in their own right. One node is the node before pointer, and the other node is the node after pointer. Initially, let pointer be the position between the candidate node and the next node, if there are any, or the end of the list, if it is the last node.

      As nodes are inserted and removed into the media element, pointer must be updated as follows:

      If a new node is inserted between the two nodes that define pointer
      Let pointer be the point between the node before pointer and the new node. In other words, insertions at pointer go after pointer.
      If the node before pointer is removed
      Let pointer be the point between the node after pointer and the node before the node after pointer. In other words, pointer doesn't move relative to the remaining nodes.
      If the node after pointer is removed
      Let pointer be the point between the node before pointer and the node after the node before pointer. Just as with the previous case, pointer doesn't move relative to the remaining nodes.

      Other changes don't affect pointer.

    2. Process candidate: If candidate does not have a src attribute, or if its src attribute's value is the empty string, then end the synchronous section, and jump down to the failed step below.

    3. ⌛ Let absolute URL be the absolute URL that would have resulted from resolving the URL specified by candidate's src attribute's value relative to the candidate when the src attribute was last changed.

    4. ⌛ If absolute URL was not obtained successfully, then end the synchronous section, and jump down to the failed step below.

    5. ⌛ If candidate has a type attribute whose value, when parsed as a MIME type (including any codecs described by the codecs parameter, for types that define that parameter), represents a type that the user agent knows it cannot render, then end the synchronous section, and jump down to the failed step below.

    6. ⌛ If candidate has a media attribute whose value does not match the environment, then end the synchronous section, and jump down to the failed step below.

    7. ⌛ Set the currentSrc attribute to absolute URL.

    8. End the synchronous section, continuing the remaining steps asynchronously.

    9. Run the resource fetch algorithm with absolute URL. If that algorithm returns without aborting this one, then the load failed.

    10. Failed: Queue a task to fire a simple event named error at the candidate element, in the context of the fetching process that was used to try to obtain candidate's corresponding media resource in the resource fetch algorithm.

    11. Asynchronously await a stable state. The synchronous section consists of all the remaining steps of this algorithm until the algorithm says the synchronous section has ended. (Steps in synchronous sections are marked with ⌛.)

    12. Forget the media element's media-resource-specific text tracks.

    13. Find next candidate: Let candidate be null.

    14. Search loop: If the node after pointer is the end of the list, then jump to the waiting step below.

    15. ⌛ If the node after pointer is a source element, let candidate be that element.

    16. ⌛ Advance pointer so that the node before pointer is now the node that was after pointer, and the node after pointer is the node after the node that used to be after pointer, if any.

    17. ⌛ If candidate is null, jump back to the search loop step. Otherwise, jump back to the process candidate step.

    18. Waiting: Set the element's networkState attribute to the NETWORK_NO_SOURCE value.

    19. ⌛ Set the element's delaying-the-load-event flag to false. This stops delaying the load event.

    20. End the synchronous section, continuing the remaining steps asynchronously.

    21. Wait until the node after pointer is a node other than the end of the list. (This step might wait forever.)

    22. Asynchronously await a stable state. The synchronous section consists of all the remaining steps of this algorithm until the algorithm says the synchronous section has ended. (Steps in synchronous sections are marked with ⌛.)

    23. ⌛ Set the element's delaying-the-load-event flag back to true (this delays the load event again, in case it hasn't been fired yet).

    24. ⌛ Set the networkState back to NETWORK_LOADING.

    25. ⌛ Jump back to the find next candidate step above.

The resource fetch algorithm for a media element and a given absolute URL is as follows:

  1. Let the current media resource be the resource given by the absolute URL passed to this algorithm. This is now the element's media resource.

  2. Optionally, run the following substeps. This is the expected behavior if the user agent intends to not attempt to fetch the resource until the use requests it explicitly (e.g. as a way to implement the preload attribute's none keyword).

    1. Set the networkState to NETWORK_IDLE.

    2. Queue a task to fire a simple event named suspend at the element.

    3. Wait for the task to be run.

    4. Wait for an implementation-defined event (e.g. the user requesting that the media element begin playback).

    5. Set the networkState to NETWORK_LOADING.

  3. Perform a potentially CORS-enabled fetch of the current media resource's absolute URL, with the mode being the state of the media element's crossorigin content attribute, the origin being the origin of the media element's Document, and the default origin behaviour set to taint.

    The resource obtained in this fashion, if any, contains the media data. It can be CORS-same-origin or CORS-cross-origin; this affects whether subtitles referenced in the media data are exposed in the API and, for video elements, whether a canvas gets tainted when the video is drawn on it.

    While the load is not suspended (see below), every 350ms (±200ms) or for every byte received, whichever is least frequent, queue a task to fire a simple event named progress at the element.

    The stall timeout is a user-agent defined length of time, which should be about three seconds. When a media element that is actively attempting to obtain media data has failed to receive any data for a duration equal to the stall timeout, the user agent must queue a task to fire a simple event named stalled at the element.

    User agents may allow users to selectively block or slow media data downloads. When a media element's download has been blocked altogether, the user agent must act as if it was stalled (as opposed to acting as if the connection was closed). The rate of the download may also be throttled automatically by the user agent, e.g. to balance the download with other connections sharing the same bandwidth.

    User agents may decide to not download more content at any time, e.g. after buffering five minutes of a one hour media resource, while waiting for the user to decide whether to play the resource or not, or while waiting for user input in an interactive resource. When a media element's download has been suspended, the user agent must queue a task to set the networkState to NETWORK_IDLE and fire a simple event named suspend at the element. If and when downloading of the resource resumes, the user agent must queue a task to set the networkState to NETWORK_LOADING. Between the queuing of these tasks, the load is suspended (so progress events don't fire, as described above).

    The preload attribute provides a hint regarding how much buffering the author thinks is advisable, even in the absence of the autoplay attribute.

    When a user agent decides to completely stall a download, e.g. if it is waiting until the user starts playback before downloading any further content, the element's delaying-the-load-event flag must be set to false. This stops delaying the load event.

    The user agent may use whatever means necessary to fetch the resource (within the constraints put forward by this and other specifications); for example, reconnecting to the server in the face of network errors, using HTTP range retrieval requests, or switching to a streaming protocol. The user agent must consider a resource erroneous only if it has given up trying to fetch it.

    This specification does not currently say whether or how to check the MIME types of the media resources, or whether or how to perform file type sniffing using the actual file data. Implementors differ in their intentions on this matter and it is therefore unclear what the right solution is. In the absence of any requirement here, the HTTP specification's strict requirement to follow the Content-Type header prevails ("Content-Type specifies the media type of the underlying data." ... "If and only if the media type is not given by a Content-Type field, the recipient MAY attempt to guess the media type via inspection of its content and/or the name extension(s) of the URI used to identify the resource.").

    The networking task source tasks to process the data as it is being fetched must, when appropriate, include the relevant substeps from the following list:

    If the media data cannot be fetched at all, due to network errors, causing the user agent to give up trying to fetch the resource
    If the media data can be fetched but is found by inspection to be in an unsupported format, or can otherwise not be rendered at all

    DNS errors, HTTP 4xx and 5xx errors (and equivalents in other protocols), and other fatal network errors that occur before the user agent has established whether the current media resource is usable, as well as the file using an unsupported container format, or using unsupported codecs for all the data, must cause the user agent to execute the following steps:

    1. The user agent should cancel the fetching process.

    2. Abort this subalgorithm, returning to the resource selection algorithm.

    If the media resource is found to have an audio track
    1. Create an AudioTrack object to represent the audio track.

    2. Update the media element's audioTracks attribute's AudioTrackList object with the new AudioTrack object.

    3. Fire an event with the name addtrack, that does not bubble and is not cancelable, and that uses the TrackEvent interface, with the track attribute initialized to the new AudioTrack object, at this AudioTrackList object.

    If the media resource is found to have a video track
    1. Create a VideoTrack object to represent the video track.

    2. Update the media element's videoTracks attribute's VideoTrackList object with the new VideoTrack object.

    3. Fire an event with the name addtrack, that does not bubble and is not cancelable, and that uses the TrackEvent interface, with the track attribute initialized to the new VideoTrack object, at this VideoTrackList object.

    Once enough of the media data has been fetched to determine the duration of the media resource, its dimensions, and other metadata

    This indicates that the resource is usable. The user agent must follow these substeps:

    1. Establish the media timeline for the purposes of the current playback position, the earliest possible position, and the initial playback position, based on the media data.

    2. Update the timeline offset to the date and time that corresponds to the zero time in the media timeline established in the previous step, if any. If no explicit time and date is given by the media resource, the timeline offset must be set to Not-a-Number (NaN).

    3. Set the current playback position and the official playback position to the earliest possible position.

    4. Update the duration attribute with the time of the last frame of the resource, if known, on the media timeline established above. If it is not known (e.g. a stream that is in principle infinite), update the duration attribute to the value positive Infinity.

      The user agent will queue a task to fire a simple event named durationchange at the element at this point.

    5. For video elements, set the videoWidth and videoHeight attributes.

    6. Set the readyState attribute to HAVE_METADATA.

      A loadedmetadata DOM event will be fired as part of setting the readyState attribute to a new value.

    7. Let jumped be false.

    8. If the media element's default playback start position is greater than zero, then seek to that time, and let jumped be true.

    9. Let the media element's default playback start position be zero.

    10. If either the media resource or the address of the current media resource indicate a particular start time, then set the initial playback position to that time and, if jumped is still false, seek to that time and let jumped be true.

      For example, with media formats that support the Media Fragments URI fragment identifier syntax, the fragment identifier can be used to indicate a start position. [MEDIAFRAG]

    11. If either the media resource or the address of the current media resource indicate a particular set of audio or video tracks to enable, then the selected audio tracks must be enabled in the element's audioTracks object, and, of the selected video tracks, the one that is listed first in the element's videoTracks object must be selected.

    12. If the media element has a current media controller, then: if jumped is true and the initial playback position, relative to the current media controller's timeline, is greater than the current media controller's media controller position, then seek the media controller to the media element's initial playback position, relative to the current media controller's timeline; otherwise, seek the media element to the media controller position, relative to the media element's timeline.

    Once the readyState attribute reaches HAVE_CURRENT_DATA, after the loadeddata event has been fired, set the element's delaying-the-load-event flag to false. This stops delaying the load event.

    A user agent that is attempting to reduce network usage while still fetching the metadata for each media resource would also stop buffering at this point, following the rules described previously, which involve the networkState attribute switching to the NETWORK_IDLE value and a suspend event firing.

    The user agent is required to determine the duration of the media resource and go through this step before playing.

    Once the entire media resource has been fetched (but potentially before any of it has been decoded)

    Fire a simple event named progress at the media element.

    Set the networkState to NETWORK_IDLE and fire a simple event named suspend at the media element.

    If the user agent ever discards any media data and then needs to resume the network activity to obtain it again, then it must queue a task to set the networkState to NETWORK_LOADING.

    If the user agent can keep the media resource loaded, then the algorithm will continue to its final step below, which aborts the algorithm.

    If the connection is interrupted after some media data has been received, causing the user agent to give up trying to fetch the resource

    Fatal network errors that occur after the user agent has established whether the current media resource is usable (i.e. once the media element's readyState attribute is no longer HAVE_NOTHING) must cause the user agent to execute the following steps:

    1. The user agent should cancel the fetching process.

    2. Set the error attribute to a new MediaError object whose code attribute is set to MEDIA_ERR_NETWORK.

    3. Fire a simple event named error at the media element.

    4. Set the element's networkState attribute to the NETWORK_IDLE value.

    5. Set the element's delaying-the-load-event flag to false. This stops delaying the load event.

    6. Abort the overall resource selection algorithm.

    If the media data is corrupted

    Fatal errors in decoding the media data that occur after the user agent has established whether the current media resource is usable must cause the user agent to execute the following steps:

    1. The user agent should cancel the fetching process.

    2. Set the error attribute to a new MediaError object whose code attribute is set to MEDIA_ERR_DECODE.

    3. Fire a simple event named error at the media element.

    4. If the media element's readyState attribute has a value equal to HAVE_NOTHING, set the element's networkState attribute to the NETWORK_EMPTY value and fire a simple event named emptied at the element. Otherwise, set the element's networkState attribute to the NETWORK_IDLE value.

    5. Set the element's delaying-the-load-event flag to false. This stops delaying the load event.

    6. Abort the overall resource selection algorithm.

    If the media data fetching process is aborted by the user

    The fetching process is aborted by the user, e.g. because the user navigated the browsing context to another page, the user agent must execute the following steps. These steps are not followed if the load() method itself is invoked while these steps are running, as the steps above handle that particular kind of abort.

    1. The user agent should cancel the fetching process.

    2. Set the error attribute to a new MediaError object whose code attribute is set to MEDIA_ERR_ABORTED.

    3. Fire a simple event named abort at the media element.

    4. If the media element's readyState attribute has a value equal to HAVE_NOTHING, set the element's networkState attribute to the NETWORK_EMPTY value and fire a simple event named emptied at the element. Otherwise, set the element's networkState attribute to the NETWORK_IDLE value.

    5. Set the element's delaying-the-load-event flag to false. This stops delaying the load event.

    6. Abort the overall resource selection algorithm.

    If the media data can be fetched but has non-fatal errors or uses, in part, codecs that are unsupported, preventing the user agent from rendering the content completely correctly but not preventing playback altogether

    The server returning data that is partially usable but cannot be optimally rendered must cause the user agent to render just the bits it can handle, and ignore the rest.

    If the media resource is found to declare a media-resource-specific text track that the user agent supports

    If the media data is CORS-same-origin, run the steps to expose a media-resource-specific text track with the relevant data.

    Cross-origin videos do not expose their subtitles, since that would allow attacks such as hostile sites reading subtitles from confidential videos on a user's intranet.

    When the networking task source has queued the last task as part of fetching the media resource (i.e. once the download has completed), if the fetching process completes without errors, including decoding the media data, and if all of the data is available to the user agent without network access, then, the user agent must move on to the next step. This might never happen, e.g. when streaming an infinite resource such as Web radio, or if the resource is longer than the user agent's ability to cache data.

    While the user agent might still need network access to obtain parts of the media resource, the user agent must remain on this step.

    For example, if the user agent has discarded the first half of a video, the user agent will remain at this step even once the playback has ended, because there is always the chance the user will seek back to the start. In fact, in this situation, once playback has ended, the user agent will end up firing a suspend event, as described earlier.

  4. If the user agent ever reaches this step (which can only happen if the entire resource gets loaded and kept available): abort the overall resource selection algorithm.


The preload attribute is an enumerated attribute. The following table lists the keywords and states for the attribute — the keywords in the left column map to the states in the cell in the second column on the same row as the keyword. The attribute can be changed even once the media resource is being buffered or played; the descriptions in the table below are to be interpreted with that in mind.

Keyword State Brief description
none None Hints to the user agent that either the author does not expect the user to need the media resource, or that the server wants to minimise unnecessary traffic. This state does not provide a hint regarding how aggressively to actually download the media resource if buffering starts anyway (e.g. once the user hits "play").
metadata Metadata Hints to the user agent that the author does not expect the user to need the media resource, but that fetching the resource metadata (dimensions, track list, duration, etc), and maybe even the first few frames, is reasonable. If the user agent precisely fetches no more than the metadata, then the media element will end up with its readyState attribute set to HAVE_METADATA; typically though, some frames will be obtained as well and it will probably be HAVE_CURRENT_DATA or HAVE_FUTURE_DATA. When the media resource is playing, hints to the user agent that bandwidth is to be considered scarce, e.g. suggesting throttling the download so that the media data is obtained at the slowest possible rate that still maintains consistent playback.
auto Automatic Hints to the user agent that the user agent can put the user's needs first without risk to the server, up to and including optimistically downloading the entire resource.

The empty string is also a valid keyword, and maps to the Automatic state. The attribute's missing value default is user-agent defined, though the Metadata state is suggested as a compromise between reducing server load and providing an optimal user experience.

Authors might switch the attribute from "none" or "metadata" to "auto" dynamically once the user begins playback. For example, on a page with many videos this might be used to indicate that the many videos are not to be downloaded unless requested, but that once one is requested it is to be downloaded aggressively.

The preload attribute is intended to provide a hint to the user agent about what the author thinks will lead to the best user experience. The attribute may be ignored altogether, for example based on explicit user preferences or based on the available connectivity.

The preload IDL attribute must reflect the content attribute of the same name, limited to only known values.

The autoplay attribute can override the preload attribute (since if the media plays, it naturally has to buffer first, regardless of the hint given by the preload attribute). Including both is not an error, however.


media . buffered

Returns a TimeRanges object that represents the ranges of the media resource that the user agent has buffered.

The buffered attribute must return a new static normalized TimeRanges object that represents the ranges of the media resource, if any, that the user agent has buffered, at the time the attribute is evaluated. Users agents must accurately determine the ranges available, even for media streams where this can only be determined by tedious inspection.

Typically this will be a single range anchored at the zero point, but if, e.g. the user agent uses HTTP range requests in response to seeking, then there could be multiple ranges.

User agents may discard previously buffered data.

Thus, a time position included within a range of the objects return by the buffered attribute at one time can end up being not included in the range(s) of objects returned by the same attribute at later times.

4.8.10.6 Offsets into the media resource
media . duration

Returns the length of the media resource, in seconds, assuming that the start of the media resource is at time zero.

Returns NaN if the duration isn't available.

Returns Infinity for unbounded streams.

media . currentTime [ = value ]

Returns the official playback position, in seconds.

Can be set, to seek to the given time.

Will throw an InvalidStateError exception if there is no selected media resource or if there is a current media controller.

A media resource has a media timeline that maps times (in seconds) to positions in the media resource. The origin of a timeline is its earliest defined position. The duration of a timeline is its last defined position.

Establishing the media timeline: If the media resource somehow specifies an explicit timeline whose origin is not negative (i.e. gives each frame a specific time offset and gives the first frame a zero or positive offset), then the media timeline should be that timeline. (Whether the media resource can specify a timeline or not depends on the media resource's format.) If the media resource specifies an explicit start time and date, then that time and date should be considered the zero point in the media timeline; the timeline offset will be the time and date, exposed using the startDate attribute.

If the media resource has a discontinuous timeline, the user agent must extend the timeline used at the start of the resource across the entire resource, so that the media timeline of the media resource increases linearly starting from the earliest possible position (as defined below), even if the underlying media data has out-of-order or even overlapping time codes.

For example, if two clips have been concatenated into one video file, but the video format exposes the original times for the two clips, the video data might expose a timeline that goes, say, 00:15..00:29 and then 00:05..00:38. However, the user agent would not expose those times; it would instead expose the times as 00:15..00:29 and 00:29..01:02, as a single video.

In the rare case of a media resource that does not have an explicit timeline, the zero time on the media timeline should correspond to the first frame of the media resource. In the even rarer case of a media resource with no explicit timings of any kind, not even frame durations, the user agent must itself determine the time for each frame in a user-agent-defined manner.

An example of a file format with no explicit timeline but with explicit frame durations is the Animated GIF format. An example of a file format with no explicit timings at all is the JPEG-push format (multipart/x-mixed-replace with JPEG frames, often used as the format for MJPEG streams).

If, in the case of a resource with no timing information, the user agent will nonetheless be able to seek to an earlier point than the first frame originally provided by the server, then the zero time should correspond to the earliest seekable time of the media resource; otherwise, it should correspond to the first frame received from the server (the point in the media resource at which the user agent began receiving the stream).

At the time of writing, there is no known format that lacks explicit frame time offsets yet still supports seeking to a frame before the first frame sent by the server.

Consider a stream from a TV broadcaster, which begins streaming on a sunny Friday afternoon in October, and always sends connecting user agents the media data on the same media timeline, with its zero time set to the start of this stream. Months later, user agents connecting to this stream will find that the first frame they receive has a time with millions of seconds. The startDate attribute would always return the date that the broadcast started; this would allow controllers to display real times in their scrubber (e.g. "2:30pm") rather than a time relative to when the broadcast began ("8 months, 4 hours, 12 minutes, and 23 seconds").

Consider a stream that carries a video with several concatenated fragments, broadcast by a server that does not allow user agents to request specific times but instead just streams the video data in a predetermined order, with the first frame delivered always being identified as the frame with time zero. If a user agent connects to this stream and receives fragments defined as covering timestamps 2010-03-20 23:15:00 UTC to 2010-03-21 00:05:00 UTC and 2010-02-12 14:25:00 UTC to 2010-02-12 14:35:00 UTC, it would expose this with a media timeline starting at 0s and extending to 3,600s (one hour). Assuming the streaming server disconnected at the end of the second clip, the duration attribute would then return 3,600. The startDate attribute would return a Date object with a time corresponding to 2010-03-20 23:15:00 UTC. However, if a different user agent connected five minutes later, it would (presumably) receive fragments covering timestamps 2010-03-20 23:20:00 UTC to 2010-03-21 00:05:00 UTC and 2010-02-12 14:25:00 UTC to 2010-02-12 14:35:00 UTC, and would expose this with a media timeline starting at 0s and extending to 3,300s (fifty five minutes). In this case, the startDate attribute would return a Date object with a time corresponding to 2010-03-20 23:20:00 UTC.

In both of these examples, the seekable attribute would give the ranges that the controller would want to actually display in its UI; typically, if the servers don't support seeking to arbitrary times, this would be the range of time from the moment the user agent connected to the stream up to the latest frame that the user agent has obtained; however, if the user agent starts discarding earlier information, the actual range might be shorter.

In any case, the user agent must ensure that the earliest possible position (as defined below) using the established media timeline, is greater than or equal to zero.

The media timeline also has an associated clock. Which clock is used is user-agent defined, and may be media resource-dependent, but it should approximate the user's wall clock.

All the media elements that share current media controller use the same clock for their media timeline.

Media elements have a current playback position, which must initially (i.e. in the absence of media data) be zero seconds. The current playback position is a time on the media timeline.

Media elements also have an official playback position, which must initially be set to zero seconds. The official playback position is an approximation of the current playback position that is kept stable while scripts are running.

Media elements also have a default playback start position, which must initially be set to zero seconds. This time is used to allow the element to be seeked even before the media is loaded.

The currentTime attribute must, on getting, return the media element's default playback start position, unless that is zero, in which case it must return the element's official playback position. The returned value must be expressed in seconds. On setting, if the media element has a current media controller, then the user agent must throw an InvalidStateError exception; otherwise, if the media element's readyState is HAVE_NOTHING, then it must set the media element's default playback start position to the new value; otherwise, it must set the official playback position to the new value and then seek to the new value. The new value must be interpreted as being in seconds.

Media elements have an initial playback position, which must initially (i.e. in the absence of media data) be zero seconds. The initial playback position is updated when a media resource is loaded. The initial playback position is a time on the media timeline.

If the media resource is a streaming resource, then the user agent might be unable to obtain certain parts of the resource after it has expired from its buffer. Similarly, some media resources might have a media timeline that doesn't start at zero. The earliest possible position is the earliest position in the stream or resource that the user agent can ever obtain again. It is also a time on the media timeline.

The earliest possible position is not explicitly exposed in the API; it corresponds to the start time of the first range in the seekable attribute's TimeRanges object, if any, or the current playback position otherwise.

When the earliest possible position changes, then: if the current playback position is before the earliest possible position, the user agent must seek to the earliest possible position; otherwise, if the user agent has not fired a timeupdate event at the element in the past 15 to 250ms and is not still running event handlers for such an event, then the user agent must queue a task to fire a simple event named timeupdate at the element.

Because of the above requirement and the requirement in the resource fetch algorithm that kicks in when the metadata of the clip becomes known, the current playback position can never be less than the earliest possible position.

If at any time the user agent learns that an audio or video track has ended and all media data relating to that track corresponds to parts of the media timeline that are before the earliest possible position, the user agent may queue a task to remove the track from the audioTracks attribute's AudioTrackList object or the videoTracks attribute's VideoTrackList object as appropriate and then fire an event with the name removetrack, that does not bubble and is not cancelable, and that uses the TrackEvent interface, with the track attribute initialized to the AudioTrack or VideoTrack object representing the track, at the media element's aforementioned AudioTrackList or VideoTrackList object.

The duration attribute must return the time of the end of the media resource, in seconds, on the media timeline. If no media data is available, then the attributes must return the Not-a-Number (NaN) value. If the media resource is not known to be bounded (e.g. streaming radio, or a live event with no announced end time), then the attribute must return the positive Infinity value.

The user agent must determine the duration of the media resource before playing any part of the media data and before setting readyState to a value equal to or greater than HAVE_METADATA, even if doing so requires fetching multiple parts of the resource.

When the length of the media resource changes to a known value (e.g. from being unknown to known, or from a previously established length to a new length) the user agent must queue a task to fire a simple event named durationchange at the media element. (The event is not fired when the duration is reset as part of loading a new media resource.) If the duration is changed such that the current playback position ends up being greater than the time of the end of the media resource, then the user agent must also seek the to the time of the end of the media resource.

If an "infinite" stream ends for some reason, then the duration would change from positive Infinity to the time of the last frame or sample in the stream, and the durationchange event would be fired. Similarly, if the user agent initially estimated the media resource's duration instead of determining it precisely, and later revises the estimate based on new information, then the duration would change and the durationchange event would be fired.

Some video files also have an explicit date and time corresponding to the zero time in the media timeline, known as the timeline offset. Initially, the timeline offset must be set to Not-a-Number (NaN).

The startDate attribute must return a new Date object representing the current timeline offset.


The loop attribute is a boolean attribute that, if specified, indicates that the media element is to seek back to the start of the media resource upon reaching the end.

The loop attribute has no effect while the element has a current media controller.

The loop IDL attribute must reflect the content attribute of the same name.

4.8.10.7 Ready states
media . readyState

Returns a value that expresses the current state of the element with respect to rendering the current playback position, from the codes in the list below.

Media elements have a ready state, which describes to what degree they are ready to be rendered at the current playback position. The possible values are as follows; the ready state of a media element at any particular time is the greatest value describing the state of the element:

HAVE_NOTHING (numeric value 0)

No information regarding the media resource is available. No data for the current playback position is available. Media elements whose networkState attribute are set to NETWORK_EMPTY are always in the HAVE_NOTHING state.

HAVE_METADATA (numeric value 1)

Enough of the resource has been obtained that the duration of the resource is available. In the case of a video element, the dimensions of the video are also available. The API will no longer throw an exception when seeking. No media data is available for the immediate current playback position.

HAVE_CURRENT_DATA (numeric value 2)

Data for the immediate current playback position is available, but either not enough data is available that the user agent could successfully advance the current playback position in the direction of playback at all without immediately reverting to the HAVE_METADATA state, or there is no more data to obtain in the direction of playback. For example, in video this corresponds to the user agent having data from the current frame, but not the next frame, when the current playback position is at the end of the current frame; and to when playback has ended.

HAVE_FUTURE_DATA (numeric value 3)

Data for the immediate current playback position is available, as well as enough data for the user agent to advance the current playback position in the direction of playback at least a little without immediately reverting to the HAVE_METADATA state, and the text tracks are ready. For example, in video this corresponds to the user agent having data for at least the current frame and the next frame when the current playback position is at the instant in time between the two frames, or to the user agent having the video data for the current frame and audio data to keep playing at least a little when the current playback position is in the middle of a frame. The user agent cannot be in this state if playback has ended, as the current playback position can never advance in this case.

HAVE_ENOUGH_DATA (numeric value 4)

All the conditions described for the HAVE_FUTURE_DATA state are met, and, in addition, either of the following conditions is also true:

In practice, the difference between HAVE_METADATA and HAVE_CURRENT_DATA is negligible. Really the only time the difference is relevant is when painting a video element onto a canvas, where it distinguishes the case where something will be drawn (HAVE_CURRENT_DATA or greater) from the case where nothing is drawn (HAVE_METADATA or less). Similarly, the difference between HAVE_CURRENT_DATA (only the current frame) and HAVE_FUTURE_DATA (at least this frame and the next) can be negligible (in the extreme, only one frame). The only time that distinction really matters is when a page provides an interface for "frame-by-frame" navigation.

When the ready state of a media element whose networkState is not NETWORK_EMPTY changes, the user agent must follow the steps given below:

  1. Apply the first applicable set of substeps from the following list:

    If the previous ready state was HAVE_NOTHING, and the new ready state is HAVE_METADATA

    Queue a task to fire a simple event named loadedmetadata at the element.

    Before this task is run, as part of the event loop mechanism, the rendering will have been updated to resize the video element if appropriate.

    If the previous ready state was HAVE_METADATA and the new ready state is HAVE_CURRENT_DATA or greater

    If this is the first time this occurs for this media element since the load() algorithm was last invoked, the user agent must queue a task to fire a simple event named loadeddata at the element.

    If the new ready state is HAVE_FUTURE_DATA or HAVE_ENOUGH_DATA, then the relevant steps below must then be run also.

    If the previous ready state was HAVE_FUTURE_DATA or more, and the new ready state is HAVE_CURRENT_DATA or less

    If the media element was potentially playing before its readyState attribute changed to a value lower than HAVE_FUTURE_DATA, and the element has not ended playback, and playback has not stopped due to errors, paused for user interaction, or paused for in-band content, the user agent must queue a task to fire a simple event named timeupdate at the element, and queue a task to fire a simple event named waiting at the element.

    If the previous ready state was HAVE_CURRENT_DATA or less, and the new ready state is HAVE_FUTURE_DATA

    The user agent must queue a task to fire a simple event named canplay.

    If the element's paused attribute is false, the user agent must queue a task to fire a simple event named playing.

    If the new ready state is HAVE_ENOUGH_DATA

    If the previous ready state was HAVE_CURRENT_DATA or less, the user agent must queue a task to fire a simple event named canplay, and, if the element's paused attribute is false, queue a task to fire a simple event named playing.

    If the autoplaying flag is true, and the paused attribute is true, and the media element has an autoplay attribute specified, and the media element's Document's active sandboxing flag set does not have the sandboxed automatic features browsing context flag set, then the user agent may also set the paused attribute to false, queue a task to fire a simple event named play, and queue a task to fire a simple event named playing.

    User agents do not need to support autoplay, and it is suggested that user agents honor user preferences on the matter. Authors are urged to use the autoplay attribute rather than using script to force the video to play, so as to allow the user to override the behavior if so desired.

    In any case, the user agent must finally queue a task to fire a simple event named canplaythrough.

  2. If the media element has a current media controller, then report the controller state for the media element's current media controller.

It is possible for the ready state of a media element to jump between these states discontinuously. For example, the state of a media element can jump straight from HAVE_METADATA to HAVE_ENOUGH_DATA without passing through the HAVE_CURRENT_DATA and HAVE_FUTURE_DATA states.

The readyState IDL attribute must, on getting, return the value described above that describes the current ready state of the media element.

The autoplay attribute is a boolean attribute. When present, the user agent (as described in the algorithm described herein) will automatically begin playback of the media resource as soon as it can do so without stopping.

Authors are urged to use the autoplay attribute rather than using script to trigger automatic playback, as this allows the user to override the automatic playback when it is not desired, e.g. when using a screen reader. Authors are also encouraged to consider not using the automatic playback behavior at all, and instead to let the user agent wait for the user to start playback explicitly.

The autoplay IDL attribute must reflect the content attribute of the same name.

4.8.10.8 Playing the media resource
media . paused

Returns true if playback is paused; false otherwise.

media . ended

Returns true if playback has reached the end of the media resource.

media . defaultPlaybackRate [ = value ]

Returns the default rate of playback, for when the user is not fast-forwarding or reversing through the media resource.

Can be set, to change the default rate of playback.

The default rate has no direct effect on playback, but if the user switches to a fast-forward mode, when they return to the normal playback mode, it is expected that the rate of playback will be returned to the default rate of playback.

When the element has a current media controller, the defaultPlaybackRate attribute is ignored and the current media controller's defaultPlaybackRate is used instead.

media . playbackRate [ = value ]

Returns the current rate playback, where 1.0 is normal speed.

Can be set, to change the rate of playback.

When the element has a current media controller, the playbackRate attribute is ignored and the current media controller's playbackRate is used instead.

media . played

Returns a TimeRanges object that represents the ranges of the media resource that the user agent has played.

media . play()

Sets the paused attribute to false, loading the media resource and beginning playback if necessary. If the playback had ended, will restart it from the start.

media . pause()

Sets the paused attribute to true, loading the media resource if necessary.

The paused attribute represents whether the media element is paused or not. The attribute must initially be true.

A media element is a blocked media element if its readyState attribute is in the HAVE_NOTHING state, the HAVE_METADATA state, or the HAVE_CURRENT_DATA state, or if the element has paused for user interaction or paused for in-band content.

A media element is said to be potentially playing when its paused attribute is false, the element has not ended playback, playback has not stopped due to errors, the element either has no current media controller or has a current media controller but is not blocked on its media controller, and the element is not a blocked media element.

A waiting DOM event can be fired as a result of an element that is potentially playing stopping playback due to its readyState attribute changing to a value lower than HAVE_FUTURE_DATA.

A media element is said to have ended playback when:

The ended attribute must return true if, the last time the event loop reached step 1, the media element had ended playback and the direction of playback was forwards, and false otherwise.

A media element is said to have stopped due to errors when the element's readyState attribute is HAVE_METADATA or greater, and the user agent encounters a non-fatal error during the processing of the media data, and due to that error, is not able to play the content at the current playback position.

A media element is said to have paused for user interaction when its paused attribute is false, the readyState attribute is either HAVE_FUTURE_DATA or HAVE_ENOUGH_DATA and the user agent has reached a point in the media resource where the user has to make a selection for the resource to continue. If the media element has a current media controller when this happens, then the user agent must report the controller state for the media element's current media controller. If the media element has a current media controller when the user makes a selection, allowing playback to resume, the user agent must similarly report the controller state for the media element's current media controller.

It is possible for a media element to have both ended playback and paused for user interaction at the same time.

When a media element that is potentially playing stops playing because it has paused for user interaction, the user agent must queue a task to fire a simple event named timeupdate at the element.

A media element is said to have paused for in-band content when its paused attribute is false, the readyState attribute is either HAVE_FUTURE_DATA or HAVE_ENOUGH_DATA and the user agent has suspended playback of the media resource in order to play content that is temporally anchored to the media resource and has a non-zero length, or to play content that is temporally anchored to a segment of the media resource but has a length longer than that segment. If the media element has a current media controller when this happens, then the user agent must report the controller state for the media element's current media controller. If the media element has a current media controller when the user agent unsuspends playback, the user agent must similarly report the controller state for the media element's current media controller.

One example of when a media element would be paused for in-band content is when the user agent is playing audio descriptions from an external WebVTT file, and the synthesized speech generated for a cue is longer than the time between the text track cue start time and the text track cue end time.


When the current playback position reaches the end of the media resource when the direction of playback is forwards, then the user agent must follow these steps:

  1. If the media element has a loop attribute specified and does not have a current media controller, then seek to the earliest possible position of the media resource and abort these steps.

  2. As defined above, the ended IDL attribute starts returning true once the event loop's current task ends.

  3. Queue a task to fire a simple event named timeupdate at the media element.

  4. Queue a task that, if the media element does not have a current media controller, and the media element has still ended playback, and the direction of playback is still forwards, and paused is false, changes paused to true and fires a simple event named pause at the media element.

  5. Queue a task to fire a simple event named ended at the media element.

  6. If the media element has a current media controller, then report the controller state for the media element's current media controller.

When the current playback position reaches the earliest possible position of the media resource when the direction of playback is backwards, then the user agent must only queue a task to fire a simple event named timeupdate at the element.


The defaultPlaybackRate attribute gives the desired speed at which the media resource is to play, as a multiple of its intrinsic speed. The attribute is mutable: on getting it must return the last value it was set to, or 1.0 if it hasn't yet been set; on setting the attribute must be set to the new value.

The defaultPlaybackRate is used by the user agent when it exposes a user interface to the user.

The playbackRate attribute gives the effective playback rate (assuming there is no current media controller overriding it), which is the speed at which the media resource plays, as a multiple of its intrinsic speed. If it is not equal to the defaultPlaybackRate, then the implication is that the user is using a feature such as fast forward or slow motion playback. The attribute is mutable: on getting it must return the last value it was set to, or 1.0 if it hasn't yet been set; on setting the attribute must be set to the new value, and the playback will change speed (if the element is potentially playing and there is no current media controller).

When the defaultPlaybackRate or playbackRate attributes change value (either by being set by script or by being changed directly by the user agent, e.g. in response to user control) the user agent must queue a task to fire a simple event named ratechange at the media element.

The defaultPlaybackRate and playbackRate attributes have no effect when the media element has a current media controller; the namesake attributes on the MediaController object are used instead in that situation.


The played attribute must return a new static normalized TimeRanges object that represents the ranges of points on the media timeline of the media resource reached through the usual monotonic increase of the current playback position during normal playback, if any, at the time the attribute is evaluated.


When the play() method on a media element is invoked, the user agent must run the following steps.

  1. If the media element's networkState attribute has the value NETWORK_EMPTY, invoke the media element's resource selection algorithm.

  2. If the playback has ended and the direction of playback is forwards, and the media element does not have a current media controller, seek to the earliest possible position of the media resource.

    This will cause the user agent to queue a task to fire a simple event named timeupdate at the media element.

  3. If the media element has a current media controller, then bring the media element up to speed with its new media controller.

  4. If the media element's paused attribute is true, run the following substeps:

    1. Change the value of paused to false.

    2. Queue a task to fire a simple event named play at the element.

    3. If the media element's readyState attribute has the value HAVE_NOTHING, HAVE_METADATA, or HAVE_CURRENT_DATA, queue a task to fire a simple event named waiting at the element.

      Otherwise, the media element's readyState attribute has the value HAVE_FUTURE_DATA or HAVE_ENOUGH_DATA: queue a task to fire a simple event named playing at the element.

  5. Set the media element's autoplaying flag to false.

  6. If the media element has a current media controller, then report the controller state for the media element's current media controller.


When the pause() method is invoked, and when the user agent is required to pause the media element, the user agent must run the following steps:

  1. If the media element's networkState attribute has the value NETWORK_EMPTY, invoke the media element's resource selection algorithm.

  2. Set the media element's autoplaying flag to false.

  3. If the media element's paused attribute is false, run the following steps:

    1. Change the value of paused to true.

    2. Queue a task to fire a simple event named timeupdate at the element.

    3. Queue a task to fire a simple event named pause at the element.

    4. Set the official playback position to the current playback position.

  4. If the media element has a current media controller, then report the controller state for the media element's current media controller.


The effective playback rate is not necessarily the element's playbackRate. When a media element has a current media controller, its effective playback rate is the MediaController's media controller playback rate. Otherwise, the effective playback rate is just the element's playbackRate. Thus, the current media controller overrides the media element.

If the effective playback rate is positive or zero, then the direction of playback is forwards. Otherwise, it is backwards.

When a media element is potentially playing and its Document is a fully active Document, its current playback position must increase monotonically at effective playback rate units of media time per unit time of the media timeline's clock. (This specification always refers to this as an increase, but that increase could actually be a decrease if the effective playback rate is negative.)

The effective playback rate can be 0.0, in which case the current playback position doesn't move, despite playback not being paused (paused doesn't become true, and the pause event doesn't fire).

This specification doesn't define how the user agent achieves the appropriate playback rate — depending on the protocol and media available, it is plausible that the user agent could negotiate with the server to have the server provide the media data at the appropriate rate, so that (except for the period between when the rate is changed and when the server updates the stream's playback rate) the client doesn't actually have to drop or interpolate any frames.

Any time the user agent provides a stable state, the official playback position must be set to the current playback position.

When the direction of playback is backwards, any corresponding audio must be muted. When the effective playback rate is so low or so high that the user agent cannot play audio usefully, the corresponding audio must also be muted. If the effective playback rate is not 1.0, the user agent may apply pitch adjustments to the audio as necessary to render it faithfully.

Media elements that are potentially playing while not in a Document must not play any video, but should play any audio component. Media elements must not stop playing just because all references to them have been removed; only once a media element is in a state where no further audio could ever be played by that element may the element be garbage collected.

It is possible for an element to which no explicit references exist to play audio, even if such an element is not still actively playing: for instance, it could have a current media controller that still has references and can still be unpaused, or it could be unpaused but stalled waiting for content to buffer.


When the current playback position of a media element changes (e.g. due to playback or seeking), the user agent must run the following steps. If the current playback position changes while the steps are running, then the user agent must wait for the steps to complete, and then must immediately rerun the steps. (These steps are thus run as often as possible or needed — if one iteration takes a long time, this can cause certain cues to be skipped over as the user agent rushes ahead to "catch up".)

  1. Let current cues be a list of cues, initialized to contain all the cues of all the hidden, showing, or showing by default text tracks of the media element (not the disabled ones) whose start times are less than or equal to the current playback position and whose end times are greater than the current playback position.

  2. Let other cues be a list of cues, initialized to contain all the cues of hidden, showing, and showing by default text tracks of the media element that are not present in current cues.

  3. Let last time be the current playback position at the time this algorithm was last run for this media element, if this is not the first time it has run.

  4. If the current playback position has, since the last time this algorithm was run, only changed through its usual monotonic increase during normal playback, then let missed cues be the list of cues in other cues whose start times are greater than or equal to last time and whose end times are less than or equal to the current playback position. Otherwise, let missed cues be an empty list.

  5. If the time was reached through the usual monotonic increase of the current playback position during normal playback, and if the user agent has not fired a timeupdate event at the element in the past 15 to 250ms and is not still running event handlers for such an event, then the user agent must queue a task to fire a simple event named timeupdate at the element. (In the other cases, such as explicit seeks, relevant events get fired as part of the overall process of changing the current playback position.)

    The event thus is not to be fired faster than about 66Hz or slower than 4Hz (assuming the event handlers don't take longer than 250ms to run). User agents are encouraged to vary the frequency of the event based on the system load and the average cost of processing the event each time, so that the UI updates are not any more frequent than the user agent can comfortably handle while decoding the video.

  6. If all of the cues in current cues have their text track cue active flag set, none of the cues in other cues have their text track cue active flag set, and missed cues is empty, then abort these steps.

  7. If the time was reached through the usual monotonic increase of the current playback position during normal playback, and there are cues in other cues that have their text track cue pause-on-exit flag set and that either have their text track cue active flag set or are also in missed cues, then immediately pause the media element.

    In the other cases, such as explicit seeks, playback is not paused by going past the end time of a cue, even if that cue has its text track cue pause-on-exit flag set.

  8. Let events be a list of tasks, initially empty. Each task in this list will be associated with a text track, a text track cue, and a time, which are used to sort the list before the tasks are queued.

    Let affected tracks be a list of text tracks, initially empty.

    When the steps below say to prepare an event named event for a text track cue target with a time time, the user agent must run these substeps:

    1. Let track be the text track with which the text track cue target is associated.

    2. Create a task to fire a simple event named event at target.

    3. Add to the newly create task to events, associated with the time time, the text track track, and the text track cue target.

    4. Add track to affected tracks.

  9. For each text track cue in missed cues, prepare an event named enter for the TextTrackCue object with the text track cue start time.

  10. For each text track cue in other cues that either has its text track cue active flag set or is in missed cues, prepare an event named exit for the TextTrackCue object with the later of the text track cue end time and the text track cue start time.

  11. For each text track cue in current cues that does not have its text track cue active flag set, prepare an event named enter for the TextTrackCue object with the text track cue start time.

  12. Sort the tasks in events in ascending time order (tasks with earlier times first).

    Further sort tasks in events that have the same time by the relative text track cue order of the text track cues associated with these tasks.

    Finally, sort tasks in events that have the same time and same text track cue order by placing tasks that fire enter events before those that fire exit events.

  13. Queue each task in events, in list order.

  14. Sort affected tracks in the same order as the text tracks appear in the media element's list of text tracks, and remove duplicates.

  15. For each text track in affected tracks, in the list order, queue a task to fire a simple event named cuechange at the TextTrack object, and, if the text track has a corresponding track element, to then fire a simple event named cuechange at the track element as well.

  16. Set the text track cue active flag of all the cues in the current cues, and unset the text track cue active flag of all the cues in the other cues.

  17. Run the rules for updating the text track rendering of each of the text tracks in affected tracks that are showing or showing by default. For example, for text tracks based on WebVTT, the rules for updating the display of WebVTT text tracks. [WEBVTT]

For the purposes of the algorithm above, a text track cue is considered to be part of a text track only if it is listed in the text track list of cues, not merely if it is associated with the text track.

If the media element's Document stops being a fully active document, then the playback will stop until the document is active again.

When a media element is removed from a Document, the user agent must run the following steps:

  1. Asynchronously await a stable state, allowing the task that removed the media element from the Document to continue. The synchronous section consists of all the remaining steps of this algorithm. (Steps in the synchronous section are marked with ⌛.)

  2. ⌛ If the media element is in a Document, abort these steps.

  3. ⌛ If the media element's networkState attribute has the value NETWORK_EMPTY, abort these steps.

  4. Pause the media element.

4.8.10.9 Seeking
media . seeking

Returns true if the user agent is currently seeking.

media . seekable

Returns a TimeRanges object that represents the ranges of the media resource to which it is possible for the user agent to seek.

The seeking attribute must initially have the value false.

When the user agent is required to seek to a particular new playback position in the media resource, it means that the user agent must run the following steps. This algorithm interacts closely with the event loop mechanism; in particular, it has a synchronous section (which is triggered as part of the event loop algorithm). Steps in that section are marked with ⌛.

  1. If the media element's readyState is HAVE_NOTHING, abort these steps.

  2. If the element's seeking IDL attribute is true, then another instance of this algorithm is already running. Abort that other instance of the algorithm without waiting for the step that it is running to complete.

  3. Set the seeking IDL attribute to true.

  4. If the seek was in response to a DOM method call or setting of an IDL attribute, then continue the script. The remainder of these steps must be run asynchronously. With the exception of the steps marked with ⌛, they could be aborted at any time by another instance of this algorithm being invoked.

  5. If the new playback position is later than the end of the media resource, then let it be the end of the media resource instead.

  6. If the new playback position is less than the earliest possible position, let it be that position instead.

  7. If the (possibly now changed) new playback position is not in one of the ranges given in the seekable attribute, then let it be the position in one of the ranges given in the seekable attribute that is the nearest to the new playback position. If two positions both satisfy that constraint (i.e. the new playback position is exactly in the middle between two ranges in the seekable attribute) then use the position that is closest to the current playback position. If there are no ranges given in the seekable attribute then set the seeking IDL attribute to false and abort these steps.

  8. Queue a task to fire a simple event named seeking at the element.

  9. Set the current playback position to the given new playback position.

    If the media element was potentially playing immediately before it started seeking, but seeking caused its readyState attribute to change to a value lower than HAVE_FUTURE_DATA, then a waiting will be fired at the element.

    The currentTime attribute does not get updated asynchronously, as it returns the official playback position, not the current playback position.

  10. Wait until the user agent has established whether or not the media data for the new playback position is available, and, if it is, until it has decoded enough data to play back that position.

  11. Await a stable state. The synchronous section consists of all the remaining steps of this algorithm. (Steps in the synchronous section are marked with ⌛.)

  12. ⌛ Set the seeking IDL attribute to false.

  13. Queue a task to fire a simple event named timeupdate at the element.

  14. Queue a task to fire a simple event named seeked at the element.

The seekable attribute must return a new static normalized TimeRanges object that represents the ranges of the media resource, if any, that the user agent is able to seek to, at the time the attribute is evaluated.

If the user agent can seek to anywhere in the media resource, e.g. because it is a simple movie file and the user agent and the server support HTTP Range requests, then the attribute would return an object with one range, whose start is the time of the first frame (the earliest possible position, typically zero), and whose end is the same as the time of the first frame plus the duration attribute's value (which would equal the time of the last frame, and might be positive Infinity).

The range might be continuously changing, e.g. if the user agent is buffering a sliding window on an infinite stream. This is the behavior seen with DVRs viewing live TV, for instance.

Media resources might be internally scripted or interactive. Thus, a media element could play in a non-linear fashion. If this happens, the user agent must act as if the algorithm for seeking was used whenever the current playback position changes in a discontinuous fashion (so that the relevant events fire). If the media element has a current media controller, then the user agent must seek the media controller appropriately instead.

4.8.10.10 Media resources with multiple media tracks

A media resource can have multiple embedded audio and video tracks. For example, in addition to the primary video and audio tracks, a media resource could have foreign-language dubbed dialogues, director's commentaries, audio descriptions, alternative angles, or sign-language overlays.

media . audioTracks

Returns an AudioTrackList object representing the audio tracks available in the media resource.

media . videoTracks

Returns a VideoTrackList object representing the video tracks available in the media resource.

The audioTracks attribute of a media element must return a live AudioTrackList object representing the audio tracks available in the media element's media resource. The same object must be returned each time.

The videoTracks attribute of a media element must return a live VideoTrackList object representing the video tracks available in the media element's media resource. The same object must be returned each time.

There are only ever one AudioTrackList object and one VideoTrackList object per media element, even if another media resource is loaded into the element: the objects are reused. (The AudioTrack and VideoTrack objects are not, though.)

In this example, a script defines a function that takes a URL to a video and a reference to an element where the video is to be placed. That function then tries to load the video, and, once it is loaded, checks to see if there is a sign-language track available. If there is, it also displays that track. Both tracks are just placed in the given container; it's assumed that styles have been applied to make this work in a pretty way!

<script>
 function loadVideo(url, container) {
   var controller = new MediaController();
   var video = document.createElement('video');
   video.src = url;
   video.autoplay = true;
   video.controls = true;
   video.controller = controller;
   container.appendChild(video);
   video.onloadedmetadata = function (event) {
     for (var i = 0; i < video.videoTracks.length; i += 1) {
       if (video.videoTracks[i].kind == 'sign') {
         var sign = document.createElement('video');
         sign.src = url + '#track=' + video.videoTracks[i].id; 
         sign.autoplay = true;
         sign.controller = controller;
         container.appendChild(sign);
         return;
       }
     }
   };
 }
</script>
4.8.10.10.1 AudioTrackList and VideoTrackList objects

The AudioTrackList and VideoTrackList interfaces are used by attributes defined in the previous section.

interface AudioTrackList : EventTarget {
  readonly attribute unsigned long length;
  getter AudioTrack (unsigned long index);
  AudioTrack? getTrackById(DOMString id);

           attribute EventHandler onchange;
           attribute EventHandler onaddtrack;
           attribute EventHandler onremovetrack;
};

interface AudioTrack {
  readonly attribute DOMString id;
  readonly attribute DOMString kind;
  readonly attribute DOMString label;
  readonly attribute DOMString language;
           attribute boolean enabled;
};

interface VideoTrackList : EventTarget {
  readonly attribute unsigned long length;
  getter VideoTrack (unsigned long index);
  VideoTrack? getTrackById(DOMString id);
  readonly attribute long selectedIndex;

           attribute EventHandler onchange;
           attribute EventHandler onaddtrack;
           attribute EventHandler onremovetrack;
};

interface VideoTrack {
  readonly attribute DOMString id;
  readonly attribute DOMString kind;
  readonly attribute DOMString label;
  readonly attribute DOMString language;
           attribute boolean selected;
};
media . audioTracks . length
media . videoTracks . length

Returns the number of tracks in the list.

audioTrack = media . audioTracks[index]
videoTrack = media . videoTracks[index]

Returns the specified AudioTrack or VideoTrack object.

audioTrack = media . audioTracks . getTrackById( id )
videoTrack = media . videoTracks . getTrackById( id )

Returns the AudioTrack or VideoTrack object with the given identifier, or null if no track has that identifier.

audioTrack . id
videoTrack . id

Returns the ID of the given track. This is the ID that can be used with a fragment identifier if the format supports the Media Fragments URI syntax, and that can be used with the getTrackById() method. [MEDIAFRAG]

audioTrack . kind
videoTrack . kind

Returns the category the given track falls into. The possible track categories are given below.

audioTrack . label
videoTrack . label

Returns the label of the given track, if known, or the empty string otherwise.

audioTrack . language
videoTrack . language

Returns the language of the given track, if known, or the empty string otherwise.

audioTrack . enabled [ = value ]

Returns true if the given track is active, and false otherwise.

Can be set, to change whether the track is enabled or not. If multiple audio tracks are enabled simultaneously, they are mixed.

media . videoTracks . selectedIndex

Returns the index of the currently selected track, if any, or −1 otherwise.

videoTrack . selected [ = value ]

Returns true if the given track is active, and false otherwise.

Can be set, to change whether the track is selected or not. Either zero or one video track is selected; selecting a new track while a previous one is selected will unselect the previous one.

An AudioTrackList object represents a dynamic list of zero or more audio tracks, of which zero or more can be enabled at a time. Each audio track is represented by an AudioTrack object.

A VideoTrackList object represents a dynamic list of zero or more video tracks, of which zero or one can be selected at a time. Each video track is represented by a VideoTrack object.

Tracks in AudioTrackList and VideoTrackList objects must be consistently ordered. If the media resource is in a format that defines an order, then that order must be used; otherwise, the order must be the relative order in which the tracks are declared in the media resource. The order used is called the natural order of the list.

Each track in a TrackList thus has an index; the first has the index 0, and each subsequent track is numbered one higher than the previous one. If a media resource dynamically adds or removes audio or video tracks, then the indices of the tracks will change dynamically. If the media resource changes entirely, then all the previous tracks will be removed and replaced with new tracks.

The AudioTrackList.length and VideoTrackList.length attributes must return the number of tracks represented by their objects at the time of getting.

The supported property indices of AudioTrackList and VideoTrackList objects at any instant are the numbers from zero to the number of tracks represented by the respective object minus one, if any tracks are represented. If an AudioTrackList or VideoTrackList object represents no tracks, it has no supported property indices.

To determine the value of an indexed property for a given index index in an AudioTrackList or VideoTrackList object list, the user agent must return the AudioTrack or VideoTrack object that represents the indexth track in list.

The AudioTrackList.getTrackById(id) and VideoTrackList.getTrackById(id) methods must return the first AudioTrack or VideoTrack object (respectively) in the AudioTrack or VideoTrack object (respectively) whose identifier is equal to the value of the id argument (in the natural order of the list, as defined above). When no tracks match the given argument, the methods must return null.

The AudioTrack and VideoTrack objects represent specific tracks of a media resource. Each track can have an identifier, category, label, and language. These aspects of a track are permanent for the lifetime of the track; even if a track is removed from a media resource's AudioTrackList or VideoTrackList objects, those aspects do not change.

In addition, AudioTrack objects can each be enabled or disabled; this is the audio track's enabled state. When an AudioTrack is created, its enabled state must be set to false (disabled). The resource fetch algorithm can override this.

Similarly, a single VideoTrack object per VideoTrackList object can be selected, this is the video track's selection state. When a VideoTrack is created, its selection state must be set to false (not selected). The resource fetch algorithm can override this.

The AudioTrack.id and VideTrack.id attributes must return the identifier of the track, if it has one, or the empty string otherwise. If the media resource is in a format that supports the Media Fragments URI fragment identifier syntax, the identifier returned for a particular track must be the same identifier that would enable the track if used as the name of a track in the track dimension of such a fragment identifier. [MEDIAFRAG]

The AudioTrack.kind and VideoTrack.kind attributes must return the category of the track, if it has one, or the empty string otherwise.

The category of a track is the string given in the first column of the table below that is the most appropriate for the track based on the definitions in the table's second and third columns, as determined by the metadata included in the track in the media resource. The cell in the third column of a row says what the category given in the cell in the first column of that row applies to; a category is only appropriate for an audio track if it applies to audio tracks, and a category is only appropriate for video tracks if it applies to video tracks. Categories must only be returned for AudioTrack objects if they are appropriate for audio, and must only be returned for VideoTrack objects if they are appropriate for video.

For Ogg files, the Role header of the track gives the relevant metadata. For DASH media resources, the Role element conveys the information. For WebM, only the FlagDefault element currently maps to a value. [OGGROLE] [DASH] [WEBMCG]

Return values for AudioTrack.kind() and VideoTrack.kind()
Category Definition Applies to... Examples
"alternative" A possible alternative to the main track, e.g. a different take of a song (audio), or a different angle (video). Audio and video. Ogg: "audio/alternate" or "video/alternate"; DASH: "alternate" without "main" and "commentary" roles, and, for audio, without the "dub" role (other roles ignored).
"captions" A version of the main video track with captions burnt in. (For legacy content; new content would use text tracks.) Video only. DASH: "caption" and "main" roles together (other roles ignored).
"description" An audio description of a video track. Audio only. Ogg: "audio/audiodesc".
"main" The primary audio or video track. Audio and video. Ogg: "audio/main" or "video/main"; WebM: the "FlagDefault" element is set; DASH: "main" role without "caption", "subtitle", and "dub" roles (other roles ignored).
"sign" A sign-language interpretation of an audio track. Video only. Ogg: "video/sign".
"subtitles" A version of the main video track with subtitles burnt in. (For legacy content; new content would use text tracks.) Video only. DASH: "subtitle" and "main" roles together (other roles ignored).
"translation" A translated version of the main audio track. Audio only. Ogg: "audio/dub". DASH: "dub" and "main" roles together (other roles ignored).
"commentary" Commentary on the primary audio or video track, e.g. a director's commentary. Audio and video. DASH: "commentary" role without "main" role (other roles ignored).
"" (empty string) No explicit kind, or the kind given by the track's metadata is not recognised by the user agent. Audio and video. Any other track type, track role, or combination of track roles not described above.

The AudioTrack.label and VideoTrack.label attributes must return the label of the track, if it has one, or the empty string otherwise.

The AudioTrack.language and VideoTrack.language attributes must return the BCP 47 language tag of the language of the track, if it has one, or the empty string otherwise. If the user agent is not able to express that language as a BCP 47 language tag (for example because the language information in the media resource's format is a free-form string without a defined interpretation), then the method must return the empty string, as if the track had no language.

The AudioTrack.enabled attribute, on getting, must return true if the track is currently enabled, and false otherwise. On setting, it must enable the track if the new value is true, and disable it otherwise. (If the track is no longer in an AudioTrackList object, then the track being enabled or disabled has no effect beyond changing the value of the attribute on the AudioTrack object.)

Whenever an audio track in an AudioTrackList is enabled or disabled, the user agent must queue a task to fire a simple event named change at the AudioTrackList object.

The VideoTrackList.selectedIndex attribute must return the index of the currently selected track, if any. If the VideoTrackList object does not currently represent any tracks, or if none of the tracks are selected, it must instead return −1.

The VideoTrack.selected attribute, on getting, must return true if the track is currently selected, and false otherwise. On setting, it must select the track if the new value is true, and unselect it otherwise. If the track is in a VideoTrackList, then all the other VideoTrack objects in that list must be unselected. (If the track is no longer in a VideoTrackList object, then the track being selected or unselected has no effect beyond changing the value of the attribute on the VideoTrack object.)

Whenever a track in a VideoTrackList that was previously not selected is selected, the user agent must queue a task to fire a simple event named change at the VideoTrackList object.


The following are the event handlers (and their corresponding event handler event types) that must be supported, as IDL attributes, by all objects implementing the AudioTrackList and VideoTrackList interfaces:

Event handler Event handler event type
onchange change
onaddtrack addtrack
onremovetrack removetrack

The task source for the tasks listed in this section is the DOM manipulation task source.

4.8.10.10.2 Selecting specific audio and video tracks declaratively

The audioTracks and videoTracks attributes allow scripts to select which track should play, but it is also possible to select specific tracks declaratively, by specifying particular tracks in the fragment identifier of the URL of the media resource. The format of the fragment identifier depends on the MIME type of the media resource. [RFC2046] [RFC3986]

In this example, a video that uses a format that supports the Media Fragments URI fragment identifier syntax is embedded in such a way that the alternative angles labeled "Alternative" are enabled instead of the default video track. [MEDIAFRAG]

<video src="myvideo#track=Alternative"></video>
4.8.10.11 Synchronising multiple media elements
4.8.10.11.1 Introduction

Each media element can have a MediaController. A MediaController is an object that coordinates the playback of multiple media elements, for instance so that a sign-language interpreter track can be overlaid on a video track, with the two being kept in sync.

By default, a media element has no MediaController. An implicit MediaController can be assigned using the mediagroup content attribute. An explicit MediaController can be assigned directly using the controller IDL attribute.

Media elements with a MediaController are said to be slaved to their controller. The MediaController modifies the playback rate and the playback volume of each of the media elements slaved to it, and ensures that when any of its slaved media elements unexpectedly stall, the others are stopped at the same time.

When a media element is slaved to a MediaController, its playback rate is fixed to that of the other tracks in the same MediaController, and any looping is disabled.

4.8.10.11.2 Media controllers
enum MediaControllerPlaybackState { "waiting", "playing", "ended" };
[Constructor]
interface MediaController {
  readonly attribute unsigned short readyState; // uses HTMLMediaElement.readyState's values

  readonly attribute TimeRanges buffered;
  readonly attribute TimeRanges seekable;
  readonly attribute unrestricted double duration;
           attribute double currentTime;

  readonly attribute boolean paused;
  readonly attribute MediaControllerPlaybackState playbackState;
  readonly attribute TimeRanges played;
  void play();
  void pause();

           attribute double defaultPlaybackRate;
           attribute double playbackRate;

           attribute double volume;
           attribute boolean muted;

           attribute EventHandler onemptied;
           attribute EventHandler onloadedmetadata;
           attribute EventHandler onloadeddata;
           attribute EventHandler oncanplay;
           attribute EventHandler oncanplaythrough;
           attribute EventHandler onplaying;
           attribute EventHandler onended;
           attribute EventHandler onwaiting;

           attribute EventHandler ondurationchange;
           attribute EventHandler ontimeupdate;
           attribute EventHandler onplay;
           attribute EventHandler onpause;
           attribute EventHandler onratechange;
           attribute EventHandler onvolumechange;
};
controller = new MediaController()

Returns a new MediaController object.

media . controller [ = controller ]

Returns the current MediaController for the media element, if any; returns null otherwise.

Can be set, to set an explicit MediaController. Doing so removes the mediagroup attribute, if any.

controller . readyState

Returns the state that the MediaController was in the last time it fired events as a result of reporting the controller state. The values of this attribute are the same as for the readyState attribute of media elements.

controller . buffered

Returns a TimeRanges object that represents the intersection of the time ranges for which the user agent has all relevant media data for all the slaved media elements.

controller . seekable

Returns a TimeRanges object that represents the intersection of the time ranges into which the user agent can seek for all the slaved media elements.

controller . duration

Returns the difference between the earliest playable moment and the latest playable moment (not considering whether the data in question is actually buffered or directly seekable, but not including time in the future for infinite streams). Will return zero if there is no media.

controller . currentTime [ = value ]

Returns the current playback position, in seconds, as a position between zero time and the current duration.

Can be set, to seek to the given time.

controller . paused

Returns true if playback is paused; false otherwise. When this attribute is true, any media element slaved to this controller will be stopped.

controller . playbackState

Returns the state that the MediaController was in the last time it fired events as a result of reporting the controller state. The value of this attribute is either "playing", indicating that the media is actively playing, "ended", indicating that the media is not playing because playback has reached the end of all the slaved media elements, or "waiting", indicating that the media is not playing for some other reason (e.g. the MediaController is paused).

controller . play()

Sets the paused attribute to false.

controller . pause()

Sets the paused attribute to true.

controller . played

Returns a TimeRanges object that represents the union of the time ranges in all the slaved media elements that have been played.

controller . defaultPlaybackRate [ = value ]

Returns the default rate of playback.

Can be set, to change the default rate of playback.

This default rate has no direct effect on playback, but if the user switches to a fast-forward mode, when they return to the normal playback mode, it is expected that rate of playback (playbackRate) will be returned to this default rate.

controller . playbackRate [ = value ]

Returns the current rate of playback.

Can be set, to change the rate of playback.

controller . volume [ = value ]

Returns the current playback volume multiplier, as a number in the range 0.0 to 1.0, where 0.0 is the quietest and 1.0 the loudest.

Can be set, to change the volume multiplier.

Throws an IndexSizeError if the new value is not in the range 0.0 .. 1.0.

controller . muted [ = value ]

Returns true if all audio is muted (regardless of other attributes either on the controller or on any media elements slaved to this controller), and false otherwise.

Can be set, to change whether the audio is muted or not.

A media element can have a current media controller, which is a MediaController object. When a media element is created without a mediagroup attribute, it does not have a current media controller. (If it is created with such an attribute, then that attribute initializes the current media controller, as defined below.)

The slaved media elements of a MediaController are the media elements whose current media controller is that MediaController. All the slaved media elements of a MediaController must use the same clock for their definition of their media timeline's unit time.


The controller attribute on a media element, on getting, must return the element's current media controller, if any, or null otherwise. On setting, the user agent must run the following steps:

  1. Let m be the media element in question.

  2. Let old controller be m's current media controller, if it currently has one, and null otherwise.

  3. Let new controller be null.

  4. Let m have no current media controller, if it currently has one.

  5. Remove the element's mediagroup content attribute, if any.

  6. If the new value is null, then jump to the update controllers step below.

  7. Let m's current media controller be the new value.

  8. Let new controller be m's current media controller.

  9. Bring the media element up to speed with its new media controller.

  10. Update controllers: If old controller and new controller are the same (whether both null or both the same controller) then abort these steps.

  11. If old controller is not null and still has one or more slaved media elements, then report the controller state for old controller.

  12. If new controller is not null, then report the controller state for new controller.


The MediaController() constructor, when invoked, must return a newly created MediaController object.


The readyState attribute must return the value to which it was most recently set. When the MediaController object is created, the attribute must be set to the value 0 (HAVE_NOTHING). The value is updated by the report the controller state algorithm below.

The seekable attribute must return a new static normalized TimeRanges object that represents the intersection of the ranges of the media resources of the slaved media elements that the user agent is able to seek to, at the time the attribute is evaluated.

The buffered attribute must return a new static normalized TimeRanges object that represents the intersection of the ranges of the media resources of the slaved media elements that the user agent has buffered, at the time the attribute is evaluated. Users agents must accurately determine the ranges available, even for media streams where this can only be determined by tedious inspection.

The duration attribute must return the media controller duration.

Every 15 to 250ms, or whenever the MediaController's media controller duration changes, whichever happens least often, the user agent must queue a task to fire a simple event named durationchange at the MediaController. If the MediaController's media controller duration decreases such that the media controller position is greater than the media controller duration, the user agent must immediately seek the media controller to media controller duration.

The currentTime attribute must return the media controller position on getting, and on setting must seek the media controller to the new value.

Every 15 to 250ms, or whenever the MediaController's media controller position changes, whichever happens least often, the user agent must queue a task to fire a simple event named timeupdate at the MediaController.


When a MediaController is created it is a playing media controller. It can be changed into a paused media controller and back either via the user agent's user interface (when the element is exposing a user interface to the user) or by script using the APIs defined in this section (see below).

The paused attribute must return true if the MediaController object is a paused media controller, and false otherwise.

When the pause() method is invoked, if the MediaController is a playing media controller then the user agent must change the MediaController into a paused media controller, queue a task to fire a simple event named pause at the MediaController, and then report the controller state of the MediaController.

When the play() method is invoked, if the MediaController is a paused media controller, the user agent must change the MediaController into a playing media controller, queue a task to fire a simple event named play at the MediaController, and then report the controller state of the MediaController.

The playbackState attribute must return the value to which it was most recently set. When the MediaController object is created, the attribute must be set to the value "waiting". The value is updated by the report the controller state algorithm below.

The played attribute must return a new static normalized TimeRanges object that represents the union of the ranges of points on the media timelines of the media resources of the slaved media elements that the user agent has so far reached through the usual monotonic increase of their current playback positions during normal playback, at the time the attribute is evaluated.


A MediaController has a media controller default playback rate and a media controller playback rate, which must both be set to 1.0 when the MediaController object is created.

The defaultPlaybackRate attribute, on getting, must return the MediaController's media controller default playback rate, and on setting, must set the MediaController's media controller default playback rate to the new value, then queue a task to fire a simple event named ratechange at the MediaController.

The playbackRate attribute, on getting, must return the MediaController's media controller playback rate, and on setting, must set the MediaController's media controller playback rate to the new value, then queue a task to fire a simple event named ratechange at the MediaController.


A MediaController has a media controller volume multiplier, which must be set to 1.0 when the MediaController object is created, and a media controller mute override, much must initially be false.

The volume attribute, on getting, must return the MediaController's media controller volume multiplier, and on setting, if the new value is in the range 0.0 to 1.0 inclusive, must set the MediaController's media controller volume multiplier to the new value and queue a task to fire a simple event named volumechange at the MediaController. If the new value is outside the range 0.0 to 1.0 inclusive, then, on setting, an IndexSizeError exception must be thrown instead.

The muted attribute, on getting, must return the MediaController's media controller mute override, and on setting, must set the MediaController's media controller mute override to the new value and queue a task to fire a simple event named volumechange at the MediaController.


The media resources of all the slaved media elements of a MediaController have a defined temporal relationship which provides relative offsets between the zero time of each such media resource: for media resources with a timeline offset, their relative offsets are the difference between their timeline offset; the zero times of all the media resources without a timeline offset are not offset from each other (i.e. the origins of their timelines are cotemporal); and finally, the zero time of the media resource with the earliest timeline offset (if any) is not offset from the zero times of the media resources without a timeline offset (i.e. the origins of media resources without a timeline offset are further cotemporal with the earliest defined point on the timeline of the media resource with the earliest timeline offset).

The media resource end position of a media resource in a media element is defined as follows: if the media resource has a finite and known duration, the media resource end position is the duration of the media resource's timeline (the last defined position on that timeline); otherwise, the media resource's duration is infinite or unknown, and the media resource end position is the time of the last frame of media data currently available for that media resource.

Each MediaController also has its own defined timeline. On this timeline, all the media resources of all the slaved media elements of the MediaController are temporally aligned according to their defined offsets. The media controller duration of that MediaController is the time from the earliest earliest possible position, relative to this MediaController timeline, of any of the media resources of the slaved media elements of the MediaController, to the time of the latest media resource end position of the media resources of the slaved media elements of the MediaController, again relative to this MediaController timeline.

Each MediaController has a media controller position. This is the time on the MediaController's timeline at which the user agent is trying to play the slaved media elements. When a MediaController is created, its media controller position is initially zero.

When the user agent is to bring a media element up to speed with its new media controller, it must seek that media element to the MediaController's media controller position relative to the media element's timeline.

When the user agent is to seek the media controller to a particular new playback position, it must follow these steps:

  1. If the new playback position is less than zero, then set it to zero.

  2. If the new playback position is greater than the media controller duration, then set it to the media controller duration.

  3. Set the media controller position to the new playback position.

  4. Seek each slaved media element to the new playback position relative to the media element timeline.

A MediaController is a blocked media controller if the MediaController is a paused media controller, or if any of its slaved media elements are blocked media elements, or if any of its slaved media elements whose autoplaying flag is true still have their paused attribute set to true, or if all of its slaved media elements have their paused attribute set to true.

A media element is blocked on its media controller if the MediaController is a blocked media controller, or if its media controller position is either before the media resource's earliest possible position relative to the MediaController's timeline or after the end of the media resource relative to the MediaController's timeline.

When a MediaController is not a blocked media controller and it has at least one slaved media element whose Document is a fully active Document, the MediaController's media controller position must increase monotonically at media controller playback rate units of time on the MediaController's timeline per unit time of the clock used by its slaved media elements.

When the zero point on the timeline of a MediaController moves relative to the timelines of the slaved media elements by a time difference ΔT, the MediaController's media controller position must be decremented by ΔT.

In some situations, e.g. when playing back a live stream without buffering anything, the media controller position would increase monotonically as described above at the same rate as the ΔT described in the previous paragraph decreases it, with the end result that for all intents and purposes, the media controller position would appear to remain constant (probably with the value 0).


A MediaController has a most recently reported readiness state, which is a number from 0 to 4 derived from the numbers used for the media element readyState attribute, and a most recently reported playback state, which is either playing, waiting, or ended.

When a MediaController is created, its most recently reported readiness state must be set to 0, and its most recently reported playback state must be set to waiting.

When a user agent is required to report the controller state for a MediaController, the user agent must run the following steps:

  1. If the MediaController has no slaved media elements, let new readiness state be 0.

    Otherwise, let it have the lowest value of the readyState IDL attributes of all of its slaved media elements.

  2. If the MediaController's most recently reported readiness state is less than the new readiness state, then run these substeps:

    1. Let next state be the MediaController's most recently reported readiness state.

    2. Loop: Increment next state by one.

    3. Queue a task to run the following steps:

      1. Set the MediaController's readyState attribute to the value next state.

      2. Fire a simple event at the MediaController object, whose name is the event name corresponding to the value of next state given in the table below.

    4. If next state is less than new readiness state, then return to the step labeled loop.

    Otherwise, if the MediaController's most recently reported readiness state is greater than new readiness state then queue a task to fire a simple event at the MediaController object, whose name is the event name corresponding to the value of new readiness state given in the table below.

    Value of new readiness state Event name
    0 emptied
    1 loadedmetadata
    2 loadeddata
    3 canplay
    4 canplaythrough
  3. Let the MediaController's most recently reported readiness state be new readiness state.

  4. Initialize new playback state by setting it to the state given for the first matching condition from the following list:

    If the MediaController has no slaved media elements
    Let new playback state be waiting.
    If all of the MediaController's slaved media elements have ended playback and the media controller playback rate is positive or zero
    Let new playback state be ended.
    If the MediaController is a blocked media controller
    Let new playback state be waiting.
    Otherwise
    Let new playback state be playing.
  5. If the MediaController's most recently reported playback state is not equal to new playback state and the new playback state is ended, then queue a task that, if the MediaController object is a playing media controller, and all of the MediaController's slaved media elements have still ended playback, and the media controller playback rate is still positive or zero, changes the MediaController object to a paused media controller and then fires a simple event named pause at the MediaController object.

  6. If the MediaController's most recently reported playback state is not equal to new playback state then queue a task to run the following steps:

    1. Set the MediaController's playbackState attribute to the value given in the second column of the row of the following table whose first column contains the new playback state.

    2. Fire a simple event at the MediaController object, whose name is the value given in the third column of the row of the following table whose first column contains the new playback state.

    New playback state New value for playbackState Event name
    playing "playing" playing
    waiting "waiting" waiting
    ended "ended" ended
  7. Let the MediaController's most recently reported playback state be new playback state.


The following are the event handlers (and their corresponding event handler event types) that must be supported, as IDL attributes, by all objects implementing the MediaController interface:

Event handler Event handler event type
onemptied emptied
onloadedmetadata loadedmetadata
onloadeddata loadeddata
oncanplay canplay
oncanplaythrough canplaythrough
onplaying playing
onended ended
onwaiting waiting
ondurationchange durationchange
ontimeupdate timeupdate
onplay play
onpause pause
onratechange ratechange
onvolumechange volumechange

The task source for the tasks listed in this section is the DOM manipulation task source.

4.8.10.11.3 Assigning a media controller declaratively

The mediagroup content attribute on media elements can be used to link multiple media elements together by implicitly creating a MediaController. The value is text; media elements with the same value are automatically linked by the user agent.

When a media element is created with a mediagroup attribute, and when a media element's mediagroup attribute is set, changed, or removed, the user agent must run the following steps:

  1. Let m be the media element in question.

  2. Let old controller be m's current media controller, if it currently has one, and null otherwise.

  3. Let new controller be null.

  4. Let m have no current media controller, if it currently has one.

  5. If m's mediagroup attribute is being removed, then jump to the update controllers step below.

  6. If there is another media element whose Document is the same as m's Document (even if one or both of these elements are not actually in the Document), and which also has a mediagroup attribute, and whose mediagroup attribute has the same value as the new value of m's mediagroup attribute, then let controller be that media element's current media controller.

    Otherwise, let controller be a newly created MediaController.

  7. Let m's current media controller be controller.

  8. Let new controller be m's current media controller.

  9. Bring the media element up to speed with its new media controller.

  10. Update controllers: If old controller and new controller are the same (whether both null or both the same controller) then abort these steps.

  11. If old controller is not null and still has one or more slaved media elements, then report the controller state for old controller.

  12. If new controller is not null, then report the controller state for new controller.

The mediaGroup IDL attribute on media elements must reflect the mediagroup content attribute.

Multiple media elements referencing the same media resource will share a single network request. This can be used to efficiently play two (video) tracks from the same media resource in two different places on the screen. Used with the mediagroup attribute, these elements can also be kept synchronised.

In this example, a sign-languge interpreter track from a movie file is overlaid on the primary video track of that same video file using two video elements, some CSS, and an implicit MediaController:

<article>
 <style scoped>
  div { margin: 1em auto; position: relative; width: 400px; height: 300px; }
  video { position; absolute; bottom: 0; right: 0; }
  video:first-child { width: 100%; height: 100%; }
  video:last-child { width: 30%; }
 </style>
 <div>
  <video src="movie.vid#track=Video&amp;track=English" autoplay controls mediagroup=movie></video>
  <video src="movie.vid#track=sign" autoplay mediagroup=movie></video>
 </div>
</article>
4.8.10.12 Timed text tracks
4.8.10.12.1 Text track model

A media element can have a group of associated text tracks, known as the media element's list of text tracks. The text tracks are sorted as follows:

  1. The text tracks corresponding to track element children of the media element, in tree order.
  2. Any text tracks added using the addTextTrack() method, in the order they were added, oldest first.
  3. Any media-resource-specific text tracks (text tracks corresponding to data in the media resource), in the order defined by the media resource's format specification.

A text track consists of:

The kind of text track

This decides how the track is handled by the user agent. The kind is represented by a string. The possible strings are:

The kind of track can change dynamically, in the case of a text track corresponding to a track element.

A label

This is a human-readable string intended to identify the track for the user. In certain cases, the label might be generated automatically.

The label of a track can change dynamically, in the case of a text track corresponding to a track element or in the case of an automatically-generated label whose value depends on variable factors such as the user's preferred user interface language.

A language

This is a string (a BCP 47 language tag) representing the language of the text track's cues. [BCP47]

The language of a text track can change dynamically, in the case of a text track corresponding to a track element.

A readiness state

One of the following:

Not loaded

Indicates that the text track's cues have not been obtained.

Loading

Indicates that the text track is loading and there have been no fatal errors encountered so far. Further cues might still be added to the track by the parser.

Loaded

Indicates that the text track has been loaded with no fatal errors.

Failed to load

Indicates that the text track was enabled, but when the user agent attempted to obtain it, this failed in some way (e.g. URL could not be resolved, network error, unknown text track format). Some or all of the cues are likely missing and will not be obtained.

The readiness state of a text track changes dynamically as the track is obtained.

A mode

One of the following:

Disabled

Indicates that the text track is not active. Other than for the purposes of exposing the track in the DOM, the user agent is ignoring the text track. No cues are active, no events are fired, and the user agent will not attempt to obtain the track's cues.

Hidden

Indicates that the text track is active, but that the user agent is not actively displaying the cues. If no attempt has yet been made to obtain the track's cues, the user agent will perform such an attempt momentarily. The user agent is maintaining a list of which cues are active, and events are being fired accordingly.

Showing
Showing by default

Indicates that the text track is active. If no attempt has yet been made to obtain the track's cues, the user agent will perform such an attempt momentarily. The user agent is maintaining a list of which cues are active, and events are being fired accordingly. In addition, for text tracks whose kind is subtitles or captions, the cues are being overlaid on the video as appropriate; for text tracks whose kind is descriptions, the user agent is making the cues available to the user in a non-visual fashion; and for text tracks whose kind is chapters, the user agent is making available to the user a mechanism by which the user can navigate to any point in the media resource by selecting a cue.

The showing by default state is used in conjunction with the default attribute on track elements to indicate that the text track was enabled due to that attribute. This allows the user agent to override the state if a later track is discovered that is more appropriate per the user's preferences.

A list of zero or more cues

A list of text track cues, along with rules for updating the text track rendering. For example, for WebVTT, the rules for updating the display of WebVTT text tracks. [WEBVTT]

The list of cues of a text track can change dynamically, either because the text track has not yet been loaded or is still loading, or due to DOM manipulation.

Each text track has a corresponding TextTrack object.

The text tracks of a media element are ready if all the text tracks whose mode was not in the disabled state when the element's resource selection algorithm last started now have a text track readiness state of loaded or failed to load.


A text track cue is the unit of time-sensitive data in a text track, corresponding for instance for subtitles and captions to the text that appears at a particular time and disappears at another time.

Each text track cue consists of:

An identifier

An arbitrary string.

A start time

The time, in seconds and fractions of a second, that describes the beginning of the range of the media data to which the cue applies.

An end time

The time, in seconds and fractions of a second, that describes the end of the range of the media data to which the cue applies.

A pause-on-exit flag

A boolean indicating whether playback of the media resource is to pause when the end of the range to which the cue applies is reached.

A writing direction

A writing direction, either horizontal (a line extends horizontally and is positioned vertically, with consecutive lines displayed below each other), vertical growing left (a line extends vertically and is positioned horizontally, with consecutive lines displayed to the left of each other), or vertical growing right (a line extends vertically and is positioned horizontally, with consecutive lines displayed to the right of each other).

If the writing direction is horizontal, then line position percentages are relative to the height of the video, and text position and size percentages are relative to the width of the video.

Otherwise, line position percentages are relative to the width of the video, and text position and size percentages are relative to the height of the video.

A snap-to-lines flag

A boolean indicating whether the line's position is a line position (positioned to a multiple of the line dimensions of the first line of the cue), or whether it is a percentage of the dimension of the video.

A line position

Either a number giving the position of the lines of the cue, to be interpreted as defined by the writing direction and snap-to-lines flag of the cue, or the special value auto, which means the position is to depend on the other active tracks.

A text track cue has a text track cue computed line position whose value is defined in terms of the other aspects of the cue. If the text track cue line position is numeric, then that is the text track cue computed line position. Otherwise, the text track cue line position is the special value auto; if the text track cue snap-to-lines flag of the text track cue is not set, the text track cue computed line position is the value 100; otherwise, it is the value returned by the following algorithm:

  1. Let cue be the text track cue.

  2. If cue is not associated with a text track, return −1 and abort these steps.

  3. Let track be the text track that the cue is associated with.

  4. Let n be the number of text tracks whose text track mode is showing or showing by default and that are in the media element's list of text tracks before track.

  5. Increment n by one.

  6. Negate n.

  7. Return n.

A text position

A number giving the position of the text of the cue within each line, to be interpreted as a percentage of the video, as defined by the writing direction.

A size

A number giving the size of the box within which the text of each line of the cue is to be aligned, to be interpreted as a percentage of the video, as defined by the writing direction.

An alignment

An alignment for the text of each line of the cue, either start alignment (the text is aligned towards its start side), middle alignment (the text is aligned centered between its start and end sides), end alignment (the text is aligned towards its end side). Which sides are the start and end sides depends on the Unicode bidirectional algorithm and the writing direction. [BIDI]

The text of the cue

The raw text of the cue, and rules for its interpretation, allowing the text to be rendered and converted to a DOM fragment.

Each text track cue has a corresponding TextTrackCue object, and can be associated with a particular text track. Once a text track cue is associated with a particular text track, the association is permanent. A text track cue's in-memory representation can be dynamically changed through this TextTrackCue API.

In addition, each text track cue has two pieces of dynamic information:

The active flag

This flag must be initially unset. The flag is used to ensure events are fired appropriately when the cue becomes active or inactive, and to make sure the right cues are rendered.

The user agent must synchronously unset this flag whenever the text track cue is removed from its text track's text track list of cues; whenever the text track itself is removed from its media element's list of text tracks or has its text track mode changed to disabled; and whenever the media element's readyState is changed back to HAVE_NOTHING. When the flag is unset in this way for one or more cues in text tracks that were showing or showing by default prior to the relevant incident, the user agent must, after having unset the flag for all the affected cues, apply the rules for updating the text track rendering of those text tracks. For example, for text tracks based on WebVTT, the rules for updating the display of WebVTT text tracks. [WEBVTT]

The display state

This is used as part of the rendering model, to keep cues in a consistent position. It must initially be empty. Whenever the text track cue active flag is unset, the user agent must empty the text track cue display state.

The text track cues of a media element's text tracks are ordered relative to each other in the text track cue order, which is determined as follows: first group the cues by their text track, with the groups being sorted in the same order as their text tracks appear in the media element's list of text tracks; then, within each group, cues must be sorted by their start time, earliest first; then, any cues with the same start time must be sorted by their end time, latest first; and finally, any cues with identical end times must be sorted in the order they were created (so e.g. for cues from a WebVTT file, that would be the order in which the cues were listed in the file). [WEBVTT]

4.8.10.12.2 Sourcing in-band text tracks

A media-resource-specific text track is a text track that corresponds to data found in the media resource.

Rules for processing and rendering such data are defined by the relevant specifications, e.g. the specification of the video format if the media resource is a video.

When a media resource contains data that the user agent recognises and supports as being equivalent to a text track, the user agent runs the steps to expose a media-resource-specific text track with the relevant data, as follows.

  1. Associate the relevant data with a new text track and its corresponding new TextTrack object. The text track is a media-resource-specific text track.

  2. Set the new text track's kind, label, and language based on the semantics of the relevant data, as defined by the relevant specification.

  3. Populate the new text track's list of cues with the cues parsed so far, folllowing the guidelines for exposing cues, and begin updating it dynamically as necessary.

  4. Set the new text track's readiness state to loaded.

  5. Set the new text track's mode to the mode consistent with the user's preferences and the requirements of the relevant specification for the data.

  6. Leave the text track list of cues empty, and associate with it the rules for updating the text track rendering appropriate for the format in question.

  7. Add the new text track to the media element's list of text tracks.

  8. Fire an event with the name addtrack, that does not bubble and is not cancelable, and that uses the TrackEvent interface, with the track attribute initialized to the text track's TextTrack object, at the media element's textTracks attribute's TextTrackList object.

When a media element is to forget the media element's media-resource-specific text tracks, the user agent must remove from the media element's list of text tracks all the media-resource-specific text tracks.

4.8.10.12.3 Sourcing out-of-band text tracks

When a track element is created, it must be associated with a new text track (with its value set as defined below) and its corresponding new TextTrack object.

The text track kind is determined from the state of the element's kind attribute according to the following table; for a state given in a cell of the first column, the kind is the string given in the second column:

State String
Subtitles subtitles
Captions captions
Descriptions descriptions
Chapters chapters
Metadata metadata

The text track label is the element's track label.

The text track language is the element's track language, if any, or the empty string otherwise.

As the kind, label, and srclang attributes are set, changed, or removed, the text track must update accordingly, as per the definitions above.

Changes to the track URL are handled in the algorithm below.

The text track list of cues is initially empty. It is dynamically modified when the referenced file is parsed. Associated with the list are the rules for updating the text track rendering appropriate for the format in question; for WebVTT, this is the rules for updating the display of WebVTT text tracks. [WEBVTT]

When a track element's parent element changes and the new parent is a media element, then the user agent must add the track element's corresponding text track to the media element's list of text tracks, and then queue a task to fire an event with the name addtrack, that does not bubble and is not cancelable, and that uses the TrackEvent interface, with the track attribute initialized to the text track's TextTrack object, at the media element's textTracks attribute's TextTrackList object.

When a track element's parent element changes and the old parent was a media element, then the user agent must remove the track element's corresponding text track from the media element's list of text tracks, and then queue a task to fire an event with the name removetrack, that does not bubble and is not cancelable, and that uses the TrackEvent interface, with the track attribute initialized to the text track's TextTrack object, at the media element's textTracks attribute's TextTrackList object.

When a text track corresponding to a track element is added to a media element's list of text tracks, the user agent must set the text track mode appropriately, as determined by the following conditions:

If the text track kind is subtitles or captions and the user has indicated an interest in having a track with this text track kind, text track language, and text track label enabled, and there is no other text track in the media element's list of text tracks with a text track kind of either subtitles or captions whose text track mode is showing
If the text track kind is descriptions and the user has indicated an interest in having text descriptions with this text track language and text track label enabled, and there is no other text track in the media element's list of text tracks with a text track kind of descriptions whose text track mode is showing

Let the text track mode be showing.

If there is a text track in the media element's list of text tracks whose text track mode is showing by default, the user agent must furthermore change that text track's text track mode to hidden.

If the text track kind is chapters and the text track language is one that the user agent has reason to believe is appropriate for the user, and there is no other text track in the media element's list of text tracks with a text track kind of chapters whose text track mode is showing

Let the text track mode be showing.

If the track element has a default attribute specified, and there is no other text track in the media element's list of text tracks whose text track mode is showing or showing by default

Let the text track mode be showing by default.

Otherwise

Let the text track mode be disabled.

When a text track corresponding to a track element is created with text track mode set to hidden, showing, or showing by default, and when a text track corresponding to a track element is created with text track mode set to disabled and subsequently changes its text track mode to hidden, showing, or showing by default for the first time, the user agent must immediately and synchronously run the following algorithm. This algorithm interacts closely with the event loop mechanism; in particular, it has a synchronous section (which is triggered as part of the event loop algorithm). The step in that section is marked with ⌛.

  1. Set the text track readiness state to loading.

  2. Let URL be the track URL of the track element.

  3. Asynchronously run the remaining steps, while continuing with whatever task was responsible for creating the text track or changing the text track mode.

  4. Download: At this point, the text track is downloaded.

    If URL is not the empty string, perform a potentially CORS-enabled fetch of URL, with the mode being the state of the media element's crossorigin content attribute, the origin being the origin of the media element's Document, and the default origin behaviour set to fail.

    The resource obtained in this fashion, if any, contains the text track data. If any data is obtained, it is by definition CORS-same-origin (cross-origin resources that are not suitably CORS-enabled do not get this far).

    The tasks queued by the fetching algorithm on the networking task source to process the data as it is being fetched must determine the type of a the resource. If the type of the resource is not a supported text track format, the load will fail, as described below. Otherwise, the resource's data must be passed to the appropriate parser (e.g. the WebVTT parser) as it is received, with the text track list of cues being used for that parser's output. [WEBVTT]

    This specification does not currently say whether or how to check the MIME types of text tracks, or whether or how to perform file type sniffing using the actual file data. Implementors differ in their intentions on this matter and it is therefore unclear what the right solution is. In the absence of any requirement here, the HTTP specification's strict requirement to follow the Content-Type header prevails ("Content-Type specifies the media type of the underlying data." ... "If and only if the media type is not given by a Content-Type field, the recipient MAY attempt to guess the media type via inspection of its content and/or the name extension(s) of the URI used to identify the resource.").

    If the fetching algorithm fails for any reason (network error, the server returns an error code, a cross-origin check fails, etc), if URL is the empty string, or if the sniffed type of the resource is not a supported text track format, then queue a task to first change the text track readiness state to failed to load and then fire a simple event named error at the track element; and then, once that task is queued, move on to the step below labeled monitoring.

    If the fetching algorithm does not fail, then the final task that is queued by the networking task source must run the following steps after it has tried to parse the data:

    1. Change the text track readiness state to loaded.

    2. If the file was successfully processed, fire a simple event named load at the track element.

      If the file was not successfully processed, e.g. the format in question is an XML format and the file contained a well-formedness error that the XML specification requires be detected and reported to the application, then fire a simple event named error at the track element.

    3. Jump to the step below labeled monitoring.

    If, while the fetching algorithm is active, either:

    ...then the user agent must run the following steps:

    1. Abort the fetching algorithm, discarding any pending tasks generated by that algorithm.

    2. Let URL be the new track URL.

    3. Jump back to the top of the step labeled download.

    Until one of the above circumstances occurs, the user agent must remain on this step.

  5. Monitoring: Wait until the track URL is no longer equal to URL, at the same time as the text track mode is set to hidden, showing, or showing by default.

  6. Wait until the text track readiness state is no longer set to loading.

  7. Await a stable state. The synchronous section consists of the following step. (The step in the synchronous section is marked with ⌛.)

  8. ⌛ Set the text track readiness state to loading.

  9. End the synchronous section, continuing the remaining steps asynchronously.

  10. Jump to the step labeled download.

4.8.10.12.4 Guidelines for exposing cues in various formats as text track cues

How a specific format's text track cues are to be interpreted for the purposes of processing by an HTML user agent is defined by that format. In the absence of such a specification, this section provides some constraints within which implementations can attempt to consistently expose such formats.

To support the text track model of HTML, each unit of timed data is converted to a text track cue. Where the mapping of the format's features to the aspects of a text track cue as defined in this specification are not defined, implementations must ensure that the mapping is consistent with the definitions of the aspects of a text track cue as defined above, as well as with the following constraints:

The text track cue identifier

Should be set to the empty string if the format has no obvious analogue to a per-cue identifier.

The text track cue pause-on-exit flag

Should be set to false.

The text track cue writing direction

Should be set to horizontal if the concept of writing direction doesn't really apply (e.g. the cue consists of a bitmap image).

The text track cue snap-to-lines flag

Should be set to false unless the format uses a rendering and positioning model for cues that is largely consistent with the WebVTT cue text rendering rules.

The text track cue line position
The text track cue text position
The text track cue size
The text track cue alignment

If the format uses a rendering and positioning model for cues that can be largely simulated using the WebVTT cue text rendering rules, then these should be set to the values that would give the same effect for WebVTT cues. Otherwise, they should be set to zero.

4.8.10.12.5 Text track API
interface TextTrackList : EventTarget {
  readonly attribute unsigned long length;
  getter TextTrack (unsigned long index);

           attribute EventHandler onaddtrack;
           attribute EventHandler onremovetrack;
};
media . textTracks . length

Returns the number of text tracks associated with the media element (e.g. from track elements). This is the number of text tracks in the media element's list of text tracks.

media . textTracks[ n ]

Returns the TextTrack object representing the nth text track in the media element's list of text tracks.

track . track

Returns the TextTrack object representing the track element's text track.

A TextTrackList object represents a dynamically updating list of text tracks in a given order.

The textTracks attribute of media elements must return a TextTrackList object representing the TextTrack objects of the text tracks in the media element's list of text tracks, in the same order as in the list of text tracks. The same object must be returned each time the attribute is accessed. [WEBIDL]

The length attribute of a TextTrackList object must return the number of text tracks in the list represented by the TextTrackList object.

The supported property indices of a TextTrackList object at any instant are the numbers from zero to the number of text tracks in the list represented by the TextTrackList object minus one, if any. If there are no text tracks in the list, there are no supported property indices.

To determine the value of an indexed property of a TextTrackList object for a given index index, the user agent must return the indexth text track in the list represented by the TextTrackList object.


enum TextTrackMode { "disabled", "hidden", "showing" };
interface TextTrack : EventTarget {
  readonly attribute DOMString kind;
  readonly attribute DOMString label;
  readonly attribute DOMString language;

           attribute TextTrackMode mode;

  readonly attribute TextTrackCueList? cues;
  readonly attribute TextTrackCueList? activeCues;

  void addCue(TextTrackCue cue);
  void removeCue(TextTrackCue cue);

           attribute EventHandler oncuechange;
};
textTrack . kind

Returns the text track kind string.

textTrack . label

Returns the text track label.

textTrack . language

Returns the text track language string.

textTrack . mode [ = value ]

Returns the text track mode, represented by a string from the following list:

"disabled"

The text track disabled mode.

"hidden"

The text track hidden mode.

"showing"

The text track showing and showing by default modes.

Can be set, to change the mode.

textTrack . cues

Returns the text track list of cues, as a TextTrackCueList object.

textTrack . activeCues

Returns the text track cues from the text track list of cues that are currently active (i.e. that start before the current playback position and end after it), as a TextTrackCueList object.

textTrack . addCue( cue )

Adds the given cue to textTrack's text track list of cues.

Throws an exception if the argument is associated with another text track or already in the list of cues.

textTrack . removeCue( cue )

Removes the given cue from textTrack's text track list of cues.

Throws an exception if the argument is associated with another text track or not in the list of cues.

textTrack = media . addTextTrack( kind [, label [, language ] ] )

Creates and returns a new TextTrack object, which is also added to the media element's list of text tracks.

The kind attribute must return the text track kind of the text track that the TextTrack object represents.

The label attribute must return the text track label of the text track that the TextTrack object represents.

The language attribute must return the text track language of the text track that the TextTrack object represents.

The mode attribute, on getting, must return the string corresponding to the text track mode of the text track that the TextTrack object represents, as defined by the following list:

"disabled"
The text track disabled mode.
"hidden"
The text track hidden mode.
"showing"
The text track showing and showing by default modes.

On setting, if the new value isn't equal to what the attribute would currently return, the new value must be processed as follows:

If the new value is "disabled"

Set the text track mode of the text track that the TextTrack object represents to the text track disabled mode.

If the new value is "hidden"

Set the text track mode of the text track that the TextTrack object represents to the text track hidden mode.

If the new value is "showing"

Set the text track mode of the text track that the TextTrack object represents to the text track showing mode.

If the mode is showing by default, then there is no way to change it to showing without first changing it to disabled or hidden.

If the text track mode of the text track that the TextTrack object represents is not the text track disabled mode, then the cues attribute must return a live TextTrackCueList object that represents the subset of the text track list of cues of the text track that the TextTrack object represents whose end times occur at or after the earliest possible position when the script started, in text track cue order. Otherwise, it must return null. When an object is returned, the same object must be returned each time.

The earliest possible position when the script started is whatever the earliest possible position was the last time the event loop reached step 1.

If the text track mode of the text track that the TextTrack object represents is not the text track disabled mode, then the activeCues attribute must return a live TextTrackCueList object that represents the subset of the text track list of cues of the text track that the TextTrack object represents whose active flag was set when the script started, in text track cue order. Otherwise, it must return null. When an object is returned, the same object must be returned each time.

A text track cue's active flag was set when the script started if its text track cue active flag was set the last time the event loop reached step 1.


The addCue(cue) method of TextTrack objects, when invoked, must run the following steps:

  1. If the given cue is already associated with a text track other than the method's TextTrack object's text track, then throw an InvalidStateError exception and abort these steps.

  2. If the given cue is already listed in the method's TextTrack object's text track's text track list of cues, then throw an InvalidStateError exception.

  3. Associate cue with the method's TextTrack object's text track, if it is not currently associated with a text track.

  4. Add cue to the method's TextTrack object's text track's text track list of cues.

The removeCue(cue) method of TextTrack objects, when invoked, must run the following steps:

  1. If the given cue is not associated with the method's TextTrack object's text track, then throw an InvalidStateError exception.

  2. If the given cue is not currently listed in the method's TextTrack object's text track's text track list of cues, then throw a NotFoundError exception.

  3. Remove cue from the method's TextTrack object's text track's text track list of cues.


The addTextTrack(kind, label, language) method of media elements, when invoked, must run the following steps:

  1. If kind is not one of the following strings, then throw a SyntaxError exception and abort these steps:

  2. If the label argument was omitted, let label be the empty string.

  3. If the language argument was omitted, let language be the empty string.

  4. Create a new TextTrack object.

  5. Create a new text track corresponding to the new object, and set its text track kind to kind, its text track label to label, its text track language to language, its text track readiness state to the text track loaded state, its text track mode to the text track hidden mode, and its text track list of cues to an empty list. Associate the text track list of cues with the rules for updating the display of WebVTT text tracks as its rules for updating the text track rendering. [WEBVTT]

  6. Add the new text track to the media element's list of text tracks.

  7. Queue a task to fire an event with the name addtrack, that does not bubble and is not cancelable, and that uses the TrackEvent interface, with the track attribute initialized to the new text track's TextTrack object, at the media element's textTracks attribute's TextTrackList object.

  8. Return the new TextTrack object.

In this example, an audio element is used to play a specific sound-effect from a sound file containing many sound effects. A cue is used to pause the audio, so that it ends exactly at the end of the clip, even if the browser is busy running some script. If the page had relied on script to pause the audio, then the start of the next clip might be heard if the browser was not able to run the script at the exact time specified.

var sfx = new Audio('sfx.wav');
var sounds = sfx.addTextTrack('metadata');

// add sounds we care about
sounds.addCue(new TextTrackCue('dog bark', 12.783, 13.612, '', '', '', true));
sounds.addCue(new TextTrackCue('kitten mew', 13.612, 15.091, '', '', '', true));

function playSound(id) {
  sfx.currentTime = sounds.getCueById(id).startTime;
  sfx.play();
}

sfx.oncanplaythrough = function () {
  playSound('dog bark');
}
window.onbeforeunload = function () {
  playSound('kitten mew');
  return 'Are you sure you want to leave this awesome page?';
}

interface TextTrackCueList {
  readonly attribute unsigned long length;
  getter TextTrackCue (unsigned long index);
  TextTrackCue? getCueById(DOMString id);
};
cuelist . length

Returns the number of cues in the list.

cuelist[index]

Returns the text track cue with index index in the list. The cues are sorted in text track cue order.

cuelist . getCueById( id )

Returns the first text track cue (in text track cue order) with text track cue identifier id.

Returns null if none of the cues have the given identifier or if the argument is the empty string.

A TextTrackCueList object represents a dynamically updating list of text track cues in a given order.

The length attribute must return the number of cues in the list represented by the TextTrackCueList object.

The supported property indices of a TextTrackCueList object at any instant are the numbers from zero to the number of cues in the list represented by the TextTrackCueList object minus one, if any. If there are no cues in the list, there are no supported property indices.

To determine the value of an indexed property for a given index index, the user agent must return the indexth text track cue in the list represented by the TextTrackCueList object.

The getCueById(id) method, when called with an argument other than the empty string, must return the first text track cue in the list represented by the TextTrackCueList object whose text track cue identifier is id, if any, or null otherwise. If the argument is the empty string, then the method must return null.


[Constructor(double startTime, double endTime, DOMString text)]
interface TextTrackCue : EventTarget {
  readonly attribute TextTrack? track;

           attribute DOMString id;
           attribute double startTime;
           attribute double endTime;
           attribute boolean pauseOnExit;
           attribute DOMString vertical;
           attribute boolean snapToLines;
           attribute long line;
           attribute long position;
           attribute long size;
           attribute DOMString align;
           attribute DOMString text;
  DocumentFragment getCueAsHTML();

           attribute EventHandler onenter;
           attribute EventHandler onexit;
};
cue = new TextTrackCue( startTime, endTime, text )

Returns a new TextTrackCue object, for use with the addCue() method.

The startTime argument sets the text track cue start time.

The endTime argument sets the text track cue end time.

The text argument sets the text track cue text.

cue . track

Returns the TextTrack object to which this text track cue belongs, if any, or null otherwise.

cue . id [ = value ]

Returns the text track cue identifier.

Can be set.

cue . startTime [ = value ]

Returns the text track cue start time, in seconds.

Can be set.

cue . endTime [ = value ]

Returns the text track cue end time, in seconds.

Can be set.

cue . pauseOnExit [ = value ]

Returns true if the text track cue pause-on-exit flag is set, false otherwise.

Can be set.

cue . vertical [ = value ]

Returns a string representing the text track cue writing direction, as follows:

If it is horizontal

The empty string.

If it is vertical growing left

The string "rl".

If it is vertical growing right

The string "lr".

Can be set.

cue . snapToLines [ = value ]

Returns true if the text track cue snap-to-lines flag is set, false otherwise.

Can be set.

cue . line [ = value ]

Returns the text track cue line position. In the case of the value being auto, the appropriate default is returned.

Can be set.

cue . position [ = value ]

Returns the text track cue text position.

Can be set.

cue . size [ = value ]

Returns the text track cue size.

Can be set.

cue . align [ = value ]

Returns a string representing the text track cue alignment, as follows:

If it is start alignment

The string "start".

If it is middle alignment

The string "middle".

If it is end alignment

The string "end".

Can be set.

cue . text [ = value ]

Returns the text track cue text in raw unparsed form.

Can be set.

fragment = cue . getCueAsHTML()

Returns the text track cue text as a DocumentFragment of HTML elements and other DOM nodes.

The TextTrackCue(startTime, endTime, text) constructor, when invoked, must run the following steps:

  1. Create a new text track cue that is not associated with any text track. Let cue be that text track cue.

  2. Let cue's text track cue start time be the value of the startTime argument, interpreted as a time in seconds.

  3. Let cue's text track cue end time be the value of the endTime argument, interpreted as a time in seconds.

  4. Let cue's text track cue text be the value of the text argument, and let the rules for its interpretation be the WebVTT cue text parsing rules, the WebVTT cue text rendering rules, and the WebVTT cue text DOM construction rules. [WEBVTT]

  5. Let cue's text track cue identifier be the empty string.

  6. Let cue's text track cue pause-on-exit flag be false.

  7. Let cue's text track cue writing direction be horizontal.

  8. Let cue's text track cue snap-to-lines flag be true.

  9. Let cue's text track cue line position be auto.

  10. Let cue's text track cue text position be 50.

  11. Let cue's text track cue size be 100.

  12. Let cue's text track cue alignment be middle alignment.

  13. Return the TextTrackCue object representing cue.

The track attribute, on getting, must return the TextTrack object of the text track with which the text track cue that the TextTrackCue object represents is associated, if any; or null otherwise.

The id attribute, on getting, must return the text track cue identifier of the text track cue that the TextTrackCue object represents. On setting, the text track cue identifier must be set to the new value.

The startTime attribute, on getting, must return the text track cue start time of the text track cue that the TextTrackCue object represents, in seconds. On setting, the text track cue start time must be set to the new value, interpreted in seconds.

The endTime attribute, on getting, must return the text track cue end time of the text track cue that the TextTrackCue object represents, in seconds. On setting, the text track cue end time must be set to the new value, interpreted in seconds.

The pauseOnExit attribute, on getting, must return true if the text track cue pause-on-exit flag of the text track cue that the TextTrackCue object represents is set; or false otherwise. On setting, the text track cue pause-on-exit flag must be set if the new value is true, and must be unset otherwise.

The vertical attribute, on getting, must return the string from the second cell of the row in the table below whose first cell is the text track cue writing direction of the text track cue that the TextTrackCue object represents:

Text track cue writing direction direction value
Horizontal "" (the empty string)
Vertical growing left "rl"
Vertical growing right "lr"

On setting, the text track cue writing direction must be set to the value given in the first cell of the row in the table above whose second cell is a case-sensitive match for the new value, if any. If none of the values match, then the user agent must instead throw a SyntaxError exception.

The snapToLines attribute, on getting, must return true if the text track cue snap-to-lines flag of the text track cue that the TextTrackCue object represents is set; or false otherwise. On setting, the text track cue snap-to-lines flag must be set if the new value is true, and must be unset otherwise.

The line attribute, on getting, must return the text track cue computed line position of the text track cue that the TextTrackCue object represents. On setting, if the text track cue snap-to-lines flag is not set, and the new value is negative or greater than 100, then throw an IndexSizeError exception. Otherwise, set the text track cue line position to the new value.

There is no way to explicitly set the text track cue line position to the special default auto value.

The position attribute, on getting, must return the text track cue text position of the text track cue that the TextTrackCue object represents. On setting, if the new value is negative or greater than 100, then throw an IndexSizeError exception. Otherwise, set the text track cue text position to the new value.

The size attribute, on getting, must return the text track cue size of the text track cue that the TextTrackCue object represents. On setting, if the new value is negative or greater than 100, then throw an IndexSizeError exception. Otherwise, set the text track cue size to the new value.

The align attribute, on getting, must return the string from the second cell of the row in the table below whose first cell is the text track cue alignment of the text track cue that the TextTrackCue object represents:

Text track cue alignment align value
Start alignment "start"
Middle alignment "middle"
End alignment "end"

On setting, the text track cue alignment must be set to the value given in the first cell of the row in the table above whose second cell is a case-sensitive match for the new value, if any. If none of the values match, then the user agent must instead throw a SyntaxError exception.

The text attribute, on getting, must return the raw text track cue text of the text track cue that the TextTrackCue object represents. On setting, the text track cue text must be set to the new value.

The getCueAsHTML() method must convert the text track cue text to a DocumentFragment for the media element's Document, using the appropriate rules for doing so. For example, for WebVTT, those rules are the WebVTT cue text parsing rules and the WebVTT cue text DOM construction rules. [WEBVTT]

4.8.10.12.6 Text tracks describing chapters

Chapters are segments of a media resource with a given title. Chapters can be nested, in the same way that sections in a document outline can have subsections.

Each text track cue in a text track being used for describing chapters has three key features: the text track cue start time, giving the start time of the chapter, the text track cue end time, giving the end time of the chapter, and the text track cue text giving the chapter title.

The rules for constructing the chapter tree from a text track are as follows. They produce a potentially nested list of chapters, each of which have a start time, end time, title, and a list of nested chapters. This algorithm discards cues that do not correctly nest within each other, or that are out of order.

  1. Let list be a copy of the list of cues of the text track being processed.

  2. Remove from list any text track cue whose text track cue end time is before its text track cue start time.

  3. Let output be an empty list of chapters, where a chapter is a record consisting of a start time, an end time, a title, and a (potentially empty) list of nested chapters. For the purpose of this algorithm, each chapter also has a parent chapter.

  4. Let current chapter be a stand-in chapter whose start time is negative infinity, whose end time is positive infinity, and whose list of nested chapters is output. (This is just used to make the algorithm easier to describe.)

  5. Loop: If list is empty, jump to the step labeled end.

  6. Let current cue be the first cue in list, and then remove it from list.

  7. If current cue's text track cue start time is less than the start time of current chapter, then return to the step labeled loop.

  8. While current cue's text track cue start time is greater than or equal to current chapter's end time, let current chapter be current chapter's parent chapter.

  9. If current cue's text track cue end time is greater than the end time of current chapter, then return to the step labeled loop.

  10. Create a new chapter new chapter, whose start time is current cue's text track cue start time, whose end time is current cue's text track cue end time, whose title is current cue's text track cue text interpreted according to its rules for interpretation, and whose list of nested chapters is empty.

  11. Append new chapter to current chapter's list of nested chapters, and let current chapter be new chapter's parent.

  12. Let current chapter be new chapter.

  13. Return to the step labeled loop.

  14. End: Return output.

The following snippet of a WebVTT file shows how nested chapters can be marked up. The file describes three 50-minute chapters, "Astrophysics", "Computational Physics", and "General Relativity". The first has three subchapters, the second has four, and the third has two. [WEBVTT]

WEBVTT

00:00:00.000 --> 00:50:00.000
Astrophysics

00:00:00.000 --> 00:10:00.000
Introduction to Astrophysics

00:10:00.000 --> 00:45:00.000
The Solar System

00:00:00.000 --> 00:10:00.000
Coursework Description

00:50:00.000 --> 01:40:00.000
Computational Physics

00:50:00.000 --> 00:55:00.000
Introduction to Programming

00:55:00.000 --> 01:30:00.000
Data Structures

01:30:00.000 --> 01:35:00.000
Answers to Last Exam

01:35:00.000 --> 01:40:00.000
Coursework Description

01:40:00.000 --> 02:30:00.000
General Relativity

01:40:00.000 --> 02:00:00.000
Tensor Algebra

02:00:00.000 --> 02:30:00.000
The General Relativistic Field Equations
4.8.10.12.7 Event definitions

The following are the event handlers that (and their corresponding event handler event types) must be supported, as IDL attributes, by all objects implementing the TextTrackList interface:

Event handler Event handler event type
onaddtrack addtrack
onremovetrack removetrack

The following are the event handlers that (and their corresponding event handler event types) must be supported, as IDL attributes, by all objects implementing the TextTrack interface:

Event handler Event handler event type
oncuechange cuechange

The following are the event handlers that (and their corresponding event handler event types) must be supported, as IDL attributes, by all objects implementing the TextTrackCue interface:

Event handler Event handler event type
onenter enter
onexit exit
4.8.10.13 User interface

The controls attribute is a boolean attribute. If present, it indicates that the author has not provided a scripted controller and would like the user agent to provide its own set of controls.

If the attribute is present, or if scripting is disabled for the media element, then the user agent should expose a user interface to the user. This user interface should include features to begin playback, pause playback, seek to an arbitrary position in the content (if the content supports arbitrary seeking), change the volume, change the display of closed captions or embedded sign-language tracks, select different audio tracks or turn on audio descriptions, and show the media content in manners more suitable to the user (e.g. full-screen video or in an independent resizable window). Other controls may also be made available.

If the media element has a current media controller, then the user agent should expose audio tracks from all the slaved media elements (although avoiding duplicates if the same media resource is being used several times). If a media resource's audio track exposed in this way has no known name, and it is the only audio track for a particular media element, the user agent should use the element's title attribute, if any, as the name (or as part of the name) of that track.

Even when the attribute is absent, however, user agents may provide controls to affect playback of the media resource (e.g. play, pause, seeking, and volume controls), but such features should not interfere with the page's normal rendering. For example, such features could be exposed in the media element's context menu.

Where possible (specifically, for starting, stopping, pausing, and unpausing playback, for seeking, for changing the rate of playback, for fast-forwarding or rewinding, for listing, enabling, and disabling text tracks, and for muting or changing the volume of the audio), user interface features exposed by the user agent must be implemented in terms of the DOM API described above, so that, e.g., all the same events fire.

When a media element has a current media controller, the user agent's user interface for pausing and unpausing playback, for seeking, for changing the rate of playback, for fast-forwarding or rewinding, and for muting or changing the volume of audio of the entire group must be implemented in terms of the MediaController API exposed on that current media controller. When a media element has a current media controller, and all the slaved media elements of that MediaController are paused, the user agent should also unpause all the slaved media elements when the user invokes a user agent interface control for beginning playback.

The "play" function in the user agent's interface must set the playbackRate attribute to the value of the defaultPlaybackRate attribute before invoking the play() method. When a media element has a current media controller, the attributes and method with those names on that MediaController object must be used. Otherwise, the attributes and method with those names on the media element itself must be used.

Features such as fast-forward or rewind must be implemented by only changing the playbackRate attribute (and not the defaultPlaybackRate attribute). Again, when a media element has a current media controller, the attributes with those names on that MediaController object must be used; otherwise, the attributes with those names on the media element itself must be used.

When a media element has a current media controller, seeking must be implemented in terms of the currentTime attribute on that MediaController object. Otherwise, the user agent must directly seek to the requested position in the media element's media timeline.

When a media element has a current media controller, user agents may additionally provide the user with controls that directly manipulate an individual media element without affecting the MediaController, but such features are considered relatively advanced and unlikely to be useful to most users.

For the purposes of listing chapters in the media resource, only text tracks in the media element's list of text tracks showing or showing by default and whose text track kind is chapters should be used. Such tracks must be interpreted according to the rules for constructing the chapter tree from a text track.

The controls IDL attribute must reflect the content attribute of the same name.


media . volume [ = value ]

Returns the current playback volume, as a number in the range 0.0 to 1.0, where 0.0 is the quietest and 1.0 the loudest.

Can be set, to change the volume.

Throws an IndexSizeError if the new value is not in the range 0.0 .. 1.0.

media . muted [ = value ]

Returns true if audio is muted, overriding the volume attribute, and false if the volume attribute is being honored.

Can be set, to change whether the audio is muted or not.

The volume attribute must return the playback volume of any audio portions of the media element, in the range 0.0 (silent) to 1.0 (loudest). Initially, the volume should be 1.0, but user agents may remember the last set value across sessions, on a per-site basis or otherwise, so the volume may start at other values. On setting, if the new value is in the range 0.0 to 1.0 inclusive, the playback volume of any audio portions of the media element must be set to the new value. If the new value is outside the range 0.0 to 1.0 inclusive, then, on setting, an IndexSizeError exception must be thrown instead.

The muted attribute must return true if the audio output is muted and false otherwise. Initially, the audio output should not be muted (false), but user agents may remember the last set value across sessions, on a per-site basis or otherwise, so the muted state may start as muted (true). On setting, if the new value is true then the audio output should be muted and if the new value is false it should be unmuted.

Whenever either of the values that would be returned by the volume and muted attributes change, the user agent must queue a task to fire a simple event named volumechange at the media element.

An element's effective media volume is determined as follows:

  1. If the user has indicated that the user agent is to override the volume of the element, then the element's effective media volume is the volume desired by the user. Abort these steps.

  2. If the element's audio output is muted, the element's effective media volume is zero. Abort these steps.

  3. If the element has a current media controller and that MediaController object's media controller mute override is true, the element's effective media volume is zero. Abort these steps.

  4. Let volume be the playback volume of the audio portions of the media element, in range 0.0 (silent) to 1.0 (loudest).

  5. If the element has a current media controller, multiply volume by that MediaController object's media controller volume multiplier.

  6. The element's effective media volume is volume, interpreted relative to the range 0.0 to 1.0, with 0.0 being silent, and 1.0 being the loudest setting, values in between increasing in loudness. The range need not be linear. The loudest setting may be lower than the system's loudest possible setting; for example the user could have set a maximum volume.

The muted attribute on media elements is a boolean attribute that controls the default state of the audio output of the media resource, potentially overriding user preferences.

When a media element is created, if it has a muted attribute specified, the user agent must mute the media element's audio output, overriding any user preference.

The defaultMuted IDL attribute must reflect the muted content attribute.

This attribute has no dynamic effect (it only controls the default state of the element).

This video (an advertisment) autoplays, but to avoid annoying users, it does so without sound, and allows the user to turn the sound on.

<video src="adverts.cgi?kind=video" controls autoplay loop muted></video>
4.8.10.14 Time ranges

Objects implementing the TimeRanges interface represent a list of ranges (periods) of time.

interface TimeRanges {
  readonly attribute unsigned long length;
  double start(unsigned long index);
  double end(unsigned long index);
};
media . length

Returns the number of ranges in the object.

time = media . start(index)

Returns the time for the start of the range with the given index.

Throws an IndexSizeError if the index is out of range.

time = media . end(index)

Returns the time for the end of the range with the given index.

Throws an IndexSizeError if the index is out of range.

The length IDL attribute must return the number of ranges represented by the object.

The start(index) method must return the position of the start of the indexth range represented by the object, in seconds measured from the start of the timeline that the object covers.

The end(index) method must return the position of the end of the indexth range represented by the object, in seconds measured from the start of the timeline that the object covers.

These methods must throw IndexSizeError exceptions if called with an index argument greater than or equal to the number of ranges represented by the object.

When a TimeRanges object is said to be a normalized TimeRanges object, the ranges it represents must obey the following criteria:

In other words, the ranges in such an object are ordered, don't overlap, aren't empty, and don't touch (adjacent ranges are folded into one bigger range).

Ranges in a TimeRanges object must be inclusive.

Thus, the end of a range would be equal to the start of a following adjacent (touching but not overlapping) range. Similarly, a range covering a whole timeline anchored at zero would have a start equal to zero and an end equal to the duration of the timeline.

The timelines used by the objects returned by the buffered, seekable and played IDL attributes of media elements must be that element's media timeline.

4.8.10.15 Event definitions
[Constructor(DOMString type, optional TrackEventInit eventInitDict)]
interface TrackEvent : Event {
  readonly attribute object? track;
};

dictionary TrackEventInit : EventInit {
  object? track;
};
event . track

Returns the track object (TextTrack, AudioTrack, or VideoTrack) to which the event relates.

The track attribute must return the value it was initialized to. When the object is created, this attribute must be initialized to null. It represents the context information for the event.

4.8.10.16 Event summary

This section is non-normative.

The following events fire on media elements as part of the processing model described above:

Event name Interface Fired when... Preconditions
loadstart Event The user agent begins looking for media data, as part of the resource selection algorithm. networkState equals NETWORK_LOADING
progress Event The user agent is fetching media data. networkState equals NETWORK_LOADING
suspend Event The user agent is intentionally not currently fetching media data. networkState equals NETWORK_IDLE
abort Event The user agent stops fetching the media data before it is completely downloaded, but not due to an error. error is an object with the code MEDIA_ERR_ABORTED. networkState equals either NETWORK_EMPTY or NETWORK_IDLE, depending on when the download was aborted.
error Event An error occurs while fetching the media data. error is an object with the code MEDIA_ERR_NETWORK or higher. networkState equals either NETWORK_EMPTY or NETWORK_IDLE, depending on when the download was aborted.
emptied Event A media element whose networkState was previously not in the NETWORK_EMPTY state has just switched to that state (either because of a fatal error during load that's about to be reported, or because the load() method was invoked while the resource selection algorithm was already running). networkState is NETWORK_EMPTY; all the IDL attributes are in their initial states.
stalled Event The user agent is trying to fetch media data, but data is unexpectedly not forthcoming. networkState is NETWORK_LOADING.
loadedmetadata Event The user agent has just determined the duration and dimensions of the media resource and the text tracks are ready. readyState is newly equal to HAVE_METADATA or greater for the first time.
loadeddata Event The user agent can render the media data at the current playback position for the first time. readyState newly increased to HAVE_CURRENT_DATA or greater for the first time.
canplay Event The user agent can resume playback of the media data, but estimates that if playback were to be started now, the media resource could not be rendered at the current playback rate up to its end without having to stop for further buffering of content. readyState newly increased to HAVE_FUTURE_DATA or greater.
canplaythrough Event The user agent estimates that if playback were to be started now, the media resource could be rendered at the current playback rate all the way to its end without having to stop for further buffering. readyState is newly equal to HAVE_ENOUGH_DATA.
playing Event Playback is ready to start after having been paused or delayed due to lack of media data. readyState is newly equal to or greater than HAVE_FUTURE_DATA and paused is false, or paused is newly false and readyState is equal to or greater than HAVE_FUTURE_DATA. Even if this event fires, the element might still not be potentially playing, e.g. if the element is blocked on its media controller (e.g. because the current media controller is paused, or another slaved media element is stalled somehow, or because the media resource has no data corresponding to the media controller position), or the element is paused for user interaction or paused for in-band content.
waiting Event Playback has stopped because the next frame is not available, but the user agent expects that frame to become available in due course. readyState is equal to or less than HAVE_CURRENT_DATA, and paused is false. Either seeking is true, or the current playback position is not contained in any of the ranges in buffered. It is possible for playback to stop for other reasons without paused being false, but those reasons do not fire this event (and when those situations resolve, a separate playing event is not fired either): e.g. the element is newly blocked on its media controller, or playback ended, or playback stopped due to errors, or the element has paused for user interaction or paused for in-band content.
seeking Event The seeking IDL attribute changed to true.
seeked Event The seeking IDL attribute changed to false.
ended Event Playback has stopped because the end of the media resource was reached. currentTime equals the end of the media resource; ended is true.
durationchange Event The duration attribute has just been updated.
timeupdate Event The current playback position changed as part of normal playback or in an especially interesting way, for example discontinuously.
play Event The element is no longer paused. Fired after the play() method has returned, or when the autoplay attribute has caused playback to begin. paused is newly false.
pause Event The element has been paused. Fired after the pause() method has returned. paused is newly true.
ratechange Event Either the defaultPlaybackRate or the playbackRate attribute has just been updated.
volumechange Event Either the volume attribute or the muted attribute has changed. Fired after the relevant attribute's setter has returned.

The following events fire on MediaController objects:

Event name Interface Fired when...
emptied Event All the slaved media elements newly have readyState set to HAVE_NOTHING or greater, or there are no longer any slaved media elements.
loadedmetadata Event All the slaved media elements newly have readyState set to HAVE_METADATA or greater.
loadeddata Event All the slaved media elements newly have readyState set to HAVE_CURRENT_DATA or greater.
canplay Event All the slaved media elements newly have readyState set to HAVE_FUTURE_DATA or greater.
canplaythrough Event All the slaved media elements newly have readyState set to HAVE_ENOUGH_DATA or greater.
playing Event The MediaController is no longer a blocked media controller.
ended Event The MediaController has reached the end of all the slaved media elements.
waiting Event The MediaController is now a blocked media controller.
ended Event All the slaved media elements have newly ended playback.
durationchange Event The duration attribute has just been updated.
timeupdate Event The media controller position changed.
play Event The paused attribute is newly false.
pause Event The paused attribute is newly true.
ratechange Event Either the defaultPlaybackRate attribute or the playbackRate attribute has just been updated.
volumechange Event Either the volume attribute or the muted attribute has just been updated.
4.8.10.17 Security and privacy considerations

The main security and privacy implications of the video and audio elements come from the ability to embed media cross-origin. There are two directions that threats can flow: from hostile content to a victim page, and from a hostile page to victim content.


If a victim page embeds hostile content, the threat is that the content might contain scripted code that attempts to interact with the Document that embeds the content. To avoid this, user agents must ensure that there is no access from the content to the embedding page. In the case of media content that uses DOM concepts, the embedded content must be treated as if it was in its own unrelated top-level browsing context.

For instance, if an SVG animation was embedded in a video element, the user agent would not give it access to the DOM of the outer page. From the perspective of scripts in the SVG resource, the SVG file would appear to be in a lone top-level browsing context with no parent.


If a hostile page embeds victim content, the threat is that the embedding page could obtain information from the content that it would not otherwise have access to. The API does expose some information: the existence of the media, its type, its duration, its size, and the performance characteristics of its host. Such information is already potentially problematic, but in practice the same information can more or less be obtained using the img element, and so it has been deemed acceptable.

However, significantly more sensitive information could be obtained if the user agent further exposes metadata within the content such as subtitles or chapter titles. Such information is therefore only exposed if the video resource passes a CORS resource sharing check. The crossorigin attribute allows authors to control how this check is performed. [CORS]

Without this restriction, an attacker could trick a user running within a corporate network into visiting a site that attempts to load a video from a previously leaked location on the corporation's intranet. If such a video included confidential plans for a new product, then being able to read the subtitles would present a serious confidentiality breach.

4.8.10.18 Best practices for authors using media elements

This section is non-normative.

Playing audio and video resources on small devices such as set-top boxes or mobile phones is often constrained by limited hardware resources in the device. For example, a device might only support three simultaneous videos. For this reason, it is a good practice to release resources held by media elements when they are done playing, either by being very careful about removing all references to the element and allowing it to be garbage collected, or, even better, by removing the element's src attribute and any source element descendants, and invoking the element's load() method.

Similarly, when the playback rate is not exactly 1.0, hardware, software, or format limitations can cause video frames to be dropped and audio to be choppy or muted.

4.8.10.19 Best practices for implementors of media elements

This section is non-normative.

How accurately various aspects of the media element API are implemented is considered a quality-of-implementation issue.

For example, when implementing the buffered attribute, how precise an implementation reports the ranges that have been buffered depends on how carefully the user agent inspects the data. Since the API reports ranges as times, but the data is obtained in byte streams, a user agent receiving a variable-bit-rate stream might only be able to determine precise times by actually decoding all of the data. User agents aren't required to do this, however; they can instead return estimates (e.g. based on the average bit rate seen so far) which get revised as more information becomes available.

As a general rule, user agents are urged to be conservative rather than optimistic. For example, it would be bad to report that everything had been buffered when it had not.

Another quality-of-implementation issue would be playing a video backwards when the codec is designed only for forward playback (e.g. there aren't many key frames, and they are far apart, and the intervening frames only have deltas from the previous frame). User agents could do a poor job, e.g. only showing key frames; however, better implementations would do more work and thus do a better job, e.g. actually decoding parts of the video forwards, storing the complete frames, and then playing the frames backwards.

Similarly, while implementations are allowed to drop buffered data at any time (there is no requirement that a user agent keep all the media data obtained for the lifetime of the media element), it is again a quality of implementation issue: user agents with sufficient resources to keep all the data around are encouraged to do so, as this allows for a better user experience. For example, if the user is watching a live stream, a user agent could allow the user only to view the live video; however, a better user agent would buffer everything and allow the user to seek through the earlier material, pause it, play it forwards and backwards, etc.

When multiple tracks are synchronised with a MediaController, it is possible for scripts to add and remove media elements from the MediaController's list of slaved media elements, even while these tracks are playing. How smoothly the media plays back in such situations is another quality-of-implementation issue.


When a media element that is paused is removed from a document and not reinserted before the next time the event loop spins, implementations that are resource constrained are encouraged to take that opportunity to release all hardware resources (like video planes, networking resources, and data buffers) used by the media element. (User agents still have to keep track of the playback position and so forth, though, in case playback is later restarted.)