Introduction

The MediaStream interface is used to represent streams of media data, typically (but not necessarily) of audio and/or video content, e.g. from a local camera. The data from a MediaStream object does not necessarily have a canonical binary form; for example, it could just be "the video currently coming from the user’s video camera". This allows user agents to manipulate media streams in whatever fashion is most suitable on the user’s platform.

Each MediaStream object can contain zero or more tracks, in particular audio and video tracks. All tracks in a MediaStream are intended to be synchronized when rendered. Different MediaStreams do not need to be synchronized.

Each track in a MediaStream object has a corresponding MediaStreamTrack object.

A MediaStreamTrack represents content comprising one or more channels, where the channels have a defined well known relationship to each other (such as a stereo or 5.1 audio signal).

A channel is the smallest unit considered in this API specification.

A MediaStream object has an input and an output. The input depends on how the object was created: a MediaStream object generated by a getUserMedia() call (which is described later in this document), for instance, might take its input from the user’s local camera. The output of the object controls how the object is used, e.g., what is saved if the object is written to a file or what is displayed if the object is used in a video element.

Each track in a MediaStream object can be disabled, meaning that it is muted in the object’s output. All tracks are initially enabled.

A MediaStream can be finished, indicating that its inputs have forever stopped providing data.

The output of a MediaStream object MUST correspond to the tracks in its input. Muted audio tracks MUST be replaced with silence. Muted video tracks MUST be replaced with blackness.

A new MediaStream object can be created from accessible media sources (that does not require any additional permissions) using the MediaStream() constructor. The constructor argument can either be an existing MediaStream object, in which case all the tracks of the given stream are added to the new MediaStream object, or an array of MediaStreamTrack objects. The latter from makes it possible to compose a stream from different source streams.

Both MediaStream and MediaStreamTrack objects can be cloned. This allows for greater control since the separate instances can be manipulated and consumed individually. A cloned MediaStream contains clones of all member tracks from the original stream.

When a MediaStream object is being generated from a local file (as opposed to a live audio/video source), the user agent SHOULD stream the data from the file in real time, not all at once. The MediaStream object is also used in contexts outside getUserMedia, such as [[!WEBRTC10]]. In both cases, ensuring a realtime stream reduces the ease with which pages can distinguish live video from pre-recorded video, which can help protect the user’s privacy.

MediaStream

The MediaStream() constructor composes a new stream out of existing tracks. It takes zero or one argument. If the argument is supplied, it MUST either be of type MediaStream or an array of MediaStreamTrack objects. When the constructor is invoked, the UA must run the following steps:

1. Let stream be a newly constructed MediaStream object.

2. Initialize stream’s id attribute to a newly generated value.

3. If the constructor’s argument is present, run the sub steps that corresponds to the argument type.

   • Array of MediaStreamTrack objects:

     Run the following sub steps for each MediaStreamTrack in the array:

     1. Add track: Let track be the MediaStreamTrack about to be processed.

     2. If track has ended or if there is already a MediaStreamTrack contained within stream that has the same underlying source as track, then abort these steps and continue with the next track (if any).

     3. Add track to stream's track set.

   • MediaStream:

     Run the sub steps labeled Add track (above) for every MediaStreamTrack in the argument stream’s track set.

4. Return stream.

A MediaStream can have multiple audio and video sources (e.g. because the user has multiple microphones, or because the real source of the stream is a media resource with many media tracks). The stream represented by a MediaStream thus has zero or more tracks.

The tracks of a MediaStream are stored in a track set. The track set MUST contain the MediaStreamTrack objects that correspond to the tracks of the stream. The relative order of the tracks in the set is user agent defined and the API will never put any requirements on the order. The proper way to find a specific MediaStreamTrack object in the set is to look it up by its id.

An object that reads data from the output of a MediaStream is referred to as a MediaStream consumer. The list of MediaStream consumers currently includes the media elements and the PeerConnection API specified in [[!WEBRTC10]].

MediaStream consumers must be able to handle tracks being added and removed. This behavior is specified per consumer.

A MediaStream object is said to be finished when all tracks belonging to the stream have ended. When this happens for any reason other than the stop() method being invoked, the user agent MUST queue a task that runs the following steps:

1. If the object’s ended attribute has the value true already, then abort these steps. (The stop() method was probably called just before the stream stopped for other reasons, e.g. the user clicked an in-page stop button and then the user agent provided stop button.)

2. Set the object’s ended attribute to true.

3. Fire a simple event named ended at the object.

If the end of the stream was reached due to a user request, the task source for this task is the user interaction task source. Otherwise the task source for this task is the networking task source.

readonly attribute DOMString id

When a MediaStream object is created, the user agent MUST generate a globally unique identifier string, and MUST initialize the object’s id attribute to that string. Such strings MUST only use characters in the ranges U+0021, U+0023 to U+0027, U+002A to U+002B, U+002D to U+002E, U+0030 to U+0039, U+0041 to U+005A, U+005E to U+007E, and MUST be 36 characters long.

The id attribute MUST return the value to which it was initialized when the object was created.

sequence<MediaStreamTrack> getAudioTracks()

Returns a sequence of MediaStreamTrack objects representing the audio tracks in this stream.

The getAudioTracks() method MUST return a sequence that represents a snapshot of all the MediaStreamTrack objects in this stream’s track set whose kind is equal to "audio". The conversion from the track set to the sequence is user agent defined and the order does not have to stable between calls.

sequence<MediaStreamTrack> getVideoTracks()

Returns a sequence of MediaStreamTrack objects representing the video tracks in this stream.

The getVideoTracks() method MUST return a sequence that represents a snapshot of all the MediaStreamTrack objects in this stream’s track set whose kind is equal to "video". The conversion from the track set to the sequence is user agent defined and the order does not have to stable between calls.

MediaStreamTrack? getTrackById(DOMString trackId)

The getTrackById() method MUST return the first MediaStreamTrack object in this stream’s track set whose id is equal to trackId. The method MUST return null if no track matches the trackId argument.

void addTrack(MediaStreamTrack track)

Adds the given MediaStreamTrack to this MediaStream.

When the addTrack() method is invoked, the user agent MUST run the following steps:

1. Let track be the MediaStreamTrack argument and stream this MediaStream object.

2. If stream is finished, throw an INVALID_STATE_ERR exception.

3. If track is already in stream’s track set, then abort these steps.

4. Add track to stream's track set.

void removeTrack(MediaStreamTrack track)

Removes the given MediaStreamTrack from this MediaStream.

When the removeTrack() method is invoked, the user agent MUST run the following steps:

1. Let track be the MediaStreamTrack argument and stream this MediaStream object.

2. If stream is finished, throw an INVALID_STATE_ERR exception.

3. If track is in stream's track set, remove it.

MediaStream clone()

Clones the given MediaStream and all its tracks.

When the clone() method is invoked, the user agent MUST run the following steps:

1. Let streamClone be a newly constructed MediaStream object.

2. Initialize streamClone's id attribute to a newly generated value.

3. Let trackSetClone be a list that contains the result of running MediaStreamTrack.clone() on all the tracks in this stream.

4. Let trackSetClone be streamClone's track set.

attribute boolean ended

The MediaStream.ended attribute MUST return true if the MediaStream has finished, and false otherwise.

When a MediaStream object is created, its ended attribute MUST be set to false, unless it is being created using the MediaStream() constructor whose arguments are lists of MediaStreamTrack objects that are all ended, in which case the MediaStream object MUST be created with its ended attribute set to true.

attribute EventHandler onended

This event handler, of type ended, MUST be supported by all objects implementing the MediaStream interface.

attribute EventHandler onaddtrack

This event handler, of type addtrack, MUST be supported by all objects implementing the MediaStream interface.

attribute EventHandler onremovetrack

This event handler, of type removetrack, MUST be supported by all objects implementing the MediaStream interface.

MediaStreamTrack

A MediaStreamTrack object represents a media source in the user agent. Several MediaStreamTrack objects can represent the same media source, e.g., when the user chooses the same camera in the UI shown by two consecutive calls to getUserMedia() .

Note that a web application can revoke all given permissions with MediaStreamTrack.stop().

MediaStreamTrackEvent

The addtrack and removetrack events use the MediaStreamTrackEvent interface.

Firing a track event named e with a MediaStreamTrack track means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the MediaStreamTrackEvent interface with the track attribute set to track, MUST be created and dispatched at the given target.

readonly attribute MediaStreamTrack track

The track attribute represents the MediaStreamTrack object associated with the event.

readonly MediaStreamTrack? track

 

Video and Audio Tracks

The MediaStreamTrack object cannot be instantiated directly. To create an instance of a MediaStreamTrack, one of its derived track types may be instantiated. These derived types are defined in this section.

Note that the camera's green light doesn't come on when a new track is created; nor does the user get prompted to enable the camera/microphone. Those actions only happen after the developer has requested that a media stream containing "new" tracks be bound to a source via getUserMedia(). Until that point tracks are inert.

CapabilityList array

Capability Lists are just an array of supported DOMString values from the possible superset of values described by each state's enumerated type.

NavigatorUserMedia

void getUserMedia(MediaStreamConstraints? constraints, NavigatorUserMediaSuccessCallback successCallback, NavigatorUserMediaErrorCallback errorCallback)

Prompts the user for permission to use their Web cam or other video or audio input.

The syntax for getUserMedia() is still under discussion. Some questions are: whether it creates media streams or just initializes ones already created; whether it creates streams synchronously (immediate return of MediaStream with callback later when user grants permission) or asynchronously (MediaStream available only upon callback).

The constraints argument is an object of type MediaStreamConstraints.

The successCallback will be invoked with a suitable MediaStream object as its argument if the user accepts valid tracks as described below.

The errorCallback will be invoked if there is a failure in finding valid tracks or if the user denies permission, both as described below.

When the getUserMedia() method is called, the user agent MUST run the following steps:

1. Let constraints be the method's first argument.

2. Let successCallback be the callback indicated by the method's second argument.

3. Let errorCallback be the callback indicated by the method's third argument.

4. Let requestedMediaTypes be the set of media types in constraints with either a dictionary value or a value of "true".

5. Let finalSet be an (initially) empty set.

6. If successCallback is null, abort these steps.

7. For each media type T in requestedMediaTypes,

   1. Let candidateSet be all possible tracks of media type T that the browser could return.

   2. If the value of the T entry of constraints is "true", jump to the step labeled final below. Otherwise, continue.

   3. For each constraint key-value pair in the "mandatory" dictionary,

      1. If the constraint is not supported by the browser, queue a task to invoke errorCallback with a new NavigatorUserMediaError object whose name attribute has the value CONSTRAINT_NOT_SATISFIED and whose constraintName atribute is set to the name of the constraint that caused the error and then jump to the step labeled failure below.

      2. Remove from the candidateSet any track that cannot satisfy the value given for the constraint.

      3. If the candidateSet no longer contains at least one track, queue a task to invoke errorCallback with a new NavigatorUserMediaError object whose name attribute has the value CONSTRAINT_NOT_SATISFIED and whose constraintName atribute is set to the name of the constraint that caused the error and then jump to the step labeled failure below. Otherwise, continue with the next mandatory constraint.

   4. Let the secondPassSet be the current contents of the candidateSet.

   5. For each constraint key-value pair in the "optional" sequence of the constraints that are for the current media type, in order,

      1. If the constraint is not supported by the browser, skip it and continue with the next constraint.

      2. Remove from the secondPassSet any tracks that cannot satisfy the value given for the constraint.

      3. If the secondPassSet is now empty, let the secondPassSet be the current contents of the candidateSet. Otherwise, let the candidateSet be the current contents of the secondPassSet.

   6. Final: Add the tracks in the candidateSet to the finalSet.

• Return, and run the remaining steps asynchronously.

• Optionally, e.g., based on a previously-established user preference, for security reasons, or due to platform limitations, jump to the step labeled failure below.

• Prompt the user in a user agent specific manner for permission to provide the entry script's origin with a MediaStream object representing a media stream.

  The provided media MUST include precisely one track of each media type in requestedMediaTypes from the finalSet. The decision of which tracks to choose from the finalSet is completely up to the user agent and may be determined by asking the user. Unless and until a new set of constraints is provided, the user agent MAY change its choice of track at any point, provided that 1) the new choice does not violate given user permissions, and 2) it notifies the application code by raising an event. It may wish to do this, for example, if the user interface or network congestion changes. Note that no such change will have an effect on the presence or absence of each type of track, merely the contents.

  Define the event that should be raised when the user agent changes its choice of track.

  User agents are encouraged to default to using the user's primary or system default camera and/or microphone (when possible) to generate the media stream. User agents MAY allow users to use any media source, including pre-recorded media files.

  If the user grants permission to use local recording devices, user agents are encouraged to include a prominent indicator that the devices are "hot" (i.e. an "on-air" or "recording" indicator).

  If the user denies permission, jump to the step labeled failure below. If the user never responds, this algorithm stalls on this step.

• Let stream be the MediaStream object for which the user granted permission.

• Queue a task to invoke successCallback with stream as its argument.

• Abort these steps.

• Failure:

• Let error be a new NavigatorUserMediaError object whose name attribute has the value PERMISSION_DENIED.

• Queue a task to invoke errorCallback with error as its argument.

• The task source for these tasks is the user interaction task source.

MediaStreamConstraints

(boolean or MediaTrackConstraints) video = false

Provide definition of video constraints here.

(boolean or MediaTrackConstraints) audio = false

Provide definition of audio constraints here.

MediaTrackConstraintSet? mandatory = null

Provide definition of mandatory constraints here.

MediaTrackConstraint[]? _optional = null

Provide definition of optional constraints here.

A MediaTrackConstraintSet is a dictionary containing one or more key-value pairs, where each key MUST be a valid registered constraint name in the IANA-hosted RTCWeb Media Constraints registry [[!RTCWEB-CONSTRAINTS]] and its value SHOULD be as defined in the associated reference(s) given in the registry.

A MediaTrackConstraint is a dictionary containing exactly one key-value pair, where the key MUST be a valid registered constraint name in the IANA-hosted RTCWeb Media Constraints registry [[!RTCWEB-CONSTRAINTS]] and the value SHOULD be as defined in the associated reference(s) given in the registry.

NavigatorUserMediaSuccessCallback

MediaStream stream

Add explanation of handleEvent

NavigatorUserMediaError and NavigatorUserMediaErrorCallback

readonly attribute DOMString name

A string representing the type of error. This string must be one of those defined by the NavigatorUserMediaErrorName enum for the error object to be valid.

readonly attribute DOMString? message

A human readable description of the error. This string may vary between different user agents.

readonly attribute DOMString? constraintName

The name of the constraint that caused the error. Only used for some types of errors. For NavigatorUserMediaError with a name of CONSTRAINT_NOT_SATISFIED , the constraintName attribute in the NavigatorUserMediaError object MUST be set to the name of the constraint that caused the error.

PERMISSION_DENIED

User denied permission for scripts from this origin to access the media device.

CONSTRAINT_NOT_SATISFIED

One of the mandatory constraints could not be satisfied.

NavigatorUserMediaError error

Add explanation of handleEvent

Constraint Registrations

IANA is requested to register the following constraints as specified in [[!RTCWEB-CONSTRAINTS]]:

The following constraint names are defined to apply to both VideoStreamTrack and AudioStreamTrack objects:

The following constraint names are defined to apply only to VideoStreamTrack objects:

The following constraint names are defined to apply only to AudioStreamTrack objects:

For constraints that accept ranges, the MinMaxConstraint dictionary is also defined. Note that the type of the value associated with min and max must be the same for both. The specific types associated with min and max are defined differently for each constraint name.

any max

The related constraint's maximum allowed value.

any min

The related constraint's minimum allowed value.