Media Capture and Streams

4.2.1 Attributes

ended of type boolean,

The MediaStream.ended attribute MUST return true if the MediaStream has finished, and false otherwise.

When a MediaStream object is created, its ended attribute MUST be set to false, unless it is being created using the MediaStream() constructor whose arguments are lists of MediaStreamTrack objects that are all ended, in which case the MediaStream object MUST be created with its ended attribute set to true.

id of type DOMString, readonly

When a MediaStream object is created, the user agent MUST generate a globally unique identifier string, and MUST initialize the object’s id attribute to that string. Such strings MUST only use characters in the ranges U+0021, U+0023 to U+0027, U+002A to U+002B, U+002D to U+002E, U+0030 to U+0039, U+0041 to U+005A, U+005E to U+007E, and MUST be 36 characters long.

The id attribute MUST return the value to which it was initialized when the object was created.

onaddtrack of type EventHandler,

This event handler, of type addtrack, MUST be supported by all objects implementing the MediaStream interface.

onended of type EventHandler,

This event handler, of type ended, MUST be supported by all objects implementing the MediaStream interface.

onremovetrack of type EventHandler,

This event handler, of type removetrack, MUST be supported by all objects implementing the MediaStream interface.

4.2.2 Methods

addTrack

Adds the given MediaStreamTrack to this MediaStream.

When the addTrack() method is invoked, the user agent MUST run the following steps:

1. Let track be the MediaStreamTrack argument and stream this MediaStream object.

2. If stream is finished, throw an INVALID_STATE_ERR exception.

3. If track is already in stream’s track set, then abort these steps.

4. Add track to stream's track set.

Parameter	Type	Nullable	Optional	Description
track	MediaStreamTrack	✘	✘

Return type: void

clone

Clones the given MediaStream and all its tracks.

When the clone() method is invoked, the user agent MUST run the following steps:

1. Let streamClone be a newly constructed MediaStream object.

2. Initialize streamClone's id attribute to a newly generated value.

3. Let trackSetClone be a list that contains the result of running MediaStreamTrack.clone() on all the tracks in this stream.

4. Let trackSetClone be streamClone's track set.

No parameters.

Return type: MediaStream

getAudioTracks

Returns a sequence of MediaStreamTrack objects representing the audio tracks in this stream.

The getAudioTracks() method MUST return a sequence that represents a snapshot of all the MediaStreamTrack objects in this stream’s track set whose kind is equal to "audio". The conversion from the track set to the sequence is user agent defined and the order does not have to stable between calls.

No parameters.

Return type: sequence<MediaStreamTrack>

getTrackById

The getTrackById() method MUST return the first MediaStreamTrack object in this stream’s track set whose id is equal to trackId. The method MUST return null if no track matches the trackId argument.

Parameter	Type	Nullable	Optional	Description
trackId	DOMString	✘	✘

Return type: MediaStreamTrack, nullable

getVideoTracks

Returns a sequence of MediaStreamTrack objects representing the video tracks in this stream.

The getVideoTracks() method MUST return a sequence that represents a snapshot of all the MediaStreamTrack objects in this stream’s track set whose kind is equal to "video". The conversion from the track set to the sequence is user agent defined and the order does not have to stable between calls.

No parameters.

Return type: sequence<MediaStreamTrack>

removeTrack

Removes the given MediaStreamTrack from this MediaStream.

When the removeTrack() method is invoked, the user agent MUST run the following steps:

1. Let track be the MediaStreamTrack argument and stream this MediaStream object.

2. If stream is finished, throw an INVALID_STATE_ERR exception.

3. If track is in stream's track set, remove it.

Parameter	Type	Nullable	Optional	Description
track	MediaStreamTrack	✘	✘

Return type: void

4.3.1 Life-cycle and Media Flow

A MediaStreamTrack has three stages in its lifecycle. A track begins as new prior to being connected to an active source.

Once connected, the started event fires and the track becomes live. In the live state, the track is active and media is available for rendering at a MediaStream consumer.

A muted or disabled MediaStreamTrack renders either silence (audio), black frames (video), or a zero-information-content equivalent. The muted/unmuted state of a track reflects if the source provides any media at this moment. The enabled/disabled state is under application control and determines if the track outputs media (to its consumers). Hence, media from the source only flows when a MediaStreamTrack object is both unmuted and enabled.

A MediaStreamTrack is muted when the source is temporarily unable to provide the track with data. A track can be muted by a user. Often this action is outside the control of the application. This could be as a result of the user hitting a hardware switch, or toggling a control in the operating system or browser chrome. A track can also be muted by the user agent. For example, a track that is a member of a MediaStream, received via a RTCPeerConnection [WEBRTC10], is muted if the application on the other side disables the corresponding track in the MediaStream being sent.

Applications are able to enable or disable a MediaStreamTrack to prevent it from rendering media from the source. A muted track will however, regardless of the enabled state, render silence and blackness. A disabled track is logically equivalent to a muted track, from a consumer point of view.

A MediaStreamTrack object is said to end when the source of the track is disconnected or exhausted.

When a MediaStreamTrack object ends for any reason (e.g., because the user rescinds the permission for the page to use the local camera, or because the data comes from a finite file and the file’s end has been reached and the user has not requested that it be looped, or because the UA has instructed the track to end for any reason, or because the reference count of the track’s underlying media source has reached zero, it is said to be ended. When track instance track ends for any reason other than the stop() method being invoked on the MediaStreamTrack object that represents track, the user agent MUST queue a task that runs the following steps:

1. If the track’s readyState attribute has the value ended already, then abort these steps.

2. Set track’s readyState attribute to ended.

3. Fire a simple event named ended at the object.

If the end of the stream was reached due to a user request, the event source for this event is the user interaction event source.

4.3.2 Tracks and Constraints

Constraints are independent of sources. However, if the sourceType is "none", the readonly attribute is true, or the remote attribute is true, the track's constraints will not be applied to the source.

Whether MediaTrackConstraints were provided at track initialization time or need to be established later at runtime, the APIs defined below allow the retrieval and manipulation of the constraints currently established on a track.

Each track maintains an internal version of the MediaTrackConstraints structure, namely a mandatory set of constraints (no duplicates), and an optional ordered list of individual constraint objects (may contain duplicates). The internal stored constraint structure is only exposed to the application using the existing MediaTrackConstraints, MediaTrackConstraintSet, MediaTrackConstraint, and similarly-derived-type dictionary objects.

When track constraints change, a user agent must queue a task to evaluate those changes when the task queue is next serviced. Similarly, if the sourceType changes, then the user agent should perform the same actions to re-evaluate the constraints of each track affected by that source change.

4.3.3 Interface Definition

interface MediaStreamTrack : EventTarget {
    readonly    attribute DOMString             kind;
    readonly    attribute DOMString             id;
    readonly    attribute DOMString             label;
                attribute boolean               enabled;
    readonly    attribute boolean               muted;
                attribute EventHandler          onmute;
                attribute EventHandler          onunmute;
    readonly    attribute boolean               readonly;
    readonly    attribute boolean               remote;
    readonly    attribute MediaStreamTrackState readyState;
                attribute EventHandler          onstarted;
                attribute EventHandler          onended;
    MediaTrackConstraints?                        constraints ();
    MediaSourceStates                             states ();
    (AllVideoCapabilites or AllAudioCapabilities) capabilities ();
    void                                          applyConstraints (MediaTrackConstraints constraints);
                attribute EventHandler          onoverconstrained;
    MediaStreamTrack                              clone ();
    void                                          stop ();
};

enum MediaStreamTrackState {
    "new",
    "live",
    "ended"
};

4.3.4 Track Source Types

enum SourceTypeEnum {
    "none",
    "camera",
    "microphone",
    "photo-camera"
};

4.4.1 Attributes

track of type MediaStreamTrack, readonly

The track attribute represents the MediaStreamTrack object associated with the event.

4.4.2 Dictionary MediaStreamTrackEventInit Members

track of type readonly MediaStreamTrack, nullable

 

4.5.1 VideoStreamTrack interface

Video tracks may be instantiated with optional media track constraints. These constraints can be later modified on the track as needed by the application, or created after-the-fact if the initial constraints are unknown to the application.

[Constructor(optional MediaTrackConstraints videoConstraints)]
interface VideoStreamTrack : MediaStreamTrack {
    static sequence<DOMString> getSourceIds ();
};

4.5.2 AudioStreamTrack

[Constructor(optional MediaTrackConstraints audioConstraints)]
interface AudioStreamTrack : MediaStreamTrack {
    static sequence<DOMString> getSourceIds ();
};

10.1.1 Methods

getUserMedia

Prompts the user for permission to use their Web cam or other video or audio input.

Issue 3

The syntax for getUserMedia() is still under discussion. Some questions are: whether it creates media streams or just initializes ones already created; whether it creates streams synchronously (immediate return of MediaStream with callback later when user grants permission) or asynchronously (MediaStream available only upon callback).

The constraints argument is an object of type MediaStreamConstraints.

The successCallback will be invoked with a suitable MediaStream object as its argument if the user accepts valid tracks as described below.

The errorCallback will be invoked if there is a failure in finding valid tracks or if the user denies permission, both as described below.

When the getUserMedia() method is called, the user agent MUST run the following steps:

1. Let constraints be the method's first argument.

2. Let successCallback be the callback indicated by the method's second argument.

3. Let errorCallback be the callback indicated by the method's third argument.

4. Let requestedMediaTypes be the set of media types in constraints with either a dictionary value or a value of "true".

5. Let finalSet be an (initially) empty set.

6. If successCallback is null, abort these steps.

7. For each media type T in requestedMediaTypes,

   1. Let candidateSet be all possible tracks of media type T that the browser could return.

   2. If the value of the T entry of constraints is "true", jump to the step labeled final below. Otherwise, continue.

   3. For each constraint key-value pair in the "mandatory" dictionary,

      1. If the constraint is not supported by the browser, queue a task to invoke errorCallback with a new NavigatorUserMediaError object whose name attribute has the value CONSTRAINT_NOT_SATISFIED and whose constraintName atribute is set to the name of the constraint that caused the error and then jump to the step labeled failure below.

      2. Remove from the candidateSet any track that cannot satisfy the value given for the constraint.

      3. If the candidateSet no longer contains at least one track, queue a task to invoke errorCallback with a new NavigatorUserMediaError object whose name attribute has the value CONSTRAINT_NOT_SATISFIED and whose constraintName atribute is set to the name of the constraint that caused the error and then jump to the step labeled failure below. Otherwise, continue with the next mandatory constraint.

   4. Let the secondPassSet be the current contents of the candidateSet.

   5. For each constraint key-value pair in the "optional" sequence of the constraints that are for the current media type, in order,

      1. If the constraint is not supported by the browser, skip it and continue with the next constraint.

      2. Remove from the secondPassSet any tracks that cannot satisfy the value given for the constraint.

      3. If the secondPassSet is now empty, let the secondPassSet be the current contents of the candidateSet. Otherwise, let the candidateSet be the current contents of the secondPassSet.

   6. Final: Add the tracks in the candidateSet to the finalSet.

• Return, and run the remaining steps asynchronously.

• Optionally, e.g., based on a previously-established user preference, for security reasons, or due to platform limitations, jump to the step labeled failure below.

• Prompt the user in a user agent specific manner for permission to provide the entry script's origin with a MediaStream object representing a media stream.

  The provided media MUST include precisely one track of each media type in requestedMediaTypes from the finalSet. The decision of which tracks to choose from the finalSet is completely up to the user agent and may be determined by asking the user. Unless and until a new set of constraints is provided, the user agent MAY change its choice of track at any point, provided that 1) the new choice does not violate given user permissions, and 2) it notifies the application code by raising an event. It may wish to do this, for example, if the user interface or network congestion changes. Note that no such change will have an effect on the presence or absence of each type of track, merely the contents.

  Issue 4

  Define the event that should be raised when the user agent changes its choice of track.

  User agents are encouraged to default to using the user's primary or system default camera and/or microphone (when possible) to generate the media stream. User agents MAY allow users to use any media source, including pre-recorded media files.

  If the user grants permission to use local recording devices, user agents are encouraged to include a prominent indicator that the devices are "hot" (i.e. an "on-air" or "recording" indicator).

  If the user denies permission, jump to the step labeled failure below. If the user never responds, this algorithm stalls on this step.

• Let stream be the MediaStream object for which the user granted permission.

• Queue a task to invoke successCallback with stream as its argument.

• Abort these steps.

• Failure:

• Let error be a new NavigatorUserMediaError object whose name attribute has the value PERMISSION_DENIED.

• Queue a task to invoke errorCallback with error as its argument.

• The task source for these tasks is the user interaction task source.

Parameter	Type	Nullable	Optional	Description
constraints	MediaStreamConstraints	✔	✘
successCallback	NavigatorUserMediaSuccessCallback	✘	✘
errorCallback	NavigatorUserMediaErrorCallback	✘	✘

Return type: void

10.2.1 Dictionary MediaStreamConstraints Members

audio of type (boolean or MediaTrackConstraints), defaulting to false

Issue 6

Provide definition of audio constraints here.

video of type (boolean or MediaTrackConstraints), defaulting to false

Issue 5

Provide definition of video constraints here.

10.2.2 Dictionary MediaTrackConstraints Members

mandatory of type MediaTrackConstraintSet, nullable, defaulting to null

Issue 7

Provide definition of mandatory constraints here.

optional of type array of MediaTrackConstraint, nullable, defaulting to null

Issue 8

Provide definition of optional constraints here.

10.3.1 Callback NavigatorUserMediaSuccessCallback Parameters

stream of type MediaStream

Issue 9

Add explanation of handleEvent

10.4.1 Attributes

constraintName of type DOMString, readonly , nullable

The name of the constraint that caused the error. Only used for some types of errors. For NavigatorUserMediaError with a name of CONSTRAINT_NOT_SATISFIED , the constraintName attribute in the NavigatorUserMediaError object MUST be set to the name of the constraint that caused the error.

message of type DOMString, readonly , nullable

A human readable description of the error. This string may vary between different user agents.

name of type DOMString, readonly

A string representing the type of error. This string must be one of those defined by the NavigatorUserMediaErrorName enum for the error object to be valid.

10.4.2 Callback NavigatorUserMediaErrorCallback Parameters

error of type NavigatorUserMediaError

Issue 10

Add explanation of handleEvent

12.1.1 Dictionary MinMaxConstraint Members

max of type any

The related constraint's maximum allowed value.

min of type any

The related constraint's minimum allowed value.