Initial Author of this Specification was Ian Hickson, Google Inc., with the following copyright statement:
© Copyright 2004-2011 Apple Computer, Inc., Mozilla Foundation, and Opera Software ASA. You are granted a license to use, reproduce and create derivative works of this document.
All subsequent changes since 26 July 2011 done by the W3C WebRTC Working Group and the Device APIs Working Group are under the following Copyright:
© 2011-2013 W3C® (MIT, ERCIM, Keio, Beihang), All Rights Reserved. Document use rules apply.
For the entire publication on the W3C site the liability and trademark rules apply.
This document defines a set of JavaScript APIs that allow local media, including audio and video, to be requested from a platform.
This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.
This document is not complete. It is subject to major changes and, while early experimentations are encouraged, it is therefore not intended for implementation. The API is based on preliminary work done in the WHATWG.
This document was published by the Web Real-Time Communication Working Group and Device APIs Working Group as an Editor's Draft. If you wish to make comments regarding this document, please send them to public-media-capture@w3.org (subscribe, archives). All comments are welcome.
Publication as an Editor's Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.
This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures (Web Real-Time Communication Working Group, Device APIs Working Group) made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.
This section is non-normative.
Access to multimedia streams (video, audio, or both) from local devices (video cameras, microphones, Web cams) can have a number of uses, such as real-time communication, recording, and surveillance.
This document defines the APIs used to get access to local devices that can generate multimedia stream data. This document also defines the MediaStream API by which JavaScript is able to manipulate the stream data or otherwise process it.
As well as sections marked as non-normative, all authoring guidelines, diagrams, examples, and notes in this specification are non-normative. Everything else in this specification is normative.
The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL in this specification are to be interpreted as described in [RFC2119].
This specification defines conformance criteria that apply to a single product: the user agent that implements the interfaces that it contains.
Implementations that use ECMAScript to implement the APIs defined in this specification must implement them in a manner consistent with the ECMAScript Bindings defined in the Web IDL specification [WEBIDL], as this specification uses that specification and terminology.
The
EventHandler
interface represents a callback used for event
handlers as defined in [HTML5].
The concepts queue a task and fires a simple event are defined in [HTML5].
The terms event handlers and event handler event types are defined in [HTML5].
A source is the "thing" providing the source of a media stream track. The source is the broadcaster of the media itself. A source can be a physical webcam, microphone, local video or audio file from the user's hard drive, network resource, or static image.
Some sources have an identifier which must be unique to the application (un-guessable by another application) and persistent between application sessions (e.g., the identifier for a given source device/application must stay the same, but not be guessable by another application). Sources that must have an identifier are camera and microphone sources; local file sources are not required to have an identifier. Source identifiers let the application save, identify the availability of, and directly request specific sources.
Other than the identifier, other bits of source identity are never directly available to the application until the user agent connects a source to a track. Once a source has been "released" to the application (either via a permissions UI, pre-configured allow-list, or some other release mechanism) the application will be able discover additional source-specific capabilities.
Sources do not have constraints -- tracks have constraints. When a source is connected to a track, it must conform to the constraints present on that track (or set of tracks).
Sources will be released (un-attached) from a track when the track is ended for any reason.
On the
object,
sources are represented by a MediaStreamTrack
sourceType
attribute. The behavior of APIs associated with the source's
capabilities and state change depending on the source
type.
Sources have capabilities
and state
. The capabilities and state
are "owned" by the source and are common to any (multiple)
tracks that happen to be using the same source (e.g., if two
different tracks objects bound to the same source ask for
the same capability or state information, they will get back
the same answer).
State refers to the immediate, current value of the source's (optionally constrained) capabilities. State is always read-only.
A source's state can change dynamically over time due to environmental conditions, sink configurations, or constraint changes. A source's state must always conform to the current set of mandatory constraints that all of the tracks it is bound to have defined, and should do its best to conform to the set of optional constraints specified.
A source's state is directly exposed to audio and video track objects through individual read-only attributes. These attributes share the same name as their corresponding capabilities and constraints.
Events are available that signal to the application that source state has changed.
A conforming user-agent must support all the state names defined in this spec.
Source capabilities are the intrinsic "features" of a source object. For each source state, there is a corresponding capability that describes whether it is supported by the source and if so, what the range of supported values are. Capabilities are expressed as either a series of states (for enumerated-type capabilities) or as a min/max range.
The values of the supported capabilities must be normalized to the ranges and enumerated types defined in this specification.
Capabilities return the same underlying per-source capabilities, regardless of any user-supplied constraints present on the source (capabilities are independent of constraints).
Source capabilities are effectively constant. Applications should be able to depend on a specific source having the same capabilities for any session.
Constraints are an optional feature for restricting the range of allowed variability on a source. Without provided constraints, implementations are free to select a source's state from the full range of its supported capabilities, and to adjust that state at any time for any reason.
Constraints may be optional or mandatory. Optional constraints are represented by an ordered list, mandatory constraints are an unordered set. The order of the optional constraints is from most important (at the head of the list) to least important (at the tail of the list).
Constraints are stored on the track object, not the source. Each track can be optionally initialized with constraints, or constraints can be added afterward through the constraint APIs defined in this spec.
Applying track level constraints to a source is conditional based on the type of source. For example, read-only sources will ignore any specified constraints on the track.
It is possible for two tracks that share a unique source to apply contradictory constraints. Under such contradictions, the implementation will mute both tracks and notify them that they are over-constrained.
Events are available that allow the application to know when constraints cannot be met by the user agent. These typically occur when the application applies constraints beyond the capability of a source, contradictory constraints, or in some cases when a source cannot sustain itself in over-constrained scenarios (overheating, etc.).
Constraints that are intended for video sources will be ignored by audio sources and vice-versa. Similarly, constraints that are not recognized will be preserved in the constraint structure, but ignored by the UA. This will allow future constraints to be defined in a backward compatible manner.
A correspondingly-named constraint exists for each corresponding source state name and capability name. In general, user agents will have more flexibility to optimize the media streaming experience the fewer constraints are applied.
The
interface is used to represent
streams of media data, typically (but not necessarily) of audio and/or
video content, e.g. from a local camera. The data from a
MediaStream
object does not necessarily have a
canonical binary form; for example, it could just be "the video currently
coming from the user's video camera". This allows user agents to
manipulate media streams in whatever fashion is most suitable on the
user's platform.MediaStream
Each
object can contain zero or more
tracks, in particular audio and video tracks. All tracks in a MediaStream
are intended to be synchronized when rendered. Different MediaStreams do
not need to be synchronized.MediaStream
Each track in a MediaStream object has a corresponding
object.MediaStreamTrack
A
represents content comprising
one or more channels, where the channels have a defined well known
relationship to each other (such as a stereo or 5.1 audio signal).MediaStreamTrack
A channel is the smallest unit considered in this API specification.
A
object has an input and an output.
The input depends on how the object was created: a
MediaStream
object generated by a MediaStream
getUserMedia()
call (which is
described later in this document), for instance, might take its input
from the user's local camera. The output of the object controls how the
object is used, e.g., what is saved if the object is written to a file or
what is displayed if the object is used in a video
element.
Each track in a
object can be
disabled, meaning that it is muted in the object's output. All tracks are
initially enabled.MediaStream
A
can be finished, indicating
that its inputs have forever stopped providing data.MediaStream
The output of a
object MUST correspond
to the tracks in its input. Muted audio tracks MUST be replaced with
silence. Muted video tracks MUST be replaced with blackness.MediaStream
A new
object can be created from
accessible media sources (that does not require any additional
permissions) using the MediaStream
MediaStream()
constructor. The constructor argument can either be an existing
object, in which case all the tracks of
the given stream are added to the new MediaStream
object, or an array of MediaStream
objects. The
latter form makes it possible to compose a stream from different source
streams.MediaStreamTrack
Both
and MediaStream
objects can be cloned. This allows for
greater control since the separate instances can be manipulated and
consumed individually. A cloned MediaStreamTrack
contains clones of all member tracks from the
original stream.MediaStream
When a
object is being generated
from a local file (as opposed to a live audio/video source), the user
agent SHOULD stream the data from the file in real time, not all at once.
The MediaStream
MediaStream
object is also used in contexts outside
getUserMedia
, such as [WEBRTC10]. In both cases, ensuring
a realtime stream reduces the ease with which pages can distinguish live
video from pre-recorded video, which can help protect the user's
privacy.
The MediaStream()
constructor composes a new stream out of existing tracks. It takes an
optional argument of type
or an array of
MediaStream
objects. When the constructor is
invoked, the UA must run the following steps:MediaStreamTrack
Let stream be a newly constructed
object.MediaStream
Initialize stream's id
attribute to a newly generated
value.
If the constructor's argument is present, run the sub steps that corresponds to the argument type.
Array
of
objects:MediaStreamTrack
Run the following sub steps for each
in the array:MediaStreamTrack
Add track: Let track be the
about to be processed.MediaStreamTrack
If track has ended, then abort these steps and continue with the next track (if any).
Add track to stream's track set.
Run the sub steps labeled Add track (above) for every
in the argument stream's
track set.
MediaStreamTrack
If stream's track set is
empty, set stream's inactive
attribute to
true
, otherwise set it to false
.
Return stream.
A
can have multiple audio and video
sources (e.g. because the user has multiple microphones, or because the
real source of the stream is a media resource with many media tracks).
The stream represented by a MediaStream
thus has zero
or more tracks.MediaStream
The tracks of a
are stored in a
track set. The track set MUST contain the
MediaStream
objects that correspond
to the tracks of the stream. The relative order of the tracks in the set
is user agent defined and the API will never put any requirements on the
order. The proper way to find a specific MediaStreamTrack
object in the set is to look it up by its
MediaStreamTrack
id
.
An object that reads data from the output of a
is referred to as a
MediaStream
consumer. The list of
MediaStream
consumers currently include the media
elements [HTML5], MediaStream
RTCPeerConnection
[WEBRTC10],
MediaRecorder
[mediastream-rec] and
ImageCapture
[mediastream-imagecap].
consumers must be able to
handle tracks being added and removed. This behavior is specified per
consumer.MediaStream
A
object is said to be
inactive when it does not have any tracks
or all tracks belonging to the stream have ended. Otherwise the
stream is active. A MediaStream
can start its life as
inactive if it is constructed without any tracks.MediaStream
When a
goes from being active to
inactive, the user agent MUST queue a task that sets the object's
MediaStream
inactive
attribute
to true
and fire a simple event named inactive
at the object. When a
goes from being inactive to active, the
user agent MUST queue a task that sets the object's MediaStream
inactive
attribute to
false
and fire a simple event named active
at the object.
If the stream's activity status changed due to a user request, the task source for this task is the user interaction task source. Otherwise the task source for this task is the networking task source.
[ Constructor,
Constructor (MediaStream
stream),
Constructor (sequence<MediaStreamTrack
> tracks)]
interface MediaStream : EventTarget {
readonly attribute DOMString id;
sequence<MediaStreamTrack
> getAudioTracks ();
sequence<MediaStreamTrack
> getVideoTracks ();
MediaStreamTrack
? getTrackById (DOMString trackId);
void addTrack (MediaStreamTrack
track);
void removeTrack (MediaStreamTrack
track);
MediaStream
clone ();
readonly attribute boolean inactive;
attribute EventHandler onactive;
attribute EventHandler oninactive;
attribute EventHandler onaddtrack;
attribute EventHandler onremovetrack;
};
MediaStream
MediaStream
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
stream |
| ✘ | ✘ |
MediaStream
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
tracks | sequence< | ✘ | ✘ |
id
of type DOMString, readonly When a
object is created, the
user agent MUST generate a globally unique identifier string, and
MUST initialize the object's MediaStream
id
attribute to that string.
Such strings MUST only use characters in the ranges U+0021, U+0023 to
U+0027, U+002A to U+002B, U+002D to U+002E, U+0030 to U+0039, U+0041
to U+005A, U+005E to U+007E, and MUST be 36 characters long.
The id
attribute MUST return the value to which it was initialized when the
object was created.
inactive
of type boolean, readonly The MediaStream.inactive
attribute returns true if the
is
inactive, and false otherwise.MediaStream
When a
object is created, its
MediaStream
inactive
attribute MUST be set to false, unless stated otherwise (for example
by the MediaStream()
constructor algorithm).
onactive
of type EventHandler, active
, MUST be supported by
all objects implementing the MediaStream
interface.onaddtrack
of type EventHandler, addtrack
, MUST be supported by
all objects implementing the MediaStream
interface.
oninactive
of type EventHandler, inactive
, MUST be supported by
all objects implementing the MediaStream
interface.onremovetrack
of type EventHandler, removetrack
, MUST be
supported by all objects implementing the MediaStream
interface.addTrack
Adds the given
to this
MediaStreamTrack
.MediaStream
When the addTrack()
method is
invoked, the user agent MUST run the following steps:
Let track be the
argument and stream this MediaStreamTrack
object.MediaStream
If stream is finished, throw an
INVALID_STATE_ERR
exception.
If track is already in stream's track set, then abort these steps.
Add track to stream's track set.
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
track |
| ✘ | ✘ |
void
clone
Clones the given
and all its
tracks.MediaStream
When the clone()
method
is invoked, the user agent MUST run the following steps:
Let streamClone be a newly constructed
object.MediaStream
Initialize streamClone's id
attribute to a newly
generated value.
Let trackSetClone be a list that contains the
result of running
MediaStreamTrack.clone()
on all the tracks in this
stream.
Let trackSetClone be streamClone's track set.
MediaStream
getAudioTracks
Returns a sequence of
objects
representing the audio tracks in this stream.MediaStreamTrack
The getAudioTracks()
method MUST return a sequence that represents a snapshot of all the
objects in this stream's
track set whose MediaStreamTrack
kind
is equal to
"audio
". The conversion from the track set to the sequence is user agent defined and
the order does not have to stable between calls.
sequence<MediaStreamTrack
>
getTrackById
The getTrackById()
method MUST return the first
object in this stream's track set whose
MediaStreamTrack
id
is equal to
trackId. The method MUST return null if no track matches
the trackId argument.
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
trackId | DOMString | ✘ | ✘ |
MediaStreamTrack
, nullablegetVideoTracks
Returns a sequence of
objects
representing the video tracks in this stream.MediaStreamTrack
The getVideoTracks()
method MUST return a sequence that represents a snapshot of all the
objects in this stream's
track set whose MediaStreamTrack
kind
is equal to
"video
". The conversion from the track set to the sequence is user agent defined and
the order does not have to stable between calls.
sequence<MediaStreamTrack
>
removeTrack
Removes the given
from this
MediaStreamTrack
.MediaStream
When the removeTrack()
method
is invoked, the user agent MUST run the following steps:
Let track be the
argument and stream this MediaStreamTrack
object.MediaStream
If stream is finished, throw an
INVALID_STATE_ERR
exception.
If track is in stream's track set, remove it.
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
track |
| ✘ | ✘ |
void
A
object represents a media
source in the user agent. Several MediaStreamTrack
objects can represent the same media source, e.g., when the user chooses
the same camera in the UI shown by two consecutive calls to
MediaStreamTrack
getUserMedia()
.
Note that a web application can revoke all given permissions
with MediaStreamTrack.stop()
.
A
has three stages in its
lifecycle; MediaStreamTrack
new
, live
and ended
.
A track begins as new
prior to being connected to an
active source.
Once connected, the started
event fires and the track
becomes live
. In the live
state, the track is
active and media is available for rendering at a
consumer.MediaStream
A muted or disabled
renders either
silence (audio), black frames (video), or a zero-information-content
equivalent. For example, a video element sourced by a muted or disabled
MediaStreamTrack
(contained within a
MediaStreamTrack
), is playing but the rendered content
is the muted output.
MediaStream
The muted/unmuted state of a track reflects if the source
provides any media at this moment. The enabled/disabled state is under
application control and determines if the track outputs media (to its
consumers). Hence, media from the source only flows when a
object is both unmuted and enabled.MediaStreamTrack
A
is muted when the source is temporarily unable to
provide the track with data. A track can be muted by a user. Often this
action is outside the control of the application. This could be as a
result of the user hitting a hardware switch, or toggling a control in
the operating system or browser chrome. A track can also be muted by the
user agent. For example, a track that is a member of a MediaStreamTrack
, received via a MediaStream
RTCPeerConnection
[WEBRTC10], is muted if the application on the other side
disables the corresponding track in the MediaStream
being
sent.
Applications are able to enable or
disable a
to prevent it from
rendering media from the source. A muted track will however, regardless
of the enabled state, render silence and blackness. A disabled track is
logically equivalent to a muted track, from a consumer point of view.
MediaStreamTrack
For a newly created
object, the
following applies. The track is always enabled unless stated otherwise
(for examlpe when cloned) and the muted state reflects the state of the
source at the time the track is created.MediaStreamTrack
A
object is said to end
when the source of the track is disconnected or exhausted.MediaStreamTrack
When a
object ends for any reason
(e.g., because the user rescinds the permission for the page to use the
local camera, or because the data comes from a finite file and the file's
end has been reached and the user has not requested that it be looped, or
because the UA has instructed the track to end for any reason, or because
the reference count of the track's underlying media source has reached
zero), it is said to be ended. When track instance
track ends for any reason other than the MediaStreamTrack
stop()
method being invoked on the
object that represents
track, the user agent MUST queue a task that runs the
following steps:MediaStreamTrack
If the track's readyState
attribute
has the value ended
already, then abort these steps.
(The stop()
method was probably called just before the track stopped for other
reasons.)
Set track's readyState
attribute to
ended
.
Fire a simple event named ended
at the object.
If the end of the stream was reached due to a user request, the event source for this event is the user interaction event source.
Constraints are independent of sources. However, if
the sourceType
is "none",
the readonly
attribute
is true
, or the remote
attribute is true
, the track's constraints will
not be applied to the source.
Whether
were provided at
track initialization time or need to be established later at
runtime, the APIs defined below allow the retrieval and
manipulation of the constraints currently established on a
track.MediaTrackConstraints
Each track maintains an internal version of
the
structure, namely a
mandatory set of constraints (no duplicates), and an optional
ordered list of individual constraint objects (may contain
duplicates). The internal stored constraint structure is only
exposed to the application using the existing
MediaTrackConstraints
,
MediaTrackConstraints
MediaTrackConstraintSet
, MediaTrackConstraint
,
and similarly-derived-type dictionary objects.
When track constraints change, a user agent must queue a task to evaluate those changes
when the task queue is next serviced. Similarly, if
the sourceType
changes, then the user agent should perform the same actions to
re-evaluate the constraints of each track affected by that
source change.
interface MediaStreamTrack : EventTarget {
readonly attribute DOMString kind;
readonly attribute DOMString id;
readonly attribute DOMString label;
attribute boolean enabled;
readonly attribute boolean muted;
attribute EventHandler onmute;
attribute EventHandler onunmute;
readonly attribute boolean _readonly;
readonly attribute boolean remote;
readonly attribute MediaStreamTrackState
readyState;
attribute EventHandler onstarted;
attribute EventHandler onended;
static void getSources (SourceInfoCallback
resultCallback);
MediaTrackConstraints
? constraints ();
MediaSourceStates
states ();
(AllVideoCapabilities
or AllAudioCapabilities
) capabilities ();
void applyConstraints (MediaTrackConstraints
constraints);
attribute EventHandler onoverconstrained;
MediaStreamTrack
clone ();
void stop ();
};
enabled
of type boolean, The MediaStreamTrack.enabled
attribute, on getting, MUST return the last value to which it was
set. On setting, it MUST be set to the new value, and then, if the
object is still associated with
a track, MUST enable the track if the new value is true, and disable
it otherwise.MediaStreamTrack
Thus, after a
is
disassociated from its track, its MediaStreamTrack
enabled
attribute still
changes value when set; it just doesn't do anything with that new
value.
id
of type DOMString, readonly Unless a
object is created as
a part a of special purpose algorithm that specifies how the track id
must be initialized, the user agent MUST generate a globally unique
identifier string and initialize the object's MediaStreamTrack
id
attribute to that string.
An example of an algorithm that specifies how the track id must be
initialized is the algorithm to represent an incoming network
component with a
object.
[WEBRTC10]MediaStreamTrack
MediaStreamTrack.id
attribute MUST return the value
to which it was initialized when the object was created.
kind
of type DOMString, readonly The MediaStreamTrack.kind
attribute MUST return the string "audio
" if the object
represents an audio track or "video
" if object represents
a video track.
label
of type DOMString, readonly User agents MAY label audio and video sources (e.g., "Internal
microphone" or "External USB Webcam"). The MediaStreamTrack.label
attribute MUST return the label of the object's corresponding track,
if any. If the corresponding track has or had no label, the attribute
MUST instead return the empty string.
Thus the kind
and label
attributes do not
change value, even if the
object
is disassociated from its corresponding track.MediaStreamTrack
muted
of type boolean, readonly The MediaStreamTrack.muted
attribute MUST return true
if the track is muted, and false
otherwise.
onended
of type EventHandler, ended
, MUST be supported by
all objects implementing the MediaStreamTrack
interface.onmute
of type EventHandler, mute
, MUST be supported by
all objects implementing the MediaStreamTrack
interface.onoverconstrained
of type EventHandler, overconstrained
, MUST be supported by
all objects implementing the MediaStreamTrack
interface.onstarted
of type EventHandler, started
, MUST be supported by
all objects implementing the MediaStreamTrack
interface.onunmute
of type EventHandler, unmute
, MUST be supported by
all objects implementing the MediaStreamTrack
interface.readonly
of type boolean, readonly readonly
attribute MUST return the value true
.
Otherwise, it must return the value false
.
readyState
of type MediaStreamTrackState
, readonly The readyState
attribute represents the state of the track. It MUST return the value
to which the user agent last set it.
remote
of type boolean, readonly RTCPeerConnection
, the remote
attribute MUST return the value true
.
Otherwise, it must return the value false
.
applyConstraints
This API will replace all existing constraints with the provided constraints (if existing constraints exist). Otherwise, it will apply the newly provided constraints to the track.
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
constraints |
| ✘ | ✘ | A new constraint structure to apply to this track. |
void
capabilities
Returns a dictionary with all of the capabilities for the
track type. If the track type is
,
the VideoStreamTrack
AllVideoCapabilities
dictionary is returned. If the track type
is
,
the AudioStreamTrack
dictionary is returned.AllAudioCapabilities
Given that implementations of various hardware may not
exactly map to the same range, an
implementation should make a reasonable attempt to
translate and scale the hardware's setting onto the mapping
provided by this specification. If this is not possible due
to the user agent's inability to retrieve a given
capability from a source, then for CapabilityRange
-typed
capabilities, the min
and max
fields will not be present on the returned dictionary, and
the supported
field will
be false
. For
-typed
capabilities, a suitable CapabilityList
"notavailable"
value
will be the sole capability in the list.
An example of the user agent providing an alternative mapping: if a source supports a hypothetical fluxCapacitance state whose type is a CapabilityRange, and the state is defined in this specification to be the range from -10 (min) to 10 (max), but the source's (hardware setting) for fluxCapacitance only supports values of "off" "medium" and "full", then the user agent should map the range value of -10 to "off", 10 should map to "full", and 0 should map to "medium". Constraints imposing a strict value of 3 will cause the user agent to attempt to set the value of "medium" on the hardware, and return a fluxCapacitance state of 0, the closest supported setting. No error event is raised in this scenario.
objects should order their enumerated
values from minimum to maximum where it makes sense, or in
the order defined by the enumerated type where
applicable.CapabilityList
See the
and AllVideoCapabilities
dictionaries for details on the expected types for the various
supported state names.AllAudioCapabilities
(AllVideoCapabilities
or AllAudioCapabilities
)
clone
Clones the given
.MediaStreamTrack
When the clone()
method
is invoked, the user agent MUST run the following steps:
Let trackClone be a newly constructed
object.MediaStreamTrack
Initialize trackClone's id
attribute to a newly
generated value.
Let trackClone inherit this track's underlying
source,
kind
,
label
and
enabled
attributes.
Return trackClone.
MediaStreamTrack
constraints
mandatory
field will not be present (it will be
undefined). If no optional constraints have been defined,
the optional
field will not be present (it will be
undefined). If neither optional, nor mandatory constraints have been
created, the value null
is returned.
MediaTrackConstraints
, nullablegetSources
, staticThe static
getSources()
method collects authorized
information for all available sources.
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
resultCallback |
| ✘ | ✘ |
void
states
MediaSourceStates
stop
When a
object's MediaStreamTrack
stop()
method
is invoked, the user agent MUST run following steps:
Let track be the current
object.MediaStreamTrack
If track has no source attached
(sourceType
is "none") or if the source is
provided by an RTCPeerConnection
, then abort these
steps.
Set track's readyState
attribute to ended
.
Permanently stop the generation of data for track's source. If the data is being generated from a live source (e.g., a microphone or camera), then the user agent SHOULD remove any active "on-air" indicator for that source. If the data is being generated from a prerecorded source (e.g. a video file), any remaining content in the file is ignored.
This will effectively
end all other
objects sharing the same
source as track.MediaStreamTrack
The task source for the tasks
queued for the stop()
method is the DOM
manipulation task source.
void
enum MediaStreamTrackState {
"new",
"live",
"ended"
};
Enumeration description | |
---|---|
new | The track type is new and has not been initialized (connected to a source of any kind). This state implies that the track's label will be the empty string. |
live |
The track is active (the track's underlying media source is making a best-effort attempt to provide data in real time). The output of a track in the |
ended |
The track has ended (the track's underlying media source is no longer providing data, and will never provide more data for this track). Once a track enters this state, it never exits it. For example, a video track in a
|
enum SourceTypeEnum {
"none",
"camera",
"microphone"
};
Enumeration description | |
---|---|
none | This track has no source. This is the case when the
track is in the "new" or "ended"
readyState . |
camera | A valid source type only for
s. The source is a local
video-producing camera source. |
microphone | A valid source type only for
s. The source is a local
audio-producing microphone source. |
callback SourceInfoCallback = void (sequence<SourceInfo
> sourceInfoList);
SourceInfoCallback
ParameterssourceInfoList
of type sequence<SourceInfo
>SourceInfo
objects representing
the result of a call to MediaStreamTrack.getSources()
.dictionary SourceInfo {
DOMString sourceId;
DOMString kind;
DOMString label;
};
SourceInfo
Memberskind
of type DOMStringlabel
of type DOMStringlabel
attribute will be filled in
with exactly the same value as would have been returned from
a call to getUserMedia()
with a
constraint specifying this
source's sourceId
.sourceId
of type DOMStringMediaSourceStates
dictionary.enum VideoFacingModeEnum {
"user",
"environment",
"left",
"right"
};
Enumeration description | |
---|---|
user | The source is facing toward the user (a self-view camera). |
environment | The source is facing away from the user (viewing the environment). |
left | The source is facing to the left of the user. |
right | The source is facing to the right of the user. |
When either the "noaccess" or "peerIdentity" constraints is applied to a MediaStreamTrack, the track shall be isolated so that its content is not accessible to the content JS. An isolated media stream may be used for two purposes:
Displayed in an appropriate tag (e.g., a video or audio element). The video element MUST have a unique origin so that it is inaccessible to the content JS. This is the same security mechanism as is used with an ordinary audio or video element which has a src= property from a separate origin.
Used as the argument to addStream() for a PeerConnection, subject to the restrictions detailed in the WebRTC document.
When the noaccess=true constraint applies to a track, that track may be added to any PeerConnection.
Open Issue: The editors worry that the above paragraph is just wrong. If the track can be added to a PeerConnection that is connect to another PeerConenction in the same application, the application could get access to the data. We sugest this should be changed from "may be added" to "may not be added". This will allow noaccess=true to be used for things like hair check dialogs.
When the peerIdentity=foo constraint applies to a track, then that track may be added only to PeerConnections with compatible peer identities as described in the WebRTC document.
Both the noaccess and peerIdentity constraints must be mandatory. Any use of them in the optional block must trigger an error.
The addtrack
and
removetrack
events
use the
interface.MediaStreamTrackEvent
Firing a track event named
e with a
track means that an event with the name e, which
does not bubble (except where otherwise stated) and is not cancelable
(except where otherwise stated), and which uses the
MediaStreamTrack
interface with the
MediaStreamTrackEvent
track
attribute set to track, MUST be created and dispatched at the
given target.
dictionary MediaStreamTrackEventInit : EventInit {
MediaStreamTrack
? track;
};
[ Constructor (DOMString type, MediaStreamTrackEventInit
eventInitDict)]
interface MediaStreamTrackEvent : Event {
readonly attribute MediaStreamTrack
track;
};
MediaStreamTrackEvent
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
type | DOMString | ✘ | ✘ | |
eventInitDict |
| ✘ | ✘ |
track
of type MediaStreamTrack
, readonly The track
attribute
represents the
object associated
with the event.MediaStreamTrack
MediaStreamTrackEventInit
Memberstrack
of type MediaStreamTrack
, nullable
The
object cannot be
instantiated directly. To create an instance of
a MediaStreamTrack
, one of its derived track
types may be instantiated. These derived types are defined in
this section.MediaStreamTrack
Note that the camera's green light
doesn't come on
when a new track is created; nor does the user get prompted to
enable the camera/microphone. Those actions only happen after
the developer has requested that a media stream
containing "new"
tracks be bound to a source
via getUserMedia()
. Until that point tracks
are inert.
Video tracks may be instantiated with optional media track constraints. These constraints can be later modified on the track as needed by the application, or created after-the-fact if the initial constraints are unknown to the application.
Example:
objects are instantiated in JavaScript using the new
operator:VideoStreamTrack
new VideoStreamTrack
();
or
newVideoStreamTrack
( { optional: [ {sourceId
: "20983-20o198-109283-098-09812" }, {width
: { min: 800, max: 1200 }}, {height
: { min: 600 }}] });
[ Constructor (optional MediaTrackConstraints
videoConstraints)]
interface VideoStreamTrack : MediaStreamTrack
{
};
VideoStreamTrack
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
videoConstraints |
| ✘ | ✔ |
Example:
objects are instantiated in JavaScript using the new operator:AudioStreamTrack
new AudioStreamTrack
();
or
newAudioStreamTrack
( { optional: [ {sourceId
: "64815-wi3c89-1839dk-x82-392aa" }, {gain
: 0.5 }] });
[ Constructor (optional MediaTrackConstraints
audioConstraints)]
interface AudioStreamTrack : MediaStreamTrack
{
};
AudioStreamTrack
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
audioConstraints |
| ✘ | ✔ |
Browsers provide a media pipeline from sources to sinks. In a browser, sinks are the <img>, <video> and <audio> tags. Traditional sources include camera, microphones, streamed content, files and web resources. The media produced by these sources typically does not change over time - these sources can be considered to be static.
The sinks that display these sources to the user (the actual
tags themselves) have a variety of controls for manipulating the
source content. For example, an <img> tag scales down a
huge source image of 1600x1200 pixels to fit in a rectangle
defined with width="400"
and height="300"
.
The getUserMedia API adds dynamic sources such as microphones and cameras - the characteristics of these sources can change in response to application needs. These sources can be considered to be dynamic in nature. A <video> element that displays media from a dynamic source can either perform scaling or it can feed back information along the media pipeline and have the source produce content more suitable for display.
Note: This sort of feedback loop is obviously just enabling an "optimization", but it's a non-trivial gain. This optimization can save battery, allow for less network congestion, etc...
Note that MediaStream
sinks
(such as <video>
, <audio>
,
and even RTCPeerConnection
) will continue to have
mechanisms to further transform the source stream beyond that
which the states, capabilities,
and constraints described in this specification
offer. (The sink transformation options, including
those of RTCPeerConnection
, are outside the scope of
this specification.)
The act of changing or applying a track constraint may affect
the state of all
tracks sharing that source and consequently all down-level sinks
that are using that source. Many sinks may be able to take these
changes in stride, such as the <video>
element or RTCPeerConnection
. Others like the
Recorder API may fail as a result of a source state change.
The RTCPeerConnection
is an interesting object
because it acts simultaneously as both a
sink and a source for over-the-network
streams. As a sink, it has source transformational capabilities
(e.g., lowering bit-rates, scaling-up or down resolutions,
adjusting frame-rates), and as a source it could have its own
states changed by a track source (though in this specification
sources with the remote
attribute set to
true do not consider the current constraints applied to a
track).
To illustrate how changes to a given source impact various
sinks, consider the following example. This example only uses
width and height, but the same principles apply to any of
the states exposed
in this proposal. In the first figure a home client has obtained
a video source from its local video camera. The source's width
and height state are 800 pixels by 600 pixels,
respectively. Three
objects on
the home client contain tracks that use this
same MediaStream
sourceId
. The three media streams are
connected to three different sinks: a <video>
element (A), another <video>
element (B), and
a peer connection (C). The peer connection is streaming the
source video to an away client. On the away client there are two
media streams with tracks that use the peer connection as a
source. These two media streams are connected to
two <video>
element sinks (Y and Z).
Note that at this moment, all of the sinks on the home client must apply a transformation to the original source's provided state dimensions. A is scaling the video up (resulting in loss of quality), B is scaling the video down, and C is also scaling the video up slightly for sending over the network. On the away client, sink Y is scaling the video way down, while sink Z is not applying any scaling.
Using the constraint APIs, the home client's video source is changed to a higher resolution (1920 by 1200 pixels).
Note that the source change immediately affects all of the sinks on the home client, but does not impact any of the sinks (or sources) on the away client. With the increase in the home client source video's dimensions, sink A no longer has to perform any scaling, while sink B must scale down even further than before. Sink C (the peer connection) must now scale down the video in order to keep the transmission constant to the away client.
While not shown, an equally valid states change request could be made of the away client video source (the peer connection on the away client's side). This would not only impact sink Y and Z in the same manner as before, but would also cause re-negotiation with the peer connection on the home client in order to alter the transformation that it is applying to the home client's video source. Such a change would not change anything related to sink A or B or the home client's video source.
Note that this specification does not define a mechanism by which a change to the away client's video source could automatically trigger a change to the home client's video source. Implementations may choose to make such source-to-sink optimizations as long as they only do so within the constraints established by the application, as the next example demonstrates.
It is fairly obvious that changes to a given source will
impact sink consumers. However, in some situations changes to a
given sink may also be cause for implementations to adjust the
characteristics of a source's stream. This is illustrated in the
following figures. In the first figure below, the home client's
video source is sending a video stream sized at 1920 by 1200
pixels. The video source is also unconstrained, such that the
exact source dimensions are flexible as far as the application
is concerned. Two
objects contain
tracks with the same MediaStream
sourceId
, and
those
s are connected to two
different MediaStream
<video>
element sinks A and B. Sink
A has been sized to width="1920"
and height="1200"
and is displaying the source's
video content without any transformations. Sink B has been sized
smaller and, as a result, is scaling the video down to fit its
rectangle of 320 pixels across by 200 pixels down.
When the application changes sink A to a smaller dimension (from 1920 to 1024 pixels wide and from 1200 to 768 pixels tall), the browser's media pipeline may recognize that none of its sinks require the higher source resolution, and needless work is being done both on the part of the source and on sink A. In such a case and without any other constraints forcing the source to continue producing the higher resolution video, the media pipeline MAY change the source resolution:
In the above figure, the home client's video source resolution was changed to the greater of that from sinkA and from sinkB in order to optimize playback. While not shown above, the same behavior could apply to peer connections and other sinks.
It is possible that constraints can be applied to a track which a source is unable to satisfy. When this happens, the user agent is required to fire an "overconstrained" event to the track informing it of this condition, and the track becomes muted. There is no mandatory side-effect on the source itself as a result of this condition.
When multiple tracks share the same source (as illustrated in the previous figures), it is also possible that two (or more) tracks can apply contradictory constraints on the source. Since there is only a single thread of control, it is always possible for the browser to determine which track created the overconstrained condition. In this situation, the user agent MUST send the "overconstrained" event only to the track that created the condition and MUST not apply any of the constraints newly requested for that track. Here is an example of this behavior.
In this example, two media streams each have a video track that share the same source. The first track has a mandatory constraint forcing on the source's fill light. That track is connected to sink N. Sink N has a width and height of 800 by 600 pixels and is scaling down the source's resolution of 1024 by 768 to fit. The other track initially has no constraints applied; it is connected to sink P. Sink P has a width and height equal to that of the source.
Now, the second track adds a mandatory constraint that the fill light should be forced off. At this point, both mandatory constraints cannot be satisfied by the source (the fill light cannot be simultaneously on and off at the same time). The second track is transitioned into the muted state and receives an "overconstrained" event. At the same time, the source notes that its remaining active sink only requires a resolution of 800 by 600 and so it adjusts its resolution down to match (this is an optional optimization that the user agent is allowed to make given the situation).
At this point, it is the responsibility of the application to fix the problem that led to the overconstrained situation (by either removing the fill light mandatory constraint on the second track, or by changing/removing the fill light mandatory constraint on the first track).
There is a variable associated with each capability that
represents the state of the source with respect to that
capability, the actual setting in use by the source. In the same
way that the current set of constraints can be returned on a
track using the constraints()
method,
the states()
method on a track returns the
values of the state variables associated with all capabilities,
as the
object. Only
states appropriate to the MediaSourceStates
sourceType
are
returned.
Note: The following specific list(s) of states DOES NOT REFLECT CONSENSUS. Many states beyond these have been proposed, and the ones listed do not have universal support. The ones below are included **** ONLY **** to provide a starting point so we can see concrete examples of what real states might look like. The particular set below was chosen to match the **also temporary** set of constraints.
dictionary MediaSourceStates {
SourceTypeEnum
sourceType;
DOMString sourceId;
unsigned long? width;
unsigned long? height;
float? frameRate;
float? aspectRatio;
VideoFacingModeEnum
? facingMode;
unsigned long? volume;
};
MediaSourceStates
MembersaspectRatio
of type float, nullablefacingMode
of type VideoFacingModeEnum
, nullableframeRate
of type float, nullableheight
of type unsigned long, nullablesourceId
of type DOMStringsourceType
of type SourceTypeEnum
volume
of type unsigned long, nullablewidth
of type unsigned long, nullableEach constraint that is supported by an implementation MUST
have an associated capability that will be returned in the
result of a call to the capabilities()
method.
Capabilities are provided as either a min/max range or a list of enumerated values. Min/max capabilities are always provided for constraints that are not enumerated types. Listed capabilities are always provided for constraints corresponding to enumerated types.
dictionary CapabilityRange {
any max;
any min;
boolean supported;
};
CapabilityRange
Membersmax
of type anyThe type of this value is specific to the capability.
If the related capability is not supported by the source,
then this field will not be provided by the user agent (it
will be undefined
).
min
of type anyThe type of this value is specific to the capability.
If the related capability is not supported by the source,
then this field will not be provided by the user agent (it
will be undefined
).
supported
of type booleantrue
if the capability is
supported, false otherwise.typedef sequence<DOMString> CapabilityList;
Capability Lists are just an array of
supported DOMString
values from the possible
superset of values described by each
state's enumerated type.
Note: The following specific list(s) of capabilities DOES NOT REFLECT CONSENSUS. Many capabilities beyond these have been proposed, and the ones listed do not have universal support. The ones below are included **** ONLY **** to provide a starting point so we can see concrete examples of what real capabilities might look like. The particular sets below were chosen to match the **also temporary** set of constraints.
dictionary AllVideoCapabilities {
CapabilityList
? sourceType;
CapabilityList
? sourceId;
CapabilityRange
? width;
CapabilityRange
? height;
CapabilityRange
? frameRate;
CapabilityRange
? aspectRatio;
CapabilityList
? facingMode;
};
AllVideoCapabilities
MembersaspectRatio
of type CapabilityRange
, nullablefacingMode
of type CapabilityList
, nullableVideoFacingModeEnum
) on the source.frameRate
of type CapabilityRange
, nullableheight
of type CapabilityRange
, nullablesourceId
of type CapabilityList
, nullablesourceType
of type CapabilityList
, nullableSourceTypeEnum
) on the current source.width
of type CapabilityRange
, nullabledictionary AllAudioCapabilities {
CapabilityRange
? volume;
};
AllAudioCapabilities
Membersvolume
of type CapabilityRange
, nullablepartial interface URL {
static DOMString createObjectURL (MediaStream
stream);
};
createObjectURL
, staticMints a Blob URL to refer to the given
.MediaStream
When the createObjectURL()
method
is called with a
argument, the user
agent MUST return a unique Blob URL for the
given MediaStream
. [FILE-API]MediaStream
For audio and video streams, the data exposed on that stream MUST
be in a format supported by the user agent for use in
audio
and video
elements.
A Blob URL is the
same as what the File API specification calls a Blob URI, except that
anything in the definition of that feature that refers to
File
and Blob
objects is hereby extended to
also apply to
objects.MediaStream
Parameter | Type | Nullable | Optional | Description |
---|---|---|---|---|
stream |
| ✘ | ✘ |
DOMString
A MediaStream
may be assigned to media elements
as defined in HTML5
[HTML5] A MediaStream
is not preloadable or
seekable and represents a simple, potentially infinite, linear
media timeline. The timeline starts at 0 and increments linearly
in real time as long as the MediaStream
is
playing. The timeline does not increment when the
MediaStream
is paused.
There are two methods by which a MediaStream may be assigned
to a media element. First, createObjectURL
can be used to obtain a URL for the MediaStream
.
That URL can then be used to set the src
attribute
of the media element itself, or of one of its <source>
children. Secondly, UAs that support this specification MUST
support the following partial interface, which allows a
MediaStream to be assigned directly to a media element.
partial interface HTMLMediaElement {
attribute MediaStream
? srcObject;
};
srcObject
of type MediaStream
, , nullableHolds the MediaStream that provides media for this
element. This attribute overrides both the src
attribute and any <source> elements. Specifically, if
srcObject
is specified, the UA MUST use it as
the source of media, even if the src
attribute
is also set or <source> children are present. If the
value of srcObject
is replaced or set to null
the UA MUST re-run the
media element load algorithm
We may want to allow direct assignment of other types as well
The UA runs the
media element load algorithm to obtain media for the media
element to display. As defined in the [HTML5]
specification, this algorithm has two basic phases:
resource selection algorithm chooses the resource to play
and resolves its URI. Then the
resource fetch phase loads the resource. Both these phases
are potentially simplified when using a MediaStream. First of
all, in the case of direct assignment, srcObject
takes priority over other means of specifying the resource.
Futhermore, it provides the object itself rather than a URI.
In this case, there is no need to run the resource selection
algorithm. Secondly, when the UA reaches the resource fetch
algorithm with a MediaStream (whether specified by URI or
direct assignment), the MediaStream is a local object so
there's nothing to fetch. Therefore, the following
modifications/restrictions to the
media element load algorithm apply:
Whenever the user agent runs the
media element load algorithm, if srcObject
is
specified, the UA must immediately go to the
resource fetch phase of the algorithm. (Note that in this
case, the UA will have the MediaStream object, and not an
absolute URI, as it would if the MediaStream had been
specified by src
or <source>. However the
load operation on a MediaStream is trivial in all cases, as
indicated in the next item.)
Whenever the user agent runs the
media element load algorithm, reaches the
resource fetch phase of this algorithm, and determines
that the media resource in question is a MediaStream, it MUST
immediately abort the
resource selection algorithm, setting the
media.readyState
to HAVE_NOTHING if media is
not yet available and to HAVE_ENOUGH_DATA once it is.
For each
in the
MediaStreamTrack
, including those that are added after
the UA enters the
media element load algorithm, the UA MUST create a
corresponding MediaStream
AudioTrack
or
VideoTrack
as defined in [HTML5]. Since the order in
the
's track set is undefined, no requirements are put how
the MediaStream
AudioTrackList
and
VideoTrackList
are ordered.
The properties of the AudioTrack
and
VideoTrack
objects MUST be initialized as follows.
Let
AudioTrack.id
and VideoTrack.id
have the value of the corresponding MediaStreamTrack.id
attribute
AudioTrack.kind
and
VideoTrack.kind
be "main"
AudioTrack.label
and
VideoTrack.label
have the value of the corresponding
MediaStreamTrack.label
attribute
AudioTrack.language
and
VideoTrack.language
be the empty string
AudioTrack.enabled
be true
Set the
VideoTrackList.selectedIndex
to the index of the first
VideoTack
, in the VideoTrackList
,
that corresponds to a
that is
not muted or disabled. If
no such MediaStreamTrack
VideoTack
exists, set the
selectedIndex
attribute to 0.
(Note that since the MediaStream is potentially endless, the UA does not exit the media element load algorithm until the MediaStream moves from the active to the inactive state.)
If a
is removed from a
MediaStreamTrack
, played by a media element, the
corresponding MediaStream
AudioTrack
or VideoTrack
MUST
be removed as well.
The UA MUST NOT buffer data from a MediaStream. When playing, the UA MUST always play the current data from the stream.
When the MediaStream is moves from the active to the
inactive state, the UA MUST raise an ended
event on the media element and set its ended
attribute to true
. Note that once ended
equals true
the media element will not play media
even if new Tracks are added to the MediaStream (causing it to
return to the active state) until the JavaScript restarts the
element, e.g., by calling play().
If a MediaStream is inactive and the media
element is ended, and the element's autoplay
attribute is set to true, should it start playing
automatically if new Tracks are added to the MediaStream?
The nature of the MediaStream
places certain restrictions on the
behavior and attribute values of the associated media element and on the
operations that can be performed on it, as shown below:
Attribute Name | Attribute Type | Valid Values When Using a MediaStream | Additional considerations |
---|---|---|---|
src
|
DOMString |
a local URI referencing a MediaStream | The markup author can use this attribute to specify
the source of the media. createObjectURI
can be used to generate a URI referring to a
MediaStream which can be used as the value of this
attribute. In such a case, revocation of the URI does
not count as a change to this field and does not
trigger the media element load algorithm. |
currentSrc
|
DOMString |
a local URI referencing a MediaStream or the empty string | The UA automatically sets the value of this attribute.
When srcObject is specified the UA MUST
set it to the empty string. |
preload
|
DOMString |
none |
A MediaStream cannot be preloaded. |
buffered
|
TimeRanges
|
buffered.length MUST return 0 . |
A MediaStream cannot be preloaded. Therefore, the amount buffered is always an empty TimeRange. |
networkState
|
unsigned short |
NETWORK_IDLE | The media element does not fetch the MediaStream so there is no network traffic. |
readyState
|
unsigned short |
HAVE_NOTHING, HAVE_ENOUGH_DATA | A may be created before there
is any data available, for example when a stream is received from a
remote peer. The value of the readyState of
the media element MUST
be HAVE_NOTHING before the first media arrives
and HAVE_ENOUGH_DATA once the first media has arrived. |
currentTime
|
double |
Any positive integer. The initial value is 0 and the values increments linearly in real time whenever the stream is playing. | The value is the current stream position, in seconds. On any
attempt to set this attribute, the user agent must throw an
InvalidStateError exception. |
duration
|
unrestricted double |
Infinity | A MediaStream does not have a pre-defined duration. |
seeking
|
boolean |
false | A MediaStream is not seekable. Therefore, this attribute
MUST always have the value
false . |
defaultPlaybackRate
|
double |
1.0 | A MediaStream is not seekable. Therefore, this attribute
MUST always have the value
1.0 and any attempt to alter it MUST fail. |
playbackRate
|
double |
1.0 | A MediaStream is not seekable. Therefore, this attribute
MUST always have the value
1.0 and any attempt to alter it MUST fail. |
played
|
TimeRanges
|
played.length MUST return 1 .played.start(0) MUST return 0 .played.end(0) MUST return the last known currentTime .
|
A MediaStream's timeline always consists of a single range, starting at 0 and extending up to the currentTime. |
seekable
|
TimeRanges
|
seekable.length MUST return 0 .seekable.start() MUST return currentTime .seekable.end() MUST return currentTime .
|
A MediaStream is not seekable. |
startDate
|
Date |
Not-a-Number (NaN) | A MediaStream does not specify a timeline offset. |
loop
|
boolean |
true, false | Setting the loop attribute has no effect since a
has no defined end and therefore
cannot be looped. |
This section is non-normative.
The following event fires on
objects:MediaStream
Event name | Interface | Fired when... |
---|---|---|
active
|
Event
|
The became active (see inactive). |
inactive
|
Event
|
The became inactive. |
addtrack
|
|
A new has been added to
this stream. Note that this event is not fired when the script
directly modifies the tracks of a .
|
removetrack
|
|
A has been removed from
this stream. Note that this event is not fired when the script
directly modifies the tracks of a .
|
The following event fires on
objects:MediaStreamTrack
Event name | Interface | Fired when... |
---|---|---|
started
|
Event
|
The
object has just transitioned from the
"new" readyState to another state.
This event fires before any other corresponding events
such as "ended" or "statechanged". |
mute
|
Event
|
The
object's source is temporarily unable to provide
data. |
unmute
|
Event
|
The
object's source is live again after having been
temporarily unable to provide data. |
overconstrained
|
Event
|
This event fires asynchronously for each affected
track (when multiple tracks share the same source) after
the user agent has evaluated the current constraints
against a given Due to being over-constrained, the user agent must mute each affected track. The affected track(s) will remain un-usable (in
the The "overconstrained" event is a simple event of
type |
ended
|
Event
|
The
object's source will no longer provide any data, either
because the user revoked the permissions, or because the source
device has been ejected, or because the remote peer stopped
sending data, or because the
stop()
method was invoked. |
dictionary MediaStreamConstraints {
(boolean or MediaTrackConstraints
) video = false;
(boolean or MediaTrackConstraints
) audio = false;
};
MediaStreamConstraints
Membersaudio
of type (boolean or MediaTrackConstraints), defaulting to false
Provide definition of audio constraints here.
video
of type (boolean or MediaTrackConstraints), defaulting to false
Provide definition of video constraints here.
dictionary MediaTrackConstraints {
MediaTrackConstraintSet? mandatory = null;
MediaTrackConstraint[]? _optional = null;
};
MediaTrackConstraints
Membersoptional
of type array of MediaTrackConstraint, nullable, defaulting to null
Provide definition of optional constraints here.
mandatory
of type MediaTrackConstraintSet, nullable, defaulting to null
Provide definition of mandatory constraints here.
A MediaTrackConstraintSet is a dictionary containing one or more key-value pairs, where each key MUST be a valid registered constraint name in the IANA-hosted RTCWeb Media Constraints registry [RTCWEB-CONSTRAINTS] and its value SHOULD be as defined in the associated reference(s) given in the registry.
A MediaTrackConstraint is a dictionary containing exactly one key-value pair, where the key MUST be a valid registered constraint name in the IANA-hosted RTCWeb Media Constraints registry [RTCWEB-CONSTRAINTS] and the value SHOULD be as defined in the associated reference(s) given in the registry.
Example MediaTrackConstraints value:
{mandatory
: { width: { min: 640 }, height: { min: 480 } },optional
: [ { width: 650 }, { width: { min: 650 }}, { frameRate: 60 }, { width: { max: 800 }}, { facingMode: "user" } ] }
This section is non-normative.
The user agent is encouraged to reserve resources when it has determined that a given call to getUserMedia() will succeed. It is preferable to reserve the resource prior to invoking the success callback provided by the web page. Subsequent calls to getUserMedia() (in this page or any other) should treat the resource that was previously allocated, as well as resources held by other applications, as busy. Resources marked as busy should not be provided as sources to the current web page, unless specified by the user. Optionally, the user agent may choose to provide a stream sourced from a busy source but only to a page whose origin matches the owner of the original stream that is keeping the source busy.
This document recommends that in the permission grant dialog or device selection interace (if one is present), the user be allowed to select any available hardware as a source for the stream requested by the page (provided the resource is able to fulfill mandatory constraints, if any were specified), in addition to the ability to substitute a video or audio source with local files and other media. A file picker may be used to provide this functionality to the user.
This document also recommends that the user be shown all resources that are currently busy as a result of prior calls to getUserMedia() (in this page or any other page that is still alive) and be allowed to terminate that stream and utilize the resource for the current page instead. If possible in the current operating environment, it is also suggested that resources currently held by other applications be presented and treated in the same manner. If the user chooses this option, the track corresponding to the resource that was provided to the page whose stream was affected must be removed.
A MediaStream
may contain more than one
video and audio track. This makes it possible to include video from two
or more webcams in a single stream object, for example. However, the
current API does not allow a page to express a need for multiple video
streams from independent sources.
It is recommended for multiple calls to getUserMedia() from the same page be allowed as a way for pages to request multiple, discrete, video or audio streams.
A single call to getUserMedia() will always return a stream with either zero or one audio tracks, and either zero or one video tracks. If a script calls getUserMedia() multiple times before reaching a stable state, this document advises the UI designer that the permission dialogs should be merged, so that the user can give permission for the use of multiple cameras and/or media sources in one dialog interaction. The constraints on each getUserMedia call can be used to decide which stream gets which media sources.
This sample code exposes a button. When clicked, the button is disabled and the user is prompted to offer a stream. The user can cause the button to be re-enabled by providing a stream (e.g., giving the page access to the local camera) and then disabling the stream (e.g., revoking that access).
<input type="button" value="Start" onclick="start()" id="startBtn"> <script> var startBtn = document.getElementById('startBtn'); function start() { navigator.getUserMedia({audio:true, video:true}, gotStream, logError); startBtn.disabled = true; } function gotStream(stream) { stream.oninactive = function () { startBtn.disabled = false; }; } function logError(error) { log(error.name + ": " + error.message); } </script>
This example allows people to take photos of themselves from the local video camera. Note that the forthcoming Image Capture specification may provide a simpler way to accomplish this.
<article> <style scoped> video { transform: scaleX(-1); } p { text-align: center; } </style> <h1>Snapshot Kiosk</h1> <section id="splash"> <p id="errorMessage">Loading...</p> </section> <section id="app" hidden> <p><video id="monitor" autoplay></video> <canvas id="photo"></canvas> <p><input type=button value="📷" onclick="snapshot()"> </section> <script> navigator.getUserMedia({video:true}, gotStream, noStream); var video = document.getElementById('monitor'); var canvas = document.getElementById('photo'); function gotStream(stream) { video.src = URL.createObjectURL(stream); stream.oninactive = noStream; video.onloadedmetadata = function () { canvas.width = video.videoWidth; canvas.height = video.videoHeight; document.getElementById('splash').hidden = true; document.getElementById('app').hidden = false; }; } function noStream() { document.getElementById('errorMessage').textContent = 'No camera available.'; } function snapshot() { canvas.getContext('2d').drawImage(video, 0, 0); } </script> </article>
IANA is requested to register the following constraints as specified in [RTCWEB-CONSTRAINTS]:
Note: The following specific list(s) of constraints DOES NOT REFLECT CONSENSUS. Many constraints beyond these have been proposed, and the ones listed do not have universal support. The ones below are included **** ONLY **** to provide a starting point so we can see concrete examples of what real constraints might look like. Along those lines, there was an attempt to include constraints of a variety of types so sample type definitions could be included as well (e.g., DOMString, MinMaxConstraint).
The following constraint names are defined to apply to both
and VideoStreamTrack
objects:AudioStreamTrack
Constraint Name | Values | Notes |
---|---|---|
sourceType | SourceTypeEnum |
Constrain the video or audio source to an exact value from the set of enumerated-type values of the SourceTypeEnum . |
sourceId | DOMString | Constrain the video or audio source to an exact source identifier value. |
noaccess | boolean | Prevent the stream from being applied to an element which is readable by the JS. |
peerIdentity | DOMString | Prevent the stream from being applied to an element which is readable by the JS. Require that any PeerConnections to which the stream is added match the provided identity. |
The following constraint names are defined to apply only to
objects:VideoStreamTrack
Constraint Name | Values | Notes |
---|---|---|
width | unsigned long or
MinMaxConstraint
|
Constrain the video source to the exact desired width or width range. |
height | unsigned long or MinMaxConstraint |
Constrain the video source to the exact desired height or height range. |
frameRate | float or MinMaxConstraint |
Constrain the video source to the exact desired frame rate (fps) or frameRate range. |
aspectRatio | float or MinMaxConstraint |
Constrain the video source to the exact desired aspect ratio represented as floating point number rounded to 10 decimal places, representing the width in pixels divided by the height in pixels. |
facingMode |
|
Constrain the video source to an exact value from the set of enumerated-type values of the . |
The following constraint names are defined to apply only to
objects:AudioStreamTrack
Constraint Name | Values | Notes |
---|---|---|
volume | unsigned long or MinMaxConstraint |
Constrain the audio source to the exact desired volume or volume range. |
For constraints that accept ranges,
the
dictionary is also
defined. Note that the type of the value associated
with MinMaxConstraint
min
and max
must be the same
for both. The specific types associated with min
and max
are defined differently for each
constraint name.
dictionary MinMaxConstraint {
any max;
any min;
};
MinMaxConstraint
Membersmax
of type anymin
of type anyThis section will be removed before publication.