This document defines a set of ECMAScript APIs in WebIDL to allow media to be sent to and received from another browser or device implementing the appropriate set of real-time protocols. This specification is being developed in conjunction with a protocol specification developed by the IETF RTCWEB group and an API specification to get access to local media devices developed by the Media Capture Task Force.

This document is neither complete nor stable, and as such is not yet suitable for commercial implementation. However, early experimentation is encouraged. The API is based on preliminary work done in the WHATWG. The Web Real-Time Communications Working Group expects this specification to evolve significantly based on:

Introduction

There are a number of facets to video-conferencing in HTML covered by this specification:

This document defines the APIs used for these features. This specification is being developed in conjunction with a protocol specification developed by the IETF RTCWEB group and an API specification to get access to local media devices [[!GETUSERMEDIA]]developed by the Media Capture Task Force. An overview of the system can be found in [[RTCWEB-OVERVIEW]] and [[RTCWEB-SECURITY]].

This specification defines conformance criteria that apply to a single product: the user agent that implements the interfaces that it contains.

Implementations that use ECMAScript to implement the APIs defined in this specification must implement them in a manner consistent with the ECMAScript Bindings defined in the Web IDL specification [[!WEBIDL]], as this specification uses that specification and terminology.

Terminology

The EventHandler interface represents a callback used for event handlers as defined in [[!HTML5]].

The concepts queue a task and fires a simple event are defined in [[!HTML5]].

The terms event, event handlers and event handler event types are defined in [[!HTML5]].

The terms MediaStream, MediaStreamTrack, Constraints, and Consumer are defined in [[!GETUSERMEDIA]].

Peer-to-peer connections

Introduction

An RTCPeerConnection allows two users to communicate directly, browser to browser. Communications are coordinated via a signaling channel which is provided by unspecified means, but generally by a script in the page via the server, e.g. using XMLHttpRequest.

Configuration

RTCConfiguration Type

sequence<RTCIceServer> iceServers

An array containing URIs of servers available to be used by ICE, such as STUN and TURN server.

RTCIceTransports iceTransports = "all"

Indicates which candidates the ICE engine is allowed to use.

RTCIdentityOption requestIdentity = "ifconfigured"

See the requestIdentity member of the RTCOfferAnswerOptions dictionary.

RTCIceServer Type

(DOMString or sequence<DOMString> urls

STUN or TURN URI(s) as defined in [[!RFC7064]] and [[!RFC7065]] or other URI types.

DOMString username

If this RTCIceServer object represents a TURN server, then this attribute specifies the username to use with that TURN server.

DOMString credential

If this RTCIceServer object represents a TURN server, then this attribute specifies the credential to use with that TURN server.

In network topologies with multiple layers of NATs, it is desirable to have a STUN server between every layer of NATs in addition to the TURN servers to minimize the peer to peer network latency.

An example array of RTCIceServer objects is:

[ { "urls": "stun:stun1.example.net" }, { "urls": "turn:turn.example.org", "username": "user", "credential": "myPassword" } ]

RTCIceTransports Enum

none
The ICE engine MUST not send or receive any packets at this point.
relay
The ICE engine MUST only use media relay candidates such as candidates passing through a TURN server. This can be used to reduce leakage of IP addresses in certain use cases.
all
The ICE engine may use any type of candidates when this value is specified.

Offer/Answer Options

These dictionaries describe the options that can be used to control the offer/answer creation process.

RTCIdentityOption requestIdentity = "ifconfigured"

The requestIdentity option indicates whether an identity should be requested. The option may be used with either of the createOffer() or createAnswer() calls, but also with the RTCPeerConnection constructor. Note that as long as DTLS-SRTP is in used, fingerprints will be sent regardless of the value of this option.

long offerToReceiveVideo

In some cases, an RTCPeerConnection may wish to receive video but not send any video. The RTCPeerConnection needs to know if it should signal to the remote side whether it wishes to receive video or not. This option allows an application to indicate its preferences for the number of video streams to receive when creating an offer.

long offerToReceiveAudio

In some cases, an RTCPeerConnection may wish to receive audio but not send any audio. The RTCPeerConnection needs to know if it should signal to the remote side whether it wishes to receive audio. This option allows an application to indicate its preferences for the number of audio streams to receive when creating an offer.

boolean voiceActivityDetection = true

Many codecs and system are capable of detecting "silence" and changing their behavior in this case by doing things such as not transmitting any media. In many cases, such as when dealing with sounds other than spoken voice or emergency calling, it is desirable to be able to turn off this behavior. This option allows the application to provide information about whether it wishes this type of processing enabled or disabled.

boolean iceRestart = false

When the value of this dictionary member is true, the generated description will have ICE credentials that are different from the current credentials (as visible in the localDescription attribute's SDP). Applying the generated description will restart ICE.

When the value of this dictionary member is false, and the localDescription attribute has valid ICE credentials, the generated description will have the same ICE credentials as the current value from the localDescription attribute.

yes
An identity MUST be requested.
no
No identity is to be requested.
ifconfigured
The value "ifconfigured" means that an identity will be requested if either the user has configured an identity in the browser or if the setIdentityProvider() call has been made in JavaScript. As this is the default value, an identity will be requested if and only if the user has configured an IdP in some way.

RTCPeerConnection Interface

The general operation of the RTCPeerConnection is described in [[!RTCWEB-JSEP]].

Operation

Calling new RTCPeerConnection(configuration ) creates an RTCPeerConnection object.

The configuration has the information to find and access the servers used by ICE. There may be multiple servers of each type and any TURN server also acts as a STUN server.

An RTCPeerConnection object has an associated ICE agent [[!ICE]], RTCPeerConnection signaling state, ICE gathering state, and ICE connection state. These are initialized when the object is created.

An RTCPeerConnection object has two associated stream sets. A local streams set, representing streams that are currently sent, and a remote streams set, representing streams that are currently received with this RTCPeerConnection object. The stream sets are initialized to empty sets when the RTCPeerConnection object is created.

When the RTCPeerConnection() constructor is invoked, the user agent MUST run the following steps:

  1. Validate the RTCConfiguration argument by running the steps defined by the updateIce() method.

  2. Let connection be a newly created RTCPeerConnection object.

  3. Create an ICE Agent as defined in [[!ICE]] and let connection's RTCPeerConnection ICE Agent be that ICE Agent and provide it the the ICE servers list. The ICE Agent will proceed with gathering as soon as the ICE transports setting is not set to none. At this point the ICE Agent does not know how many ICE components it needs (and hence the number of candidates to gather), but it can make a reasonable assumption such as 2. As the RTCPeerConnection object gets more information, the ICE Agent can adjust the number of components.

  4. Set connection's RTCPeerConnection signalingState to stable.

  5. Set connection's RTCPeerConnection ice connection state to new.

  6. Set connection's RTCPeerConnection ice gathering state to new.

  7. Initialize an internal variable to represent a queue of operations with an empty set.

  8. Return connection.

Once the RTCPeerConnection object has been initialized, for every call to createOffer, setLocalDescription, createAnswer and setRemoteDescription; execute the following steps:

  1. Append an object representing the current call being handled (i.e. function name and corresponding arguments) to the operations array.

  2. If the length of the operations array is exactly 1, execute the function from the front of the queue asynchronously.

  3. When the asynchronous operation completes (either successfully or with an error), remove the corresponding object from the operations array. After removal, if the array is non-empty, execute the first object queued asynchronously and repeat this step on completion.

The general idea is to have only one among createOffer, setLocalDescription, createAnswer and setRemoteDescription executing at any given time. If subsequent calls are made while one of them is still executing, they are added to a queue and processed when the previous operation is fully completed. It is valid, and expected, for normal error handling procedures to be applied.

Additionally, during the lifetime of the RTCPeerConnection object, the following procedures are followed when an ICE event occurs:

  1. If the RTCPeerConnection ice gathering state is new and the ICE transports setting is not set to none, the user agent MUST queue a task to start gathering ICE addresses and set the ice gathering state to gathering.

  2. If the ICE Agent has found one or more candidate pairs for each MediaStreamTrack that forms a valid connection, the ICE connection state is changed to "connected".

  3. When the ICE Agent finishes checking all candidate pairs, if at least one connection has been found for each MediaStreamTrack, the iceConnectionState is changed to "completed"; else the iceConnectionState is changed to "failed".

When the ICE Agent needs to notify the script about the candidate gathering progress, the user agent must queue a task to run the following steps:

  1. Let connection be the RTCPeerConnection object associated with this ICE Agent.

  2. If connection's RTCPeerConnection signalingState is closed, abort these steps.

  3. If the intent of the ICE Agent is to notify the script that:

    • A new candidate is available.

      Add the candidate to connection's localDescription and create a RTCIceCandidate object to represent the candidate. Let newCandidate be that object.

    • The gathering process is done.

      Set connection's ice gathering state to completed and let newCandidate be null.

  4. Fire a icecandidate event named icecandidate with newCandidate at connection.

User agents negotiate the codec resolution, bitrate, and other media parameters. It is RECOMMENDED that user agents initially negotiate for the maximum resolution of a video stream. For streams that are then rendered (using a video element), it is RECOMMENDED that user agents renegotiate for a resolution that matches the rendered display size.

The word "components" in this context refers to an RTP media flow and does not have anything to do with how [[ICE]] uses the term "component".

When a user agent has reached the point where a MediaStream can be created to represent incoming components, the user agent MUST run the following steps:

  1. Let connection be the RTCPeerConnection expecting this media.

  2. Create a MediaStream object stream, to represent the incoming media stream.

  3. Run the algorithm to represent an incoming component with a track for each incoming component.

    The creation of new incoming MediaStreams may be triggered either by SDP negotiation or by the receipt of media on a given flow.

  4. Queue a task to run the following substeps:

    1. If the connection's RTCPeerConnection signalingState is closed, abort these steps.

    2. Add stream to connection's remote streams set.

    3. Fire a stream event named addstream with stream at the connection object.

When a user agent has negotiated media for a component that belongs to a media stream that is already represented by an existing MediaStream object, the user agent MUST associate the component with that MediaStream object.

When an RTCPeerConnection finds that a stream from the remote peer has been removed, the user agent MUST follow these steps:

  1. Let connection be the RTCPeerConnection associated with the stream being removed.

  2. Let stream be the MediaStream object that represents the media stream being removed, if any. If there isn't one, then abort these steps.

  3. By definition, stream is now ended.

    A task is thus queued to update stream and fire an event.

  4. Queue a task to run the following substeps:

    1. If the connection's RTCPeerConnection signalingState is closed, abort these steps.

    2. Remove stream from connection's remote streams set.

    3. Fire a stream event named removestream with stream at the connection object.

The task source for the tasks listed in this section is the networking task source.

If something in the browser changes that causes the RTCPeerConnection object to need to initiate a new session description negotiation, a negotiationneeded event is fired at the RTCPeerConnection object.

In particular, if an RTCPeerConnection object is consuming a MediaStream on which a track is added, by, e.g., the addTrack() method being invoked, the RTCPeerConnection object MUST fire the "negotiationneeded" event. Removal of media components must also trigger "negotiationneeded".

To prevent network sniffing from allowing a fourth party to establish a connection to a peer using the information sent out-of-band to the other peer and thus spoofing the client, the configuration information SHOULD always be transmitted using an encrypted connection.

Interface Definition

Constructor (RTCConfiguration configuration)
See the RTCPeerConnection constructor algorithm.
void createOffer (RTCSessionDescriptionCallback successCallback, RTCPeerConnectionErrorCallback failureCallback, optional RTCOfferOptions options)

The createOffer method generates a blob of SDP that contains an RFC 3264 offer with the supported configurations for the session, including descriptions of the local MediaStreams attached to this RTCPeerConnection, the codec/RTP/RTCP options supported by this implementation, and any candidates that have been gathered by the ICE Agent. The options parameter may be supplied to provide additional control over the offer generated.

As an offer, the generated SDP will contain the full set of capabilities supported by the session (as opposed to an answer, which will include only a specific negotiated subset to use); for each SDP line, the generation of the SDP must follow the appropriate process for generating an offer. In the event createOffer is called after the session is established, createOffer will generate an offer that is compatible with the current session, incorporating any changes that have been made to the session since the last complete offer-answer exchange, such as addition or removal of streams. If no changes have been made, the offer will include the capabilities of the current local description as well as any additional capabilities that could be negotiated in an updated offer.

Session descriptions generated by createOffer MUST be immediately usable by setLocalDescription without causing an error as long as setLocalDescription is called within the successCallback function. If a system has limited resources (e.g. a finite number of decoders), createOffer needs to return an offer that reflects the current state of the system, so that setLocalDescription will succeed when it attempts to acquire those resources. The session descriptions MUST remain usable by setLocalDescription without causing an error until at least end of the successCallback function. Calling this method is needed to get the ICE user name fragment and password.

If the RTCPeerConnection is configured to generate Identity assertions, then the session description SHALL contain an appropriate assertion.

If this RTCPeerConnection object is closed before the SDP generation process completes, the USER agent MUST suppress the result and not call any of the result callbacks.

If the SDP generation process completed successfully, the user agent MUST queue a task to invoke successCallback with a newly created RTCSessionDescription object, representing the generated offer, as its argument.

If the SDP generation process failed for any reason, the user agent MUST queue a task to invoke failureCallback with an DOMError object of type TBD as its argument.

To Do: Discuss privacy aspects of this from a fingerprinting point of view - it's probably around as bad as access to a canvas :-)

void createAnswer (RTCSessionDescriptionCallback successCallback, RTCPeerConnectionErrorCallback failureCallback, optional RTCOfferAnswerOptions options)

The createAnswer method generates an [[!SDP]] answer with the supported configuration for the session that is compatible with the parameters in the remote configuration. Like createOffer, the returned blob contains descriptions of the local MediaStreams attached to this RTCPeerConnection, the codec/RTP/RTCP options negotiated for this session, and any candidates that have been gathered by the ICE Agent. The options parameter may be supplied to provide additional control over the generated answer.

As an answer, the generated SDP will contain a specific configuration that, along with the corresponding offer, specifies how the media plane should be established. The generation of the SDP must follow the appropriate process for generating an answer.

Session descriptions generated by createAnswer must be immediately usable by setLocalDescription without generating an error if setLocalDescription is called from the successCallback function. Like createOffer, the returned description should reflect the current state of the system. The session descriptions MUST remain usable by setLocalDescription without causing an error until at least the end of the successCallback function. Calling this method is needed to get the ICE user name fragment and password.

An answer can be marked as provisional, as described in [[!RTCWEB-JSEP]], by setting the type to "pranswer".

If the RTCPeerConnection is configured to generate Identity assertions, then the session description SHALL contain an appropriate assertion.

If this RTCPeerConnection object is closed before the SDP generation process completes, the USER agent MUST suppress the result and not call any of the result callbacks.

If the SDP generation process completed successfully, the user agent MUST queue a task to invoke successCallback with a newly created RTCSessionDescription object, representing the generated answer, as its argument.

If the SDP generation process failed for any reason, the user agent MUST queue a task to invoke failureCallback with an DOMError object of type TBD as its argument.

void setLocalDescription (RTCSessionDescription description, VoidFunction successCallback, RTCPeerConnectionErrorCallback failureCallback)

The setLocalDescription() method instructs the RTCPeerConnection to apply the supplied RTCSessionDescription as the local description.

This API changes the local media state. In order to successfully handle scenarios where the application wants to offer to change from one media format to a different, incompatible format, the RTCPeerConnection must be able to simultaneously support use of both the old and new local descriptions (e.g. support codecs that exist in both descriptions) until a final answer is received, at which point the RTCPeerConnection can fully adopt the new local description, or rollback to the old description if the remote side denied the change.

ISSUE: how to indicate to rollback?

To Do: specify what parts of the SDP can be changed between the createOffer and setLocalDescription

When the method is invoked, the user agent must follow the processing model described by the following list:

  • If this RTCPeerConnection object's signaling state is closed, the user agent MUST throw an InvalidStateError exception and abort this operation.

  • If a local description contains a different set of ICE credentials, then the ICE Agent MUST trigger an ICE restart. When ICE restarts, the gathering state will be changed back to "gathering", if it was not already gathering. If the IceConnectionState was "completed", it will be changed back to "connected".

  • If the process to apply the RTCSessionDescription argument fails for any reason, then user agent must queue a task runs the following steps:

    1. Let connection be the RTCPeerConnection object on with this method was invoked.

    2. If connection's signaling state is closed, then abort these steps.

    3. If the reason for the failure is:

      • The content of the RTCSessionDescription argument is invalid or the type is wrong for the current signaling state of connection.

        Let errorType be InvalidSessionDescriptionError.

      • The RTCSessionDescription is a valid description but cannot be applied at the media layer.

        TODO ISSUE - next few points are probably wrong. Make sure to check this in setRemote too.

        This can happen, e.g., if there are insufficient resources to apply the SDP. The user agent MUST then rollback as necessary if the new description was partially applied when the failure occurred.

        If rollback was not necessary or was completed successfully, let errorType be IncompatibleSessionDescriptionError. If rollback was not possible, let errorType be InternalError and set connection's signaling state to closed.

    4. Invoke the failureCallback with an DOMError object, whose name attribute is errorType, as its argument.

  • If the RTCSessionDescription argument is applied successfully, then user agent must queue a task runs the following steps:

    1. Let connection be the RTCPeerConnection object on with this metod was invoked.

    2. If connection's signaling state is closed, then abort these steps.

    3. Set connection's description attribute (localDescription or remoteDescription depending on the setting operation) to the RTCSessionDescription argument.

    4. If the local description was set, connection's ice gathering state is new, and the local description contains media, then set connection's ice gathering state to gathering.

    5. If the local description was set with content that caused an ICE restart, then set connection's ice gathering state to gathering.

    6. Set connection's signalingState accordingly.

    7. Fire a simple event named signalingstatechange at connection.

    8. Queue a new task that, if connection's signalingState is not closed, invokes the successCallback.

readonly attribute RTCSessionDescription? localDescription

The localDescription attribute MUST return the RTCSessionDescription that was most recently passed to setLocalDescription(), plus any local candidates that have been generated by the ICE Agent since then.

A null object will be returned if the local description has not yet been set.

void setRemoteDescription (RTCSessionDescription description, VoidFunction successCallback, RTCPeerConnectionErrorCallback failureCallback)

The setRemoteDescription() method instructs the RTCPeerConnection to apply the supplied RTCSessionDescription as the remote offer or answer. This API changes the local media state.

When the method is invoked, the user agent must follow the processing model of setLocalDescription(), with the following additional conditions:

  • If an a=identity attribute is present in the session description, the browser validates the identity assertion.. Identity validation completes asynchronously and does not block the completion of setRemoteDescription, unless there is a target peer identity.

  • If the "peerIdentity" constraint is applied to the RTCPeerConnection, this establishes a target peer identity. Alternatively, if the RTCPeerConnection has previously authenticated the identity of the peer (that is, there is a current value for peerIdentity), then this also establishes a target peer identity.

    If there is a target peer identity, then setRemoteDescription fails unless it contains an identity assertion that matches the target peer identity. The RTCPeerConnection MAY be closed if the validated peer identity does not match the target peer identity. [[TODO: determine if it is possible at this point to back out the change. Seems unlikely.]]

readonly attribute RTCSessionDescription? remoteDescription

The remoteDescription attribute MUST return the RTCSessionDescription that was most recently passed to setRemoteDescription(), plus any remote candidates that have been supplied via addIceCandidate() since then.

A null object will be returned if the remote description has not yet been set.

readonly attribute RTCSignalingState signalingState

The signalingState attribute MUST return the RTCPeerConnection object's RTCPeerConnection signaling state.

void updateIce (RTCConfiguration configuration)

The updateIce method updates the ICE Agent process of gathering local candidates and pinging remote candidates.

This call may result in a change to the state of the ICE Agent, and may result in a change to media state if it results in connectivity being established.

When the updateIce() method is invoked, the user MUST run the following steps to process the RTCConfiguration dictionary:

  1. If the iceTransports member is present, let its value be the ICE Agent's ICE transports setting.

  2. If the iceTransports member was omitted and the ICE Agent's ICE transports setting is unset, set the ICE Agent's ICE transports setting to the iceTransports dictionary member default value.

  3. If the iceServers dictionary member is present, but its value is an empty list, then throw an InvalidAccessError and abort these steps. If the list, on the other hand, has elements, each element must be validated by running the following sub-steps:

    1. Let server be the current list element.

    2. If the server.urls dictionary member is omitted or an empty list, then throw an InvalidAccessError and abort these steps.

    3. If server.urls is a string, let urls be a list consisting of just that string. Otherwise, let urls refer to the server.urls list.

    4. For each url in urls, parse the url and obtain scheme name. If the parsing fails or if scheme name is not implemented by the browser, throw a SyntaxError and abort these steps.

    5. If scheme name is "turn" and either of the dictionary members server.username or server.credential are omitted, then throw an InvalidAccessError and abort these steps.

    6. After passing the validation, let the iceServers dictionary member be the ICE Agent's ICE servers list.

      If a new list of servers replaces the ICE Agent's existing ICE servers list, no action will taken until the RTCPeerConnection's ice gathering state transitions to gathering. If a script wants this to happen immediately, it should do an ICE restart.

    7. If the iceServers dictionary member was omitted, and the ICE Agent's ICE servers list is unset, throw an InvalidAccessError and abort these steps.

    The exception types throw in the above algorithm are provisional (until we decide what to do in each case).
void addIceCandidate (RTCIceCandidate candidate, VoidFunction successCallback, RTCPeerConnectionErrorCallback failureCallback)

The addIceCandidate() method provides a remote candidate to the ICE Agent. In addition to being added to the remote description, connectivity checks will be sent to the new candidates as long as the ICE Transports setting is not set to none. This call will result in a change to the connection state of the ICE Agent, and may result in a change to media state if it results in different connectivity being established.

If the candidate parameter is malformed, throw a SyntaxError exception and abort these steps.

If the candidate is successfully applied, the user agent MUST queue a task to invoke successCallback.

If the candidate could not be successfully applied, the user agent MUST queue a task to invoke failureCallback with a DOMError object whose name attribute has the value TBD (TODO InvalidCandidate and InvalidMidIndex).

What errors do we need here? Should we reuse the *SessionDescriptionError names or invent new ones for candidates? Should this method be queued?
readonly attribute RTCIceGatheringState iceGatheringState

The iceGatheringState attribute MUST return the gathering state of the RTCPeerConnection ICE Agent connection state.

readonly attribute RTCIceConnectionState iceConnectionState

The iceConnectionState attribute MUST return the state of the RTCPeerConnection ICE Agent ICE state.

RTCConfiguration getConfiguration()

Returns a RTCConfiguration object representing the current configuration of this RTCPeerConnection object.

When this method is call, the user agent MUST construct new RTCConfiguration object to be returned, and initialize it using the ICE Agent's ICE transports setting and ICE servers list.

sequence<MediaStream> getLocalStreams()

Returns a sequence of MediaStream objects representing the streams that are currently sent with this RTCPeerConnection object.

The getLocalStreams() method MUST return a new sequence that represents a snapshot of all the MediaStream objects in this RTCPeerConnection object’s local streams set. The conversion from the streams set to the sequence, to be returned, is user agent defined and the order does not have to stable between calls.

sequence<MediaStream> getRemoteStreams()

Returns a sequence of MediaStream objects representing the streams that are currently received with this RTCPeerConnection object.

The getRemoteStreams() method MUST return a new sequence that represents a snapshot of all the MediaStream objects in this RTCPeerConnection object’s remote streams set. The conversion from the streams set to the sequence, to be returned, is user agent defined and the order does not have to stable between calls.

MediaStream? getStreamById(DOMString streamId)

If a MediaStream object, with an id equal to streamId, exists in this RTCPeerConnection object’s stream sets (local streams set or remote streams set), then the getStreamById() method MUST return that MediaStream object. The method MUST return null if no stream matches the streamId argument.

For this method to make sense, we need to make sure that ids are unique within the two stream sets of a RTCPeerConnection. This is not the case today when a peer re-adds a stream that is received. Two different stream instances will now have the same id at both peers; one in the remote stream set and one in the local stream set.

One way to resolve this is to not allow re-adding a stream instance that is received (guard on id). If an application really needs this functionality it's really easy to make a clone of the stream, which will give it a new id, and send the clone.

void addStream (MediaStream stream)

Adds a new stream to the RTCPeerConnection.

When the addStream() method is invoked, the user agent MUST run the following steps:

  1. Let connection be the RTCPeerConnection object on which the MediaStream, stream, is to be added.

  2. If connection's RTCPeerConnection signalingState is closed, throw an InvalidStateError exception and abort these steps.

  3. If stream is already in connection's local streams set, then abort these steps.

  4. Add stream to connection's local streams set.

  5. A stream could have contents that are inaccessible to the application. This can be due to being marked with a peerIdentity option or anything that would make a stream CORS cross-origin. These streams can be added to the local streams set but content MUST NOT be transmitted, with one exception.

    A stream marked with the peerIdentity option can be transmitted if the RTCPeerConnection has successfully validated the identity of the peer to have the same identity as the value of the peerIdentity option on the stream.

    All other streams that are not accessible to the application MUST NOT be sent to the peer, with silence (audio), black frames (video) or equivalently absent content being sent in place of stream content.

    Note that this property can change over time.

  6. If connection's RTCPeerConnection signalingState is stable, then fire a negotiationneeded event at connection.

void removeStream (MediaStream stream)

Removes the given stream from the RTCPeerConnection.

When the other peer stops sending a stream in this manner, a removestream event is fired at the RTCPeerConnection object.

When the removeStream() method is invoked, the user agent MUST run the following steps:

  1. Let connection be the RTCPeerConnection object on which the MediaStream, stream, is to be removed.

  2. If connection's RTCPeerConnection signalingState is closed, throw an InvalidStateError exception.

  3. If stream is not in connection's local streams set, then abort these steps.

  4. Remove stream from connection's local streams set.

  5. If connection's RTCPeerConnection signalingState is stable, then fire a negotiationneeded event at connection.

void close ()

When the RTCPeerConnection close() method is invoked, the user agent MUST run the following steps:

  1. If the RTCPeerConnection object's RTCPeerConnection signalingState is closed, abort these steps.
  2. Destroy the RTCPeerConnection ICE Agent, abruptly ending any active ICE processing and any active streaming, and releasing any relevant resources (e.g. TURN permissions).

  3. Set the object's RTCPeerConnection signalingState to closed.

attribute EventHandler onnegotiationneeded
This event handler, of event handler event type negotiationneeded , MUST be supported by all objects implementing the RTCPeerConnection interface.
attribute EventHandler onicecandidate
This event handler, of event handler event type icecandidate, MUST be supported by all objects implementing the RTCPeerConnection interface.
attribute EventHandler onsignalingstatechange
This event handler, of event handler event type signalingstatechange, MUST be supported by all objects implementing the RTCPeerConnection interface. It is called any time the readyState changes, i.e., from a call to setLocalDescription, a call to setRemoteDescription, or code. It does not fire for the initial state change into new.
attribute EventHandler onaddstream
This event handler, of event handler event type addstream, MUST be fired by all objects implementing the RTCPeerConnection interface. It is called any time a MediaStream is added by the remote peer. This will be fired only as a result of setRemoteDescription. Onnaddstream happens as early as possible after the setRemoteDescription. This callback does not wait for a given media stream to be accepted or rejected via SDP negotiation.
attribute EventHandler onremovestream
This event handler, of event handler event type removestream, MUST be fired by all objects implementing the RTCPeerConnection interface. It is called any time a MediaStream is removed by the remote peer. This will be fired only as a result of setRemoteDescription.
attribute EventHandler oniceconnectionstatechange
This event handler, of event handler event type iceconnectionstatechange, MUST be fired by all objects implementing the RTCPeerConnection interface. It is called any time the iceConnectionState changes.

Garbage collection

A Window object has a strong reference to any RTCPeerConnection objects created from the constructor whose global object is that Window object.

State Definitions

RTCPeerState Enum

stable
There is no offer­answer exchange in progress. This is also the initial state in which case the local and remote descriptions are empty.
have-local-offer
A local description, of type "offer", has been successfully applied.
have-remote-offer
A remote description, of type "offer", has been successfully applied.
have-local-pranswer
A remote description of type "offer" has been successfully applied and a local description of type "pranswer" has been successfully applied.
have-remote-pranswer
A local description of type "offer" has been successfully applied and a remote description of type "pranswer" has been successfully applied.
closed
The connection is closed.

The non-normative peer state transitions are: The non-normative peer state transition diagram

An example set of transitions might be:

Caller transition:

  • new RTCPeerConnection(): stable
  • setLocal(offer): have-local-offer
  • setRemote(pranswer): have-remote-pranswer
  • setRemote(answer): stable
  • close(): closed

Callee transition:

  • new RTCPeerConnection(): stable
  • setRemote(offer): have-remote-offer
  • setLocal(pranswer): have-local-pranswer
  • setLocal(answer): stable
  • close(): closed

RTCIceGatheringState Enum

new
The object was just created, and no networking has occurred yet.
gathering
The ICE engine is in the process of gathering candidates for this RTCPeerConnection.
complete
The ICE engine has completed gathering. Events such as adding a new interface or a new TURN server will cause the state to go back to gathering.

RTCIceConnectionState Enum

new
The ICE Agent is gathering addresses and/or waiting for remote candidates to be supplied.
checking
The ICE Agent has received remote candidates on at least one component, and is checking candidate pairs but has not yet found a connection. In addition to checking, it may also still be gathering.
connected
The ICE Agent has found a usable connection for all components but is still checking other candidate pairs to see if there is a better connection. It may also still be gathering.
completed
The ICE Agent has finished gathering and checking and found a connection for all components. Open issue: it is not clear how the non controlling ICE side knows it is in the state.
failed
The ICE Agent is finished checking all candidate pairs and failed to find a connection for at least one component. Connections may have been found for some components.
disconnected
Liveness checks have failed for one or more components. This is more aggressive than failed, and may trigger intermittently (and resolve itself without action) on a flaky network.
closed
The ICE Agent has shut down and is no longer responding to STUN requests.

States take either the value of any component or all components, as outlined below:

  • checking occurs if ANY component has received a candidate and can start checking
  • connected occurs if ALL components have established a working connection
  • completed occurs if ALL components have finalized the running of their ICE processes
  • failed occurs if ANY component has given up trying to connect
  • disconnected occurs if ANY component has failed liveness checks
  • closed occurs only if RTCPeerConnection.close() has been called.

If a component is discarded as a result of signaling (e.g. RTCP mux or BUNDLE), the state may advance directly from checking to completed.

An example transition might look like:

  • new RTCPeerConnection(): new
  • (new, remote candidates received): checking
  • (checking, found usable connection): connected
  • (checking, gave up): failed
  • (connected, finished all checks): completed
  • (completed, lost connectivity): disconnected
  • (any state, ICE restart occurs): new
  • close(): closed

The non-normative ICE state transitions are: The non-normative ICE state transition diagram

Callback Definitions

RTCPeerConnectionErrorCallback

DOMError error
An error object encapsulating information about what went wrong.

Error Handling

General Principles

Errors are indicated in two ways: exceptions and objects passed to error callbacks. Exceptions are thrown to indicate invalid state and other programming errors. For example when a method is called when the RTCPeerConnection is in an invalid state, or a state in which that particular method is not allowed to be executed. In all other cases, an error object MUST be provided to the error callback.

RTCSdpError

readonly attribute long sdpLineNumber
The line number of an RTCSessionDescription at which the error was encountered.

Ask the DOM team to extend their list with the following errors. The error names and their descriptions are directly copied from the old RTCErrorName enum and might need some adjustment before being added to the public list of errors.

  • InvalidSessionDescriptionError: The provided RTCSessionDescription contained invalid SDP, or the type was wrong for the current state of the RTCPeerConnection. User agents SHOULD provide as much additional information in the error message as possible, including the sdpLineNumber, if appropriate.
  • IncompatibleSessionDescriptionError: The provided RTCSessionDescription contained SDP that could not be correctly applied to the RTCPeerConnection due to its current state. User agents SHOULD provide as much additional information in the error message as possible, including the sdpLineNumber, if appropriate.
  • IncompatibleConstraintsError: The provided MediaConstraints could not be correctly applied to the RTCPeerConnection due to its current state. User agents SHOULD provide as much additional information in the error message as possible.
  • IncompatibleMediaStreamTrackError: The provided MediaStreamTrack is not an element of a MediaStream that is currently in the RTCPeerConnection's localStreams attribute.
  • InternalError: The RTCPeerConnection encountered an error that it could not recover from.

Session Description Model

RTCSdpType

The RTCSdpType enum describes the type of an RTCSessionDescription instance.

offer

An RTCSdpType of "offer" indicates that a description should be treated as an [[!SDP]] offer.

pranswer

An RTCSdpType of "pranswer" indicates that a description should be treated as an [[!SDP]] answer, but not a final answer. A description used as an SDP "pranswer" may be applied as a response to a SDP offer, or an update to a previously sent SDP "pranswer".

answer

An RTCSdpType of "answer" indicates that a description should be treated as an [[!SDP]] final answer, and the offer-answer exchange should be considered complete. A description used as an SDP answer may be applied as a response to an SDP offer or as an update to a previously sent SDP "pranswer".

RTCSessionDescription Class

Constructor (optional RTCSessionDescriptionInit descriptionInitDict)
The RTCSessionDescription() constructor takes an optional dictionary argument, descriptionInitDict, whose content is used to initialize the new RTCSessionDescription object. If a dictionary key is not present in descriptionInitDict, the corresponding attribute will be initialized to null. If the constructor is run without the dictionary argument, all attributes will be initialized to null. This class is a future extensible carrier for the data contained in it and does not perform any substantive processing.
attribute RTCSdpType? type
The type of SDP this RTCSessionDescription represents.
attribute DOMString? sdp
The string representation of the SDP [[!SDP]]
serializer = { attribute }
RTCSdpType type
DOMString sdp

RTCSessionDescriptionCallback

RTCSessionDescription sdp
The object containing the SDP [[!SDP]].

Interfaces for Connectivity Establishment

RTCIceCandidate Type

This class is a future extensible carrier for the data contained in it and does not perform any substantive processing.

Constructor (optional RTCIceCandidateInit candidateInitDict)
The RTCIceCandidate() constructor takes an optional dictionary argument, candidateInitDict, whose content is used to initialize the new RTCIceCandidate object. If a dictionary key is not present in candidateInitDict, the corresponding attribute will be initialized to null. If the constructor is run without the dictionary argument, all attributes will be initialized to null.
attribute DOMString? candidate
This carries the candidate-attribute as defined in section 15.1 of [[!ICE]].
attribute DOMString? sdpMid
If present, this contains the identifier of the "media stream identification" as defined in [[!RFC3388]] for the m-line this candidate is associated with.
attribute unsigned short? sdpMLineIndex
This indicates the index (starting at zero) of the m-line in the SDP this candidate is associated with.
serializer = { attribute}
DOMString candidate
DOMString sdpMid
unsigned short sdpMLineIndex

RTCPeerConnectionIceEvent

The icecandidate event of the RTCPeerConnection uses the RTCPeerConnectionIceEvent interface.

Firing an RTCPeerConnectionIceEvent event named e with an RTCIceCandidate candidate means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the RTCPeerConnectionIceEvent interface with the candidate attribute set to the new ICE candidate, MUST be created and dispatched at the given target.

Constructor(DOMString type, RTCPeerConnectionIceEventInit eventInitDict)
readonly attribute RTCIceCandidate candidate

The candidate attribute is the RTCIceCandidate object with the new ICE candidate that caused the event.

RTCIceCandidate candidate

TODO

Peer-to-peer Data API

The Peer-to-peer Data API lets a web application send and receive generic application data peer-to-peer. The API for sending and receiving data models the behavior of WebSockets [[WEBSOCKETS-API]].

RTCPeerConnection Interface Extensions

The Peer-to-peer data API extends the RTCPeerConnection interface as described below.

RTCDataChannel createDataChannel([TreatNullAs=EmptyString] DOMString label, optional RTCDataChannelInit dataChannelDict)

Creates a new RTCDataChannel object with the given label. The RTCDataChannelInit dictionary can be used to configure properties of the underlying channel such as data reliability.

When the createDataChannel() method is invoked, the user agent MUST run the following steps.

  1. If the RTCPeerConnection object’s RTCPeerConnection signalingState is closed, throw an InvalidStateError exception and abort these steps.

  2. Let channel be a newly created RTCDataChannel object.

  3. Initialize channel's label attribute to the value of the first argument.

  4. If the second (dictionary) argument is present, set channel's ordered, maxPacketLifeTime, maxRetransmits, protocol, negotiated and id attributes to the values of their corresponding dictionary members (if present in the dictionary).

  5. If both the maxPacketLifeTime and maxRetransmits attributes are set (not null), then throw a SyntaxError exception and abort these steps.

  6. If an attribute, either maxPacketLifeTime or maxRetransmits, has been set to indicate unreliable mode, and that value exceeds the maximum value supported by the user agent, the value must be set to the user agents maximum value.

  7. If id attribute is uninitialized (not set via the dictionary), initialize it to a value generated by the user agent, according to the WebRTC DataChannel Protocol specification, and skip to the next step. Otherwise, if the value of the id attribute is taken by an existing RTCDataChannel, throw a ResourceInUse exception and abort these steps.

  8. Return channel and continue the following steps in the background.

  9. Create channel's associated underlying data transport and configure it according to the relevant properties of channel.

attribute EventHandler ondatachannel
This event handler, of type datachannel, MUST be supported by all objects implementing the RTCPeerConnection interface.

RTCDataChannel

The RTCDataChannel interface represents a bi-directional data channel between two peers. A RTCDataChannel is created via a factory method on an RTCPeerConnection object. The messages sent between the browsers are described in [[!RTCWEB-DATA]] and [[!RTCWEB-DATA-PROTOCOL]].

There are two ways to establish a connection with RTCDataChannel. The first way is to simply create a RTCDataChannel at one of the peers with the negotiated RTCDataChannelInit dictionary member unset or set to its default value false. This will announce the new channel in-band and trigger a RTCDataChannelEvent with the corresponding RTCDataChannel object at the other peer. The second way is to let the application negotiate the RTCDataChannel. To do this, create a RTCDataChannel object with the negotiated RTCDataChannelInit dictionary member set to true, and signal out-of-band (e.g. via a web server) to the other side that it should create a corresponding RTCDataChannel with the negotiated RTCDataChannelInit dictionary member set to true and the same id. This will connect the two separately created RTCDataChannel objects. The second way makes it possible to create channels with asymmetric properties and to create channels in a declarative way by specifying matching ids.

Each RTCDataChannel has an associated underlying data transport that is used to transport actual data to the other peer. The transport properties of the underlying data transport, such as in order delivery settings and reliability mode, are configured by the peer as the channel is created. The properties of a channel cannot change after the channel has been created. The actual wire protocol between the peers is specified by the WebRTC DataChannel Protocol specification (TODO: reference needed).

A RTCDataChannel can be configured to operate in different reliability modes. A reliable channel ensures that the data is delivered at the other peer through retransmissions. An unreliable channel is configured to either limit the number of retransmissions (maxRetransmits ) or set a time during which transmissions (including retransmissions) are allowed (maxPacketLifeTime). These properties can not be used simultaneously and an attempt to do so will result in an error. Not setting any of these properties results in a reliable channel.

A RTCDataChannel, created with createDataChannel() or dispatched via a RTCDataChannelEvent, MUST initially be in the connecting state. When the RTCDataChannel object’s underlying data transport is ready, the user agent MUST announce the RTCDataChannel as open.

When the user agent is to announce a RTCDataChannel as open, the user agent MUST queue a task to run the following steps:

  1. If the associated RTCPeerConnection object's RTCPeerConnection signalingState is closed, abort these steps.

  2. Let channel be the RTCDataChannel object to be announced.

  3. Set channel's readyState attribute to open.

  4. Fire a simple event named open at channel.

When an underlying data transport is to be announced (the other peer created a channel with negotiated unset or set to false), the user agent of the peer that did not initiate the creation process MUST queue a task to run the following steps:

  1. If the associated RTCPeerConnection object's RTCPeerConnection signalingState is closed, abort these steps.

  2. Let channel be a newly created RTCDataChannel object.

  3. Let configuration be an information bundle received from the other peer as a part of the process to establish the underlying data transport described by the WebRTC DataChannel Protocol specification.

  4. Initialize channel's label, ordered, maxPacketLifeTime, maxRetransmits, protocol, negotiated and id attributes to their corresponding values in configuration.

  5. Set channel's readyState attribute to connecting.

  6. Fire a datachannel event named datachannel with channel at the RTCPeerConnection object.

An RTCDataChannel object's underlying data transport may be torn down in a non-abrupt manner by running the closing procedure. When that happens the user agent MUST, unless the procedure was initiated by the close() method, queue a task that sets the object's readyState attribute to closing. This will eventually render the data transport closed.

References to protocol specification are needed.

When a RTCDataChannel object's underlying data transport has been closed, the user agent MUST queue a task to run the following steps:

  1. Let channel be the RTCDataChannel object whose transport was closed.

    The data transport protocol will specify what happens to, e.g. buffered data, when the data transport is closed.
  2. Set channel's readyState attribute to closed.

  3. If the transport was closed with an error, fire an NetworkError event at channel.

  4. Fire a simple event named close at channel.

readonly attribute DOMString label

The RTCDataChannel.label attribute represents a label that can be used to distinguish this RTCDataChannel object from other RTCDataChannel objects. The attribute MUST return the value to which it was set when the RTCDataChannel object was created.

readonly attribute boolean ordered

The RTCDataChannel.ordered attribute returns true if the RTCDataChannel is ordered, and false if other of order delivery is allowed. The attribute MUST be initialized to true by default and MUST return the value to which it was set when the RTCDataChannel was created.

readonly attribute unsigned? maxPacketLifeTime

The RTCDataChannel.maxPacketLifeTime attribute returns the length of the time window (in milliseconds) during which transmissions and retransmissions may occur in unreliable mode, or null if unset. The attribute MUST be initialized to null by default and MUST return the value to which it was set when the RTCDataChannel was created.

readonly attribute unsigned? maxRetransmits

The RTCDataChannel.maxRetransmits attribute returns the maximum number of retransmissions that are attempted in unreliable mode, or null if unset. The attribute MUST be initialized to null by default and MUST return the value to which it was set when the RTCDataChannel was created.

readonly attribute DOMString protocol

The RTCDataChannel.protocol attribute returns the name of the sub-protocol used with this RTCDataChannel if any, or the empty string otherwise. The attribute MUST be initialized to the empty string by default and MUST return the value to which it was set when the RTCDataChannel was created.

readonly attribute boolean negotiated

The RTCDataChannel.negotiated attribute returns true if this RTCDataChannel was negotiated by the application, or false otherwise. The attribute MUST be initialized to false by default and MUST return the value to which it was set when the RTCDataChannel was created.

readonly attribute unsigned short id

The RTCDataChannel.id attribute returns the id for this RTCDataChannel . The id was either assigned by the user agent at channel creation time or selected by the script. The attribute MUST return the value to which it was set when the RTCDataChannel was created.

readonly attribute RTCDataChannelState readyState

The RTCDataChannel.readyState attribute represents the state of the RTCDataChannel object. It MUST return the value to which the user agent last set it (as defined by the processing model algorithms).

readonly attribute unsigned long bufferedAmount

The bufferedAmount attribute MUST return the number of bytes of application data (UTF-8 text and binary data) that have been queued using send() but that, as of the last time the event loop started executing a task, had not yet been transmitted to the network. (This thus includes any text sent during the execution of the current task, regardless of whether the user agent is able to transmit text asynchronously with script execution.) This does not include framing overhead incurred by the protocol, or buffering done by the operating system or network hardware. If the channel is closed, this attribute's value will only increase with each call to the send() method (the attribute does not reset to zero once the channel closes).

attribute EventHandler onopen
This event handler, of type open, MUST be supported by all objects implementing the RTCDataChannel interface.
attribute EventHandler onerror
This event handler, of type error, MUST be supported by all objects implementing the RTCDataChannel interface.
attribute EventHandler onclose
This event handler, of type close, MUST be supported by all objects implementing the RTCDataChannel interface.
void close()

Closes the RTCDataChannel. It may be called regardless of whether the RTCDataChannel object was created by this peer or the remote peer.

When the RTCDataChannel close() method is called, the user agent MUST run the following steps:

  1. Let channel be the RTCDataChannel object which is about to be closed.

  2. If channel's readyState is closing or closed, then abort these steps.

  3. Set channel's readyState attribute to closing.

  4. If the closing procedure has not started yet, start it.

attribute EventHandler onmessage
This event handler, of type message ,MUST be supported by all objects implementing the RTCDataChannel interface.
attribute DOMString binaryType

The binaryType attribute MUST, on getting, return the value to which it was last set. On setting, the user agent must set the IDL attribute to the new value. When a RTCDataChannel object is created, the binaryType attribute MUST be initialized to the string "blob".

This attribute controls how binary data is exposed to scripts. See the [[WEBSOCKETS-API]] for more information.

void send(DOMString data)

Run the steps described by the send() algorithm with argument type string object.

void send(Blob data)

Run the steps described by the send() algorithm with argument type Blob object.

void send(ArrayBuffer data)

Run the steps described by the send() algorithm with argument type ArrayBuffer object.

void send(ArrayBufferView data)

Run the steps described by the send() algorithm with argument type ArrayBufferView object.

boolean ordered = true

If set to false, data is allowed to be delivered out of order. The default value of true, guarantees that data will be delivered in order.

unsigned short maxPacketLifeTime

Limits the time during which the channel will transmit or retransmit data if not acknowledged. This value may be clamped if it exceeds the maximum value supported by the user agent.

unsigned short maxRetransmits

Limits the number of times a channel will retransmit data if not successfully delivered. This value may be clamped if it exceeds the maximum value supported by the user agent..

DOMString protocol = ""

Subprotocol name used for this channel.

boolean negotiated = false

The default value of false tells the user agent to announce the channel in-band and instruct the other peer to dispatch a corresponding RTCDataChannel object. If set to true, it is up to the application to negotiate the channel and create a RTCDataChannel object with the same id at the other peer.

unsigned short id

Overrides the default selection of id for this channel.

The send() method is overloaded to handle different data argument types. When any version of the method is called, the user agent MUST run the following steps:

  1. Let channel be the RTCDataChannel object on which data is to be sent.

  2. If channel’s readyState attribute is connecting, throw an InvalidStateError exception and abort these steps.

  3. Execute the sub step that corresponds to the type of the methods argument:

    • string object:

      Let data be the result of converting the argument object to a sequence of Unicode characters and increase the bufferedAmount attribute by the number of bytes needed to express data as UTF-8.

    • Blob object:

      Let data be the raw data represented by the Blob object and increase the bufferedAmount attribute by the size of data, in bytes.

    • ArrayBuffer object:

      Let data be the data stored in the buffer described by the ArrayBuffer object and increase the bufferedAmount attribute by the length of the ArrayBuffer in bytes.

    • ArrayBufferView object:

      Let data be the data stored in the section of the buffer described by the ArrayBuffer object that the ArrayBufferView object references and increase the bufferedAmount attribute by the length of the ArrayBufferView in bytes.

  4. If channel’s underlying data transport is not established yet, or if the closing procedure has started, then abort these steps.

  5. Attempt to send data on channel’s underlying data transport; if the data cannot be sent, e.g. because it would need to be buffered but the buffer is full, the user agent MUST abruptly close channel’s underlying data transport with an error.

connecting

The user agent is attempting to establish the underlying data transport. This is the initial state of a RTCDataChannel object created with createDataChannel() .

open

The underlying data transport is established and communication is possible. This is the initial state of a RTCDataChannel object dispatched as a part of a RTCDataChannelEvent .

closing

The procedure to close down the underlying data transport has started.

closed

The underlying data transport has been closed or could not be established.

RTCDataChannelEvent

The datachannel event uses the RTCDataChannelEvent interface.

Firing a datachannel event named e with a RTCDataChannel channel means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the RTCDataChannelEvent interface with the channel attribute set to channel, MUST be created and dispatched at the given target.

Constructor(DOMString type, RTCDataChannelEventInit eventInitDict)
readonly attribute RTCDataChannel channel

The channel attribute represents the RTCDataChannel object associated with the event.

RTCDataChannel channel

TODO

Garbage Collection

A RTCDataChannel object MUST not be garbage collected if its

Peer-to-peer DTMF

In order to send DTMF (phone keypad) values across an RTCPeerConnection, the user agent needs to know which MediaStreamTrack on which RTCPeerConnection will carry the DTMF. This section describes an interface on RTCPeerConnection to associate DTMF capability with a MediaStreamTrack for that RTCPeerConnection. Details of how DTMF is sent to the other peer are described in [[!RTCWEB-AUDIO]].

RTCPeerConnection Interface Extensions

The Peer-to-peer DTMF API extends the RTCPeerConnection interface as described below.

RTCDTMFSender createDTMFSender (MediaStreamTrack track)

The createDTMFSender() method creates an RTCDTMFSender that references the given MediaStreamTrack. The MediaStreamTrack MUST be an element of a MediaStream that's currently in the RTCPeerConnection object's local streams set; if not, throw an InvalidParameter exception and abort these steps.

RTCDTMFSender

An RTCDTMFSender is created by calling the createDTMFSender() method on an RTCPeerConnection. This constructs an object that exposes the functions required to send DTMF on the given MediaStreamTrack.

readonly attribute boolean canInsertDTMF

The canInsertDTMF attribute MUST indicate if the RTCDTMFSender is capable of sending DTMF.

void insertDTMF(in DOMString tones, optional long duration, long interToneGap)

An RTCDTMFSender object’s insertDTMF() method is used to send DTMF tones.

The tones parameter is treated as a series of characters. The characters 0 through 9, A through D, #, and * generate the associated DTMF tones. The characters a to d are equivalent to A to D. The character ',' indicates a delay of 2 seconds before processing the next character in the tones parameter. Unrecognized characters are ignored.

The duration parameter indicates the duration in ms to use for each character passed in the tones parameters. The duration cannot be more than 6000 ms or less than 40 ms. The default duration is 100 ms for each tone.

The interToneGap parameter indicates the gap between tones. It MUST be at least 30 ms. The default value is 70 ms.

The browser MAY increase the duration and interToneGap times to cause the times that DTMF start and stop to align with the boundaries of RTP packets but it MUST not increase either of them by more than the duration of a single RTP audio packet.

ISSUE: How are invalid values handled?

When the insertDTMF() method is invoked, the user agent MUST run the following steps:

  1. If the associated MediaStreamTrack is not connected to the associated RTCPeerConnection, return.
  2. If the canInsertDTMF attribute is false, return.
  3. Set the value of the toneBuffer attribute to the value of the tones argument, the value of the duration attribute to the duration argument if specified, and the value of the interToneGap to the interToneGap argument, if specified.
  4. If toneBuffer is an empty string, return.
  5. If a Playout task is scheduled to be run; abort these steps; otherwise queue a task that runs the following steps (Playout task):
    1. If toneBuffer is an empty string, fire an event named tonechange with an empty string at the RTCDTMFSender object and abort these steps.
    2. Remove the first character from toneBuffer and let that character be tone.
    3. Start playout of tone for duration ms on the associated RTP media stream, using the appropriate codec.
    4. Queue a task to be executed in duration + interToneGap ms from now that runs the steps labelled Playout task.
    5. Fire an event named tonechange with a string consisting of tone at the RTCDTMFSender object.

Calling insertDTMF() with an empty tones parameter can be used to cancel all tones queued to play after the currently playing tone.

readonly attribute MediaStreamTrack track

The track attribute MUST return the MediaStreamTrack given as argument to the createDTMFSender() method.

attribute EventHandler ontonechange

This event handler uses the RTCDTMFToneChangeEvent interface to return the character for each tone as it is played out. See RTCDTMFToneChangeEvent for details.

readonly attribute DOMString toneBuffer

The toneBuffer attribute MUST return a list of the tones remaining to be played out. For the syntax, content, and interpretation of this list, see insertDTMF.

readonly attribute long duration

The duration attribute MUST return the current tone duration value. This value will be the value last set via the insertDTMF() method, or the default value of 100 ms if insertDTMF() was called without specifying the duration.

readonly attribute long interToneGap

The interToneGap attribute MUST return the current value of the between-tone gap. This value will be the value last set via the insertDTMF() method, or the default value of 70 ms if insertDTMF() was called without specifying the interToneGap.

RTCDTMFToneChangeEvent

The tonechange event uses the RTCDTMFToneChangeEvent interface.

Firing a tonechange event named e with a DOMString tone means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the RTCDTMFToneChangeEvent interface with the tone attribute set to tone, MUST be created and dispatched at the given target.

Constructor(DOMString type, RTCDTMFToneChangeEventInit eventInitDict)
readonly attribute DOMString tone

The tone attribute contains the character for the tone that has just begun playout (see insertDTMF()). If the value is the empty string, it indicates that the previous tone has completed playback.

DOMString tone

TODO

Statistics Model

Introduction

The basic statistics model is that the browser maintains a set of statistics referenced by a selector. The selector may, for example, be a MediaStreamTrack. For a track to be a valid selector, it must be a member of a MediaStream that is sent or received by the RTCPeerConnection object on which the stats request was issued. The calling Web application provides the selector to the getStats() method and the browser emits (in the JavaScript) a set of statistics that it believes is relevant to the selector.

Evaluate the need for other selectors than MediaStreamTrack.

The statistics returned are designed in such a way that repeated queries can be linked by the RTCStats id dictionary member. Thus, a Web application can make measurements over a given time period by requesting measurements at the beginning and end of that period.

RTCPeerConnection Interface Extensions

The Statistics API extends the RTCPeerConnection interface as described below.

void getStats(MediaStreamTrack? selector, RTCStatsCallback successCallback, RTCPeerConnectionErrorCallback failureCallback)

Gathers stats for the given selector and reports the result asynchronously.

When the getStats() method is invoked, the user agent MUST queue a task to run the following steps:

  1. If the RTCPeerConnection object's RTCPeerConnection signalingState is closed, throw an InvalidStateError exception.

  2. Return, but continue the following steps in the background.

  3. Let selectorArg be the methods first argument.

  4. If selectorArg is an invalid selector, the user agent MUST queue a task to invoke the failure callback (the method's third argument).

  5. Start gathering the stats indicated by selectorArg. In case selectorArg is null, stats MUST be gathered for the whole RTCPeerConnection object.

  6. When the relevant stats have been gathered, queue a task to invoke the success callback (the method's second argument) with a new RTCStatsReport object, representing the gathered stats, as its argument.

RTCStatsCallback

RTCStatsReport report

A RTCStatsReport representing the gathered stats.

RTCStatsReport Object

The getStats() method delivers a successful result in the form of a RTCStatsReport object. A RTCStatsReport object represents a map between strings, identifying the inspected objects (RTCStats.id), and their corresponding RTCStats objects.

An RTCStatsReport may be composed of several RTCStats objects, each reporting stats for one underlying object that the implementation thinks is relevant for the selector. One achieves the total for the selector by summing over all the stats of a certain type; for instance, if a MediaStreamTrack is carried by multiple SSRCs over the network, the RTCStatsReport may contain one RTCStats object per SSRC (which can be distinguished by the value of the "ssrc" stats attribute).

getter RTCStats (DOMString id)

Getter to retrieve the RTCStats objects that this stats report is composed of.

The set of supported property names [[!WEBIDL]] is defined as the ids of all the RTCStats objects that has been generated for this stats report. The order of the property names is left to the user agent.

RTCStats Dictionary

An RTCStats dictionary represents the stats gathered by inspecting a specific object relevant to a selector. The RTCStats dictionary is a base type that specifies as set of default attributes, such as timestamp and type. Specific stats are added by extending the RTCStats dictionary.

Note that while stats names are standardized, any given implementation may be using experimental values or values not yet known to the Web application. Thus, applications MUST be prepared to deal with unknown stats.

OPEN ISSUE: Need to define an IANA registry for this and populate with pointers to existing things such as the RTCP statistics.

Statistics need to be synchronized with each other in order to yield reasonable values in computation; for instance, if "bytesSent" and "packetsSent" are both reported, they both need to be reported over the same interval, so that "average packet size" can be computed as "bytes / packets" - if the intervals are different, this will yield errors. Thus implementations MUST return synchronized values for all stats in a RTCStats object.

DOMHiResTimeStamp timestamp

The timestamp, of type DOMHiResTimeStamp [[!HIGHRES-TIME]], associated with this object. The time is relative to the UNIX epoch (Jan 1, 1970, UTC).

RTCStatsType type

The type of this object.

The type attribute MUST be initialized to the name of the most specific type this RTCStats dictionary represents.

DOMString id

A unique id that is associated with the object that was inspected to produce this RTCStats object. Two RTCStats objects, extracted from two different RTCStatsReport objects, MUST have the same id if they were produced by inspecting the same underlying object. User agents are free to pick any format for the id as long as it meets the requirements above.

Consider naming id something that indicates that the id refers to the underlying object that was inspected to produce the stats, instead of being an id for the JavaScript object. Suggestions: statsObjectId, reporterId, srcId.
inbound-rtp
Inbound RTP.
outbound-rtp
Outbund RTP.

Derived Stats Dictionaries

DOMString ssrc

...

DOMString remoteId

The remoteId can be used to look up the corresponding RTCStats object that represents stats reported by the other peer.

unsigned long packetsReceived

...

unsigned long bytesReceived

...

unsigned long packetsSent

...

unsigned long bytesSent

...

Example

Consider the case where the user is experiencing bad sound and the application wants to determine if the cause of it is packet loss. The following example code might be used:

var baselineReport, currentReport;
var selector = pc.getRemoteStreams()[0].getAudioTracks()[0];

pc.getStats(selector, function (report) {
    baselineReport = report;
});

// ... wait a bit
setTimeout(function () {
    pc.getStats(selector, function (report) {
        currentReport = report;
        processStats();
    });
}, aBit);

function processStats() {
    // compare the elements from the current report with the baseline
    for each (var now in currentReport) {
        if (now.type != "outbund-rtp")
            continue;

        // get the corresponding stats from the baseline report
        base = baselineReport[now.id];

        if (base) {
            remoteNow = currentReport[now.remoteId];
            remoteBase = baselineReport[base.remoteId];

            var packetsSent = now.packetsSent - base.packetsSent;
            var packetsReceived = remoteNow.packetsReceived - remoteBase.packetsReceived;

            // if fractionLost is > 0.3, we have probably found the culprit
            var fractionLost = (packetsSent - packetsReceived) / packetsSent;
        }
    }
}

Identity

Identity Provider Interaction

WebRTC offers and answers (and hence the channels established by RTCPeerConnection objects) can be authenticated by using a web-based Identity Provider (IdP). The idea is that the entity sending the offer/answer acts as the Authenticating Party (AP) and obtains an identity assertion from the IdP which it attaches to the offer/answer. The consumer of the offer/answer (i.e., the RTCPeerConnection on which setRemoteDescription() is called) acts as the Relying Party (RP) and verifies the assertion.

The interaction with the IdP is designed to decouple the browser from any particular identity provider; the browser need only know how to load the IdP's JavaScript—which is deterministic from the IdP's identity—and the generic protocol for requesting and verifying assertions. The IdP provides whatever logic is necessary to bridge the generic protocol to the IdP's specific requirements. Thus, a single browser can support any number of identity protocols, including being forward compatible with IdPs which did not exist at the time the browser was written. The generic protocol details are described in [[!RTCWEB-SECURITY-ARCH]]. This document specifies the procedures required to instantiate the IdP proxy, request identity assertions, and consume the results.

Identity Provider Selection

In order to communicate with the IdP, the browser instantiates an isolated interpreted context, effectively an invisible IFRAME. The initial contents of the context are loaded from a URI derived from the IdP's domain name, as described in [[!RTCWEB-SECURITY-ARCH]].

For purposes of generating assertions, the IdP shall be chosen as follows:

  1. If the setIdentityProvider() method has been called, the IdP provided shall be used.
  2. If the setIdentityProvider() method has not been called, then the browser can use an IdP configured into the browser.

In order to verify assertions, the IdP domain name and protocol are taken from the domain and protocol fields of the identity assertion.

Instantiating an IdP Proxy

The browser creates an IdP proxy by loading an isolated, invisible IFRAME with HTML content from the IdP URI. The URI for the IdP is a well-known URI formed from the domain and protocol fields, as specified in [[!RTCWEB-SECURITY-ARCH]].

When an IdP proxy is requiured, the browser performs the following steps:

  1. An invisible, sandboxed IFRAME is created within the browser context. The IFRAME sandbox attribute is set to "allow-forms allow-scripts allow-same-origin" to limit the capabilities available to the IdP. The browser MUST prevent the IdP proxy from navigating the browsing context to a different location. The browser MUST prevent the IdP proxy from interacting with the user (this includes, in particular, popup windows and user dialogs).
  2. Once the IdP proxy is created, the browser creates a MessageChannel [[!webmessaging]] within the context of the IdP proxy and assigns one port from the channel to a variable named rtcwebIdentityPort on the window. This message channel forms the basis of communication between the browser and the IdP proxy. Since it is an essential security property of the web sandbox that a page is unable to insert objects into content from another origin, this ensures that the IdP proxy can trust that messages originating from window.rtcwebIdentityPort are from RTCPeerConnection and not some other page. This protection ensures that pages from other origins are unable to instantiate IdP proxies and obtain identity assertions.
  3. The IdP proxy completes loading and informs the RTCPeerConnection that it is ready by sending a "READY" message to the message channel port [[!RTCWEB-SECURITY-ARCH]]. Once this message is received by the RTCPeerConnection, the IdP is considered ready to receive requests to generate or verify identity assertions.

[TODO: This is not sufficient unless we expect the IdP to protect this information. Otherwise, the a=identity information can be copied from a session with "good" properties to any other session with the same fingerprint information. Since we want to reuse credentials, that would be bad.] The identity mechanism MUST provide an indication to the remote side of whether it requires the stream contents to be protected. Implementations MUST have an user interface that indicates the different cases and identity for these.

Requesting Identity Assertions

The identity assertion request process involves the following steps:

  1. The RTCPeerConnection instantiates an IdP proxy as described in Identity Provider Selection section and waits for the IdP to signal that it is ready.
  2. The IdP sends a "SIGN" message to the IdP proxy. This message includes the material the RTCPeerConnection desires to be bound to the user's identity.
  3. If the user has been authenticated by the IdP, and the IdP is willing to generate an identity assertion, the IdP generates an identity assertion. This step depends entirely on the IdP. The methods by which an IdP authenticates users or generates assertions is not specified, though this could involve interacting with the IdP server or other servers.
  4. The IdP proxy sends a response containing the identity assertion to the RTCPeerConnection over the message channel.
  5. The RTCPeerConnection MAY store the identity assertion for use with future offers or answers.
  6. If the identity request was triggered by a createOffer() or createAnswer(), then the assertion is inserted in the offer/answer SDP.

The format and contents of the messages that are exchanged are described in detail in [[!RTCWEB-SECURITY-ARCH]].

The IdP proxy can return an "ERROR" response. If an error is encountered, the browser MUST generate an idpassertionerror event. No "a=identity" attribute is added to SDP as a result.

The browser SHOULD limit the time that it will allow for this process. This includes both the loading of the IdP proxy and the identity assertion generation. Failure to do so potentially causes the corresponding operation to take an indefinite amount of time. This timer can be cancelled when the IdP produces a response. The timer running to completion can be treated as equivalent to an error from the IdP.

User Login Procedure

An IdP could respond to a request to generate an identity assertion with a "LOGINNEEDED" error. This indicates that the site does not have the necessary information available to it (such as cookies) to authorize the creation of an identity assertion.

The "LOGINNEEDED" response includes a URL for a page where the authorization process can be completed. This URL is exposed to the application through the loginUrl attribute of the idpassertionerror event. This URL might be to a page where a user is able to enter their (IdP) username and password, or otherwise provide any information the IdP needs to authorize a assertion request.

An application can load the login URL in an IFRAME or popup; the resulting page then provides the user with an opportunity to provide information necessary to complete the authorization process.

Once the authorization process is complete, the page loaded in the IFRAME or popup sends a message using postMessage [[!webmessaging]] to the page that loaded it (through the window.opener attribute for popups, or through window.parent for pages loaded in an IFRAME). The message MUST be the DOMString "LOGINDONE". This message informs the application that another attempt at generating an identity assertion is likely to be successful.

Verifying Identity Assertions

Identity assertion validation happens when setRemoteDescription is invoked on RTCPeerConnection. The process runs asynchronously, meaning that validation of an identity assertion does not block the completion of setRemoteDescription.

The identity assertion request process involves the following steps:

  1. The RTCPeerConnection instantiates an IdP proxy as described in Identity Provider Selection section and waits for the IdP to signal that it is ready.
  2. The IdP sends a "VERIFY" message to the IdP proxy. This message includes the assertion from the offer/answer which is to be verified.
  3. The IdP proxy verifies the identity assertion (depending on the authentication protocol this could involve interacting with the IDP server).
  4. Once the assertion is verified, the IdP proxy sends a response containing the verified assertion results to the RTCPeerConnection over the message channel.
  5. The RTCPeerConnection validates that the fingerprint provided by the IdP in the validation response matches the certificate fingerprint that is, or will be, used for communications. This is either by:
    • Ensuring that there is only a single a=fingerprint value across all media sections in the SDP that matches the fingerprint provided by the IdP.
    • Waiting for all DTLS connections to be establishes and checking that the certificate fingerprints on all connections matches the one provided by the IdP.
  6. The RTCPeerConnection validates that the domain portion of the identity matches the domain of the IdP as described in [[!RTCWEB-SECURITY-ARCH]].
  7. The RTCPeerConnection stores the assertion in the peerIdentity attribute and raises a simple event named peeridentity at itself. The assertion information to be displayed MUST contain the domain name of the IdP as provided in the assertion.
  8. The browser MAY display identity information to a user in browser UI. Any user identity information that is displayed in this fashion MUST use a mechanism that cannot be spoofed by content.

The IdP might fail to validate the identity assertion by providing an "ERROR" response to the validation request. Validation can also fail due to the additional checks performed by the browser. In both cases, the process terminates and no identity information is exposed to the application or the user.

The browser MUST raise an idpvalidationerror event if validation of an identity assertion fails for any reason.

If the "peerIdentity" constraint is applied to the RTCPeerConnection, any error MUST cause setRemoteDescription to fail.

The browser SHOULD limit the time that it will allow for this process. This includes both the loading of the IdP proxy and the identity assertion validation. Failure to do so potentially causes the corresponding operation to take an indefinite amount of time. This timer can be cancelled when the IdP produces a response. The timer running to completion can be treated as equivalent to an error from the IdP.

It is possible that different values for the "a=identity" attribute is provided at a media level in SDP. A browser MAY either choose to treat this as an error or ignore the attribute. If multiple different assertions are validated, then they MUST produce identical identity values.

The format and contents of the messages that are exchanged are described in detail in [[!RTCWEB-SECURITY-ARCH]].

RTCPeerConnection Interface Extensions

The Identity API extends the RTCPeerConnection interface as described below.

void setIdentityProvider(DOMString provider, optional DOMString protocol, optional DOMString username)

Sets the identity provider to be used for a given RTCPeerConnection object. Applications need not make this call; if the browser is already configured for an IdP, then that configured IdP will be used to get an assertion.

When the setIdentityProvider() method is invoked, the user agent MUST run the following steps:

  1. If the connection's RTCPeerConnection signalingState is closed, throw an InvalidStateError exception and abort these steps.

  2. Set the current identity provider values to the triplet (provider, protocol, username).

  3. If any identity provider value has changed, discard any stored identity assertion.

Identity provider information is not used until an identity assertion is required, either in response to a call to getIdentityAssertion, or the need to generate SDP with either createOffer or createAnswer.

void getIdentityAssertion()

Initiates the process of obtaining an identity assertion. Applications need not make this call. It is merely intended to allow them to start the process of obtaining identity assertions before a call is initiated. If an identity is needed, either because the browser has been configured with a default identity provider or because the setIdentityProvider() method was called, then an identity will be automatically requested when an offer or answer is created.

When getIdentityAssertion is invoked, queue a task to run the following steps:

  1. If the connection's RTCPeerConnection signalingState is closed, abort these steps.

  2. Request an identity assertion from the IdP.

readonly attribute RTCIdentityAssertion? peerIdentity

Contains the peer identity assertion information if an identity assertion was provided and verified. Once this value is set to a non-null value, it cannot change.

attribute EventHandler onidentityresult
This event handler, of event handler event type identityresult, MUST be fired by all objects implementing the RTCPeerConnection interface. This event is fired when an identity assertion is successfully generated. Note: this event is fired when an identity assertion is generated during the creation of an offer or answer.
attribute EventHandler onpeeridentity
This simple event handler of type peeridentity MUST be fired when a peer identity from a peer is successfully validated.
attribute EventHandler onidpassertionerror
This event handler of type idpassertionerror MUST be fired when an IdP encounters an error in generating an identity assertion.
attribute EventHandler onidpvalidationerror
This event handler of type idvalidationperror MUST be fired when an IdP encounters an error in validating an identity assertion.

RTCIdentityAssertion Type

DOMString idp

A domain name representing the identity provider.

DOMString name

An RFC5322-conformant [[RFC5322]] representation of the verified peer identity. This identity will have been verified via the procedures described in [RTCWEB-SECURITY-ARCH].

RTCIdentityEvent Type

The RTCIdentiytEvent is raised when an IdP produces an identity assertion.

attribute DOMString assertion

A string containing the encoded identity assertion (the information that would be added to the "a=identity" line in SDP [[!RTCWEB-SECURITY-ARCH]]).

RTCIdentityErrorEvent Type

The RTCIdentityErrorEvent is raised when an IdP fails to successfully produce an identity assertion.

attribute DOMString idp
The domain name of the identity provider that is providing the error response.
attribute DOMString protocol
The IdP protocol that is in use.
attribute DOMString? loginUrl
An IdP that is unable to generate an identity assertion due to a lack of sufficient user authentication information can provide a URL to a page where the user can complete authentication. If the IdP provides this URL, this attribute includes the value provided by the IdP.

Examples

The identity system is designed so that applications need not take any special action in order for users to generate and verify identity assertions; if a user has configured an IdP into their browser, then the browser will automatically request/generate assertions and the other side will automatically verify them and display the results. However, applications may wish to exercise tighter control over the identity system as shown by the following examples.

This example shows how to configure the identity provider and protocol.

  pc.setIdentityProvider("example.com", "default", "alice@example.com");

This example shows how to consume identity assertions inside a Web application.

  pc.onpeeridentity = function(e) {
    console.log("IdP= " + e.target.peerIdentity.idp +
                " identity=" + e.target.peerIdentity.name);
  };

Media Stream API Extensions for Network Use

Introduction

The MediaStream interface, as defined in the [[!GETUSERMEDIA]] specification, typically represents a stream of data of audio and/or video. A MediaStream may be extended to represent a stream that either comes from or is sent to a remote node (and not just the local camera, for instance). The extensions required to enable this capability on the MediaStream object will be described in this section. How the media is transmitted to the peer is described in [[!RTCWEB-RTP]], [[!RTCWEB-AUDIO]], [[!RTCWEB-VIDEO]], and [[!RTCWEB-TRANSPORT]].

A MediaStream as defined in [[!GETUSERMEDIA]] may contain zero or more MediaStreamTrack objects. A MediaStreamTrack sent to another peer will appear as one and only one MediaStreamTrack to the recipient. A peer is defined as a user agent that supports this specification.

Channels are the smallest unit considered in the MediaStream specification. Channels are intended to be encoded together for transmission as, for instance, an RTP payload type. All of the channels that a codec needs to encode jointly MUST be in the same MediaStreamTrack and the codecs SHOULD be able to encode, or discard, all the channels in the track.

The concepts of an input and output to a given MediaStream apply in the case of MediaStream objects transmitted over the network as well. A MediaStream created by an RTCPeerConnection object (described later in this document) will take as input the data received from a remote peer. Similarly, a MediaStream from a local source, for instance a camera via [[!GETUSERMEDIA]], will have an output that represents what is transmitted to a remote peer if the object is used with an RTCPeerConnection object.

The concept of duplicating MediaStream objects as described in [[!GETUSERMEDIA]] is also applicable here. This feature can be used, for instance, in a video-conferencing scenario to display the local video from the user’s camera and microphone in a local monitor, while only transmitting the audio to the remote peer (e.g. in response to the user using a "video mute" feature). Combining tracks from different MediaStream objects into a new MediaStream is useful in certain situations.

In this document, we only specify aspects of the following objects that are relevant when used along with an RTCPeerConnection. Please refer to the original definitions of the objects in the [[!GETUSERMEDIA]] document for general information on using MediaStream and MediaStreamTrack.

MediaStream

id

The id attribute specified in MediaStream returns an id that is unique to this stream, so that streams can be recognized after they are sent through the RTCPeerConnection API.

When a MediaStream is created to represent a stream obtained from a remote peer, the id attribute is initialized from information provided by the remote source.

The id of a MediaStream object is unique to the source of the stream, but that does not mean it is not possible to end up with duplicates. For example, a locally generated stream could be sent from one user agent to a remote peer using RTCPeerConnection and then sent back to the original user agent in the same manner, in which case the original user agent will have multiple streams with the same id (the locally-generated one and the one received from the remote peer).

Events on MediaStream

A new media track may be associated with an existing MediaStream. For example, if a remote peer adds a new MediaStreamTrack object to a MediaStream that is being sent over an RTCPeerConnection, this is observed on the local user agent. If this happens for the reason exemplified, or for any other reason than the addTrack() method being invoked locally on a MediaStream or tracks being added as the stream is created (i.e. the stream is initialized with tracks), the user agent MUST run the following steps:

  1. Let stream be the target MediaStream object.

  2. Represent component with track: Run the following steps to create a track representing the incoming component:

    1. Create a MediaStreamTrack object track to represent the component.

    2. Initialize track’s kind attribute to "audio" or "video" depending on the media type of the incoming component.

    3. Initialize track’s id attribute to the component track id.

    4. Initialize track’s label attribute to "remote audio" or "remote video" depending on the media type of the incoming component.

    5. Initialize track’s readyState attribute to muted.

    6. Add track to stream’s track set.

  3. Fire a track event named addtrack with the newly created MediaStreamTrack object at stream.

An existing media track may also be disassociated from a MediaStream. If this happens for any other reason than the removeTrack() method being invoked locally on a MediaStream or the stream being destroyed, the user agent MUST run the following steps:

  1. Let stream be the target MediaStream object.

  2. Let track be the MediaStreamTrack object representing the media component about to be removed.

  3. Remove track from stream’s track set.

  4. Fire a track event named removetrack with track at stream.

The event source for the onended event in the networked case is the RTCPeerConnection object.

MediaStreamTrack

A MediaStreamTrack object’s reference to its MediaStream in the non-local media source case (an RTP source, as is the case for a MediaStream received over an RTCPeerConnection) is always strong.

When a track belongs to a MediaStream that comes from a remote peer and the remote peer has permanently stopped sending data the ended event MUST be fired on the track, as specified in [[!GETUSERMEDIA]].

ISSUE: How do you know when it has stopped? This seems like an SDP question, not a media-level question.

A track in a MediaStream, received with an RTCPeerConnection, MUST have its readyState attribute [[!GETUSERMEDIA]] set to muted until media data arrives.

In addition, a MediaStreamTrack has its readyState set to muted on the remote peer if the local user agent disables the corresponding MediaStreamTrack in the MediaStream that is being sent. When the addstream event triggers on an RTCPeerConnection, all MediaStreamTrack objects in the resulting MediaStream are muted until media data can be read from the RTP source.

ISSUE: How do you know when it has been disabled? This seems like an SDP question, not a media-level question.

MediaStreamEvent

The addstream and removestream events use the MediaStreamEvent interface.

Firing a stream event named e with a MediaStream stream means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the MediaStreamEvent interface with the stream attribute set to stream, MUST be created and dispatched at the given target.

Constructor(DOMString type, MediaStreamEventInit eventInitDict)
readonly attribute MediaStream? stream

The stream attribute represents the MediaStream object associated with the event.

MediaStream stream

TODO

Examples and Call Flows

Simple Peer-to-peer Example

When two peers decide they are going to set up a connection to each other, they both go through these steps. The STUN/TURN server configuration describes a server they can use to get things like their public IP address or to set up NAT traversal. They also have to send data for the signaling channel to each other using the same out-of-band mechanism they used to establish that they were going to communicate in the first place.


var signalingChannel = new SignalingChannel();
var configuration = { "iceServers": [{ "url": "stun:stun.example.org" }] };
var pc;

// call start() to initiate
function start() {
    pc = new RTCPeerConnection(configuration);

    // send any ice candidates to the other peer
    pc.onicecandidate = function (evt) {
        if (evt.candidate)
            signalingChannel.send(JSON.stringify({ "candidate": evt.candidate }));
    };

    // let the "negotiationneeded" event trigger offer generation
    pc.onnegotiationneeded = function () {
        pc.createOffer(localDescCreated, logError);
    }

    // once remote stream arrives, show it in the remote video element
    pc.onaddstream = function (evt) {
        remoteView.src = URL.createObjectURL(evt.stream);
    };

    // get a local stream, show it in a self-view and add it to be sent
    navigator.getUserMedia({ "audio": true, "video": true }, function (stream) {
        selfView.src = URL.createObjectURL(stream);
        pc.addStream(stream);
    }, logError);
}

function localDescCreated(desc) {
    pc.setLocalDescription(desc, function () {
        signalingChannel.send(JSON.stringify({ "sdp": pc.localDescription }));
    }, logError);
}

signalingChannel.onmessage = function (evt) {
    if (!pc)
        start();

    var message = JSON.parse(evt.data);
    if (message.sdp)
        pc.setRemoteDescription(new RTCSessionDescription(message.sdp), function () {
            // if we received an offer, we need to answer
            if (pc.remoteDescription.type == "offer")
                pc.createAnswer(localDescCreated, logError);
        }, logError);
    else
        pc.addIceCandidate(new RTCIceCandidate(message.candidate),
            function () {}, logError);
};

function logError(error) {
    log(error.name + ": " + error.message);
}

Advanced Peer-to-peer Example

This example shows the more complete functionality.

TODO

Peer-to-peer Data Example

This example shows how to create a RTCDataChannel object and perform the offer/answer exchange required to connect the channel to the other peer. The RTCDataChannel is used in the context of a simple chat application and listeners are attached to monitor when the channel is ready, messages are received and when the channel is closed.

This example uses the negotiationneeded event to initiate the offer/answer dialog. The exact behavior surrounding the negotiationneeded event is not specified in detail at the moment. This example can hopefully help to drive that discussion. An assumption made in this example is that the event only triggers when a new negotiation should be started. This means that an action (such as addStream()) that normally would have fired the negotiationneeded event will not do so during an ongoing offer/answer dialog.

var signalingChannel = new SignalingChannel();
var configuration = { "iceServers": [{ "url": "stun:stun.example.org" }] };
var pc;
var channel;

// call start(true) to initiate
function start(isInitiator) {
    pc = new RTCPeerConnection(configuration);

    // send any ice candidates to the other peer
    pc.onicecandidate = function (evt) {
        if (evt.candidate)
            signalingChannel.send(JSON.stringify({ "candidate": evt.candidate }));
    };

    // let the "negotiationneeded" event trigger offer generation
    pc.onnegotiationneeded = function () {
        pc.createOffer(localDescCreated, logError);
    }

    if (isInitiator) {
        // create data channel and setup chat
        channel = pc.createDataChannel("chat");
        setupChat();
    } else {
        // setup chat on incoming data channel
        pc.ondatachannel = function (evt) {
            channel = evt.channel;
            setupChat();
        };
    }
}

function localDescCreated(desc) {
    pc.setLocalDescription(desc, function () {
        signalingChannel.send(JSON.stringify({ "sdp": pc.localDescription }));
    }, logError);
}

signalingChannel.onmessage = function (evt) {
    if (!pc)
        start(false);

    var message = JSON.parse(evt.data);
    if (message.sdp)
        pc.setRemoteDescription(new RTCSessionDescription(message.sdp), function () {
            // if we received an offer, we need to answer
            if (pc.remoteDescription.type == "offer")
                pc.createAnswer(localDescCreated, logError);
        }, logError);
    else
        pc.addIceCandidate(new RTCIceCandidate(message.candidate),
            function () {}, logError);
};

function setupChat() {
    channel.onopen = function () {
        // e.g. enable send button
        enableChat(channel);
    };

    channel.onmessage = function (evt) {
        showChatMessage(evt.data);
    };
}

function sendChatMessage(msg) {
    channel.send(msg);
}

function logError(error) {
    log(error.name + ": " + error.message);
}

Call Flow Browser to Browser

Editors' Note: This example flow needs to be discussed on the list and is likely wrong in many ways.

This shows an example of one possible call flow between two browsers. This does not show the procedure to get access to local media or every callback that gets fired but instead tries to reduce it down to only show the key events and messages.

A message sequence chart detailing a call flow between two browsers

DTMF Example

Examples assume that pc is a connected RTCPeerConnection, and track is an audio track on that connection.

Sending the DTMF signal "1234" with 500 ms duration per tone:

var sender = pc.createDTMFSender(track);
if (sender.canInsertDTMF) {
    var duration = 500;
    sender.insertDTMF("1234", duration);
} else
    log("DTMF function not available");

Send the DTMF signal "1234", and light up the active key using lightKey(key) while the tone is playing (assuming that lightKey("") will darken all the keys):

var sender = pc.createDTMFSender(track);
sender.ontonechange = function (e) {
    if (!e.tone)
        return;
    // light up the key when playout starts
    lightKey(e.tone);
    // turn off the light after tone duration
    setTimeout(lightKey, sender.duration, "");
};
sender.insertDTMF("1234");

Send a 1-second "1" tone followed by a 2-second "2" tone:

var sender = pc.createDTMFSender(track);
sender.ontonechange = function (e) {
    if (e.tone == "1")
        sender.insertDTMF("2", 2000);
};
sender.insertDTMF("1", 1000);

It is always safe to append to the tone buffer. This example appends before any tone playout has started as well as during playout.

var sender = pc.createDTMFSender(track);
sender.insertDTMF("123");
// append more tones to the tone buffer before playout has begun
sender.insertDTMF(sender.toneBuffer + "456");

sender.ontonechange = function (e) {
    if (e.tone == "1")
        // append more tones when playout has begun
        sender.insertDTMF(sender.toneBuffer + "789");
};

Send the DTMF signal "123" and abort after sending "2".

var sender = pc.createDTMFSender(track);
sender.ontonechange = function (e) {
    if (e.tone == "2")
        // empty the buffer to not play any tone after "2"
        sender.insertDTMF("");
};
sender.insertDTMF("123");

Event summary

The following events fire on RTCDataChannel objects:

Event name Interface Fired when...
open Event The RTCDataChannel object's underlying data transport has been established (or re-established).
MessageEvent Event A message was successfully received. TODO: Ref where MessageEvent is defined?
error Event TODO.
close Event The RTCDataChannel object's underlying data transport has bee closed.

The following events fire on RTCPeerConnection objects:

Event name Interface Fired when...
connecting Event TODO
addstream MediaStreamEvent A new stream has been added to the remote streams set.
removestream MediaStreamEvent A stream has been removed from the remote streams set.
negotiationneeded Event The browser wishes to inform the application that session negotiation needs to be done at some point in the near future.
signalingstatechange Event The RTCPeerConnection signalingState has changed. This state change is the result of either setLocalDescription() or setRemoteDescription() being invoked.
iceconnectionstatechange Event The RTCPeerConnection ice connection state has changed.
icecandidate RTCPeerConnectionIceEvent A new RTCIceCandidate is made available to the script.
datachannel RTCDataChannelEvent A new RTCDataChannel is dispatched to the script in response to the other peer creating a channel.
identityresult RTCIdentityEvent A new RTCIdentityEvent is dispatched to the script when an identity assertion is successfully generated by an IdP.
peeridentity Event A new Event is dispatched to the script when an identity assertion provided by a peer is successfully validated.
idpassertionerror RTCIdentityErrorEvent A new RTCIdentityErrorEvent is dispatched to the script when an IdP encounters an error while generating an identity assertion.
idpvalidationerror RTCIdentityErrorEvent A new RTCIdentityErrorEvent is dispatched to the script when an IdP encounters an error while validating an identity assertion.

The following events fire on RTCDTMFSender objects:

Event name Interface Fired when...
tonechange Event The RTCDTMFSender object has either just begun playout of a tone (returned as the tone attribute) or just ended playout of a tone (returned as an empty value in the tone attribute).

Security Considerations

TBD

Change Log

This section will be removed before publication.

Changes since March 21, 2014

  1. Changes to identity-related text:
  2. Bug 25155: maxRetransmitTime is not the name of the SCTP concept it points to.

Changes since January 27, 2014

  1. Refined identity assertion generation and validation.
  2. Default DTMF gap changed from 50 to 70 ms.
  3. Bug 24875: Examples in the WebRTC spec are not updated As per the modified API.

Changes since August 30, 2013

  1. Make RTCPeerConnection close method be idempotent.
  2. Clarified ICE server configuration could contain URI types other than STUN and TURN.
  3. Changed the DTMF timing values.
  4. Allow offerToReceiveAudio/video indicate number of streams to offer.
  5. ACTION-98: Added text about clamping of maxRetransmitTime and maxRetransmits.
  6. ACTION-88: Removed nullable types from dictionaries (added attribute default values for attributes that would be left uninitialized without the init dictionary present.
  7. InvalidMediaStreamTrackError changed to InvalidParameter.
  8. Fire NetworkError when the data transport is closed with an error.
  9. Add an exception for data channel with trying to use existing code.
  10. Change maxRetransmits to be an unsigned type.
  11. Clarify state changes when ICE restarts.
  12. Added InvalidStateError exception for operations on a RTCPeerConnection that is closed.
  13. Major changes to Identity Proxy section.
  14. (ACTION: 95) Moved IceTransports (constraint) to RTCConfiguration dictionary.
  15. (ACTION: 95) Introduced RTCOfferAnswerOptions and RTCOfferOptions dictionaries.
  16. (ACTION: 95) Removed constraints argument from addStream() (and removed IANA Constraints section).
  17. Added validation of the RTCConfiguration dictionary argument(s).
  18. Added getConfiguration() on RTCPeerConnection.

Changes since June 3, 2013

  1. Removed synchronous section left-overs.
  2. RTCIceServer now accepts multiple URLs.
  3. Redefined the meaning of negotiated for DataChannel.
  4. Made iceServers a sequence (instead of an Array).
  5. Updated error reporting (to use DOMError and camel cased names).
  6. Added success and failure callbacks to addIceCandidate().
  7. Made local/remoteDescription attributes nullable.
  8. Added username member to RTCIceServer dictionary.

Changes since March 22, 2013

  1. Added IceRestart constraint.
  2. Big updates on DataChannel API to use new channel setup procedures.

Changes since Feb 22, 2013

  1. Example review: Updated DTMF and Stats examples. Added text about when to fire "negotiationneeded" event to align with examples.
  2. Updated RTCPeerConnection state machine. Added a shared processing model for setLocalDescription()/setRemoteDescription().
  3. Updated simple callflow to match the current API.

Changes since Jan 16, 2013

  1. Initial import of Statistics API to version 2.
  2. Integration of Statistics API version 2.5 started.
  3. Updated Statistics API to match Boston/list discussions.
  4. Extracted API extensions introduced by features, such as the P2P Data API, from the RTCPeerConnection API.
  5. Updated DTMF algorithm to dispatch an event when insertDTMF() is called with an empty string to cancel future tones.
  6. Updated DTMF algorithm to not cancel and reschedule if a playout task is running (only update toneBuffer and other values).

Changes since Dec 12, 2012

  1. Changed AudioMediaStreamTrack to RTCDTMFSender and gave it its own section. Updated text to reflect most recent agreements. Also added examples section.
  2. Replaced the localStreams and remoteStreams attributes with functions returning sequences of MediaStream objects.
  3. Added spec text for attributes and methods adopted from the WebSocket interface.
  4. Changed the state ENUMs and transition diagrams.
  5. Aligned the data channel processing model a bit more with WebSockets (mainly closing the underlying transport).

Changes since Nov 13, 2012

  1. Made some clarifications as to how operation queuing works, and fixed a few errors with the error handling description.
  2. Introduced new representation of tracks in a stream (removed MediaStreamTrackList). Added algorithm for creating a track to represent an incoming network media component.
  3. Renamed MediaStream.label to MediaStream.id (the definition needs some more work).

Changes since Nov 03, 2012

  1. Added text describing the queuing mechanism for RTCPeerConnection.
  2. Updated simple P2P example to include all mandatory (error) callbacks.
  3. Updated P2P data example to include all mandatory (error) callbacks. Also added some missing RTC prefixes.

Changes since Oct 19, 2012

  1. Clarified how createOffer() and createAnswer() use their callbacks.
  2. Made all failure callbacks mandatory.
  3. Added error object types, general error handling principles, and rules for when errors should be thrown.

Changes since Sept 23, 2012

  1. Restructured the document layout and created separate sections for features like Peer-to-peer Data API, Statistics and Identity.

Changes since Aug 16, 2012

  1. Replaced stringifier with serializer on RTCSessionDescription and RTCIceCandidate (used when JSON.stringify() is called).
  2. Removed offer and createProvisionalAnswer arguments from the createAnswer() method.
  3. Removed restart argument from the updateIce() method.
  4. Made RTCDataChannel an EventTarget
  5. Updated simple RTCPeerConnection example to match spec changes.
  6. Added section about RTCDataChannel garbage collection.
  7. Added stuff for identity proxy.
  8. Added stuff for stats.
  9. Added stuff peer and ice state reporting.
  10. Minor changes to sequence diagrams.
  11. Added a more complete RTCDataChannel example
  12. Various fixes from Dan's Idp API review.
  13. Patched the Stats API.

Changes since Aug 13, 2012

  1. Made the RTCSessionDescription and RTCIceCandidate constructors take dictionaries instead of a strings. Also added detailed stringifier algorithm.
  2. Went through the list of issues (issue numbers are only valid with HEAD at fcda53c460). Closed (fixed/wontfix): 1, 8, 10, 13, 14, 16, 18, 19, 22, 23, 24. Converted to notes: 4, 12. Updated: 9.
  3. Incorporate changes proposed by Li Li.
  4. Use an enum for DataChannelState and fix IDLs where using an optional argument also requires all previous optional arguments to have a default value.

Changes since Jul 20, 2012

  1. Added RTC Prefix to names (including the notes below).
  2. Moved to new definition of configuration and ice servers object.
  3. Added correlating lines to candidate structure.
  4. Converted setLocalDescription and setRemoteDescription to be asynchronous.
  5. Added call flows.

Changes since Jul 13, 2012

  1. Removed peer attribute from RTCPeerConnectionIceEvent (duplicates functionality of Event.target attribute).
  2. Removed RTCIceCandidateCallback (no longer used).
  3. Removed RTCPeerConnectionEvent (we use a simple event instead).
  4. Removed RTCSdpType argument from setLocalDescription() and setRemoteDescription(). Updated simple example to match.

Changes since May 28, 2012

  1. Changed names to use RTC Prefix.
  2. Changed the data structure used to pass in STUN and TURN servers in configuration.
  3. Updated simple RTCPeerConnection example (RTCPeerConnection constructor arguments; use icecandidate event).
  4. Initial import of new Data API.
  5. Removed some left-overs from the old Data Stream API.
  6. Renamed "underlying data channel" to "underlying data transport". Fixed closing procedures. Fixed some typos.

Changes since April 27, 2012

  1. Major rewrite of RTCPeerConnection section to line up with IETF JSEP draft.
  2. Added simple RTCPeerConnection example. Initial update of RTCSessionDescription and RTCIceCandidate to support serialization and construction.

Changes since 21 April 2012

  1. Moved MediaStream and related definitions to getUserMedia.
  2. Removed section "Obtaining local multimedia content".
  3. Updated getUserMedia() calls in examples (changes in Media Capture TF spec).
  4. Introduced MediaStreamTrackList interface with support for adding and removing tracks.
  5. Updated the algorithm that is run when RTCPeerConnection receives a stream (create new stream when negotiated instead of when data arrives).

Changes since 12 January 2012

  1. Clarified the relation of Stream, Track, and Channel.

Changes since 17 October 2011

  1. Tweak the introduction text and add a reference to the IETF RTCWEB group.
  2. Changed the first argument to getUserMedia to be an object.
  3. Added a MediaStreamHints object as a second argument to RTCPeerConnection.addStream.
  4. Added AudioMediaStreamTrack class and DTMF interface.

Changes since 23 August 2011

  1. Separated the SDP and ICE Agent into separate agents and added explicit state attributes for each.
  2. Removed the send method from PeerConenction and associated callback function.
  3. Modified MediaStream() constructor to take a list of MediaStreamTrack objects instead of a MediaStream. Removed text about MediaStream parent and child relationship.
  4. Added abstract.
  5. Moved a few paragraphs from the MediaStreamTrack.label section to the MediaStream.label section (where they belong).
  6. Split MediaStream.tracks into MediaStream.audioTracks and MediaStream.videoTracks.
  7. Removed a sentence that implied that track access is limited to LocalMediaStream.
  8. Updated a few getUserMedia()-examples to use MediaStreamOptions.
  9. Replaced calls to URL.getObjectURL() with URL.createObjectURL() in example code.
  10. Fixed some broken getUserMedia() links.
  11. Introduced state handling on MediaStreamTrack (removed state handling from MediaStream).
  12. Reintroduced onended on MediaStream to simplify checking if all tracks are ended.
  13. Aligned the MediaStreamTrack ended event dispatching behavior with that of MediaStream.
  14. Updated the LocalMediaStream.stop() algorithm to implicitly use the end track algorithm.
  15. Replaced an occurrence the term finished track with ended track (to align with rest of spec).
  16. Moved (and extended) the explanation about track references and media sources from LocalMediaStream to MediaStreamTrack.

Acknowledgements

The editors wish to thank the Working Group chairs and Team Contact, Harald Alvestrand, Stefan Håkansson and Dominique Hazaël-Massieux, for their support. Substantial text in this specification was provided by many people including Martin Thomson, Harald Alvestrand, Justin Uberti, and Eric Rescorla.