W3C

WebRTC 1.0: Real-time Communication Between Browsers

W3C Editor's Draft 27 April 2012

This version:
http://dev.w3.org/2011/webrtc/editor/webrtc.html
Latest published version:
http://www.w3.org/TR/webrtc/
Latest editor's draft:
http://dev.w3.org/2011/webrtc/editor/webrtc.html
Previous version:
http://dev.w3.org/2011/webrtc/editor/webrtc-20120112.html
Editors:
Adam Bergkvist, Ericsson
Daniel C. Burnett, Voxeo
Cullen Jennings, Cisco
Anant Narayanan, Mozilla

Abstract

This document defines a set of APIs to represent streaming media, including audio and video, in JavaScript, to allow media to be sent over the network to another browser or device implementing the appropriate set of real-time protocols, and media received from another browser or device to be processed and displayed locally. This specification is being developed in conjunction with a protocol specification developed by the IETF RTCWEB group and an API specification to get access to local media devices developed by the Media Capture Task Force.

Status of This Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This document is not complete. It is subject to major changes and, while early experimentations are encouraged, it is therefore not intended for implementation. The API is based on preliminary work done in the WHATWG. The Web Real-Time Communications Working Group expects this specification to evolve significantly based on:

As the specification matures, the group hopes to strike the right balance between a low-level API that would enable interested parties to tweak potentially complex system parameters, and a more high-level API that Web developers can use without a priori technical knowledge about real-time communications.

This document was published by the Web Real-Time Communications Working Group as an Editor's Draft. If you wish to make comments regarding this document, please send them to public-webrtc@w3.org (subscribe, archives). All feedback is welcome.

Publication as an Editor's Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

Table of Contents

1. Conformance

As well as sections marked as non-normative, all authoring guidelines, diagrams, examples, and notes in this specification are non-normative. Everything else in this specification is normative.

The key words must, must not, required, should, should not, recommended, may, and optional in this specification are to be interpreted as described in [RFC2119].

Implementations that use ECMAScript to implement the APIs defined in this specification must implement them in a manner consistent with the ECMAScript Bindings defined in the Web IDL specification [WEBIDL], as this specification uses that specification and terminology.

2. Introduction

This section is non-normative.

There are a number of facets to video-conferencing in HTML covered by this specification:

This document defines the APIs used for these features. This specification is being developed in conjunction with a protocol specification developed by the IETF RTCWEB group and an API specification to get access to local media devices developed by the Media Capture Task Force.

3. Network Stream API

3.1 Introduction

The MediaStream interface, as defined in the [GETUSERMEDIA] specification, typically represents a stream of data of audio and/or video. This object may be extended to also represent a stream that either comes from or is sent to a remote node (and not just the local camera, for instance). The extensions required to enable this capability on the MediaStream object will be described in this document.

A MediaStream as defined in [GETUSERMEDIA] may contain zero or more MediaStreamTrack objects. A MediaStreamTrack sent to another peer must appear as one and only one MediaStreamTrack to the recipient.

Channels are the smallest unit considered in the MediaStream specification. Channels are intended to be encoded together for transmission as, for instance, an RTP payload type. All of the channels that a codec needs to encode jointly must be in the same MediaStreamTrack and the codecs should be able to encode, or discard, all the channels in the track.

The concepts of an input and output to a given MediaStream apply in the case of MediaStream objects transmitted over the network as well. A MediaStream created by a PeerConnection object (later described in this document) will take as input the data received from a remote peer. Similarly, a MediaStream from a local source, for instance a camera via [GETUSERMEDIA] will have an output that represents what is transmitted to a remote peer if the object is used with a PeerConnection object.

The concept of duplicating MediaStream objects as described in [GETUSERMEDIA] is also applicable here. This feature can be used, for instance, in a video-conferencing scenario to display the local video from the user’s camera and microphone in a local monitor, while only transmitting the audio to the remote peer (e.g. in response to the user using a "video mute" feature). Combining tracks from different MediaStream objects into a new MediaStream is useful in certain cases.

3.2 Interface definitions

In this section, we only specify aspects of the the following objects that are relevant when used along with a PeerConnection. Please refer to the original definitions of the objects in the [GETUSERMEDIA] document for general information on using MediaStream and MediaStreamTrack both in and outside the context of PeerConnection.

3.2.1 MediaStream

3.2.1.1 label

The label attribute specified in MediaStream returns a label that is unique to this stream, so that streams can be recognized after they are sent through the PeerConnection API.

When a MediaStream is created to represent a stream obtained from a remote peer, the label attribute is initialized from information provided by the remote source.

The label of a MediaStream object is unique to the source of the stream, but that does not mean it is not possible to end up with duplicates. For example, a locally generated stream could be sent from one user to a remote peer using PeerConnection, and then sent back to the original user in the same manner, in which case the original user will have multiple streams with the same label (the locally-generated one and the one received from the remote peer).

3.2.1.2 Events on MediaStream

A new media component may be associated with an existing MediaStream. This happens, e.g., on the A-side when the B-side adds a new MediaStreamTrack object to one of the track lists of a MediaStream that is being sent over a PeerConnection. If this happens for the reason exemplified, or for any other reason than the add() [GETUSERMEDIA] method being invoked locally on a MediaStreamTrackList or tracks are being added as the stream is created (i.e. the stream is initialized with tracks), the user agent must run the following steps:

  1. Create a MediaStreamTrack object track to represent the new media component.

  2. If track’s kind attribute equals "audio", add it to the MediaStream object’s audioTracks MediaStreamTrackList object.

  3. If track’s kind attribute equals "video", add it to the MediaStream object’s videoTracks MediaStreamTrackList object.

  4. Fire a track event named addtrack with the newly created track at the MediaStreamTrackList object.

An existing media component may also be disassociated from a MediaStream. If this happens for any other reason than the remove() [GETUSERMEDIA] method being invoked locally on a MediaStreamTrackList or the stream is being destroyed, the user agent must run the following steps:

  1. Let track be the MediaStreamTrack object representing the media component about to be removed.

  2. Remove track from the MediaStreamTrackList object.

  3. Fire a track event named removetrack with track at the MediaStreamTrackList object.

The event source for the onended event in the networked case is the PeerConnection object.

3.2.2 MediaStreamTrack

A MediaStreamTrack object’s reference to its MediaStream in the non-local media source case (an RTP source, as is the case for a MediaStream received over a PeerConnection) is always strong.

When a track belongs to a MediaStream that comes from a remote peer and the remote peer has permanently stopped sending data the ended event must be fired on the track, as specified in [GETUSERMEDIA].

A track in a MediaStream, received with a PeerConnection, must have its readyState attribute [GETUSERMEDIA] set to MUTED (1) until media data arrives.

In addition, a MediaStreamTrack has its readyState set to MUTED on the B-side if the A-side disables the corresponding MediaStreamTrack in the MediaStream that is being sent. When the addstream event triggers on a PeerConnection, all MediaStreamTrack objects in the resulting MediaStream are muted until media data can be read from the RTP source.

3.3 AudioMediaStreamTrack

The AudioMediaStreamTrack is a specialization of of a normal MediaStreamTrack that only carries audio and is extended to have the capability to send and/or receive DTMF codes.

interface AudioMediaStreamTrack : MediaStreamTrack {
    readonly attribute boolean canInsertDTMF;
    void insertDTMF (DOMString tones, optional long duration);
};

3.3.1 Attributes

canInsertDTMF of type boolean, readonly

The canInsertDTMF attribute must indicate if the AudioMediaStreamTrack is capable of sending DTMF.

3.3.2 Methods

insertDTMF

When a AudioMediaStreamTrack object’s insertDTMF() method is invoked, the user agent must queue a task that that sends the DTMF tones.

The tone parameters is treated as a series of characters. The characters 0 to 9, A to D, #, and * generated the associated DTMF tones. The characters a to d are equivalent to A to D. The character , indicates a an delay of 2 seconds before processing the next character in the tones parameter. Unrecognized characters are ignored.

The duration parameters indicates the duration in ms to play the each DTMF passed in the tones parameters. The duration can not be more than 6000 or less than 70. The default duration is 100 ms for each tone. The gap between tones must be at least 50 ms but should be as short as possible.

If insertDTMF is called on the same object while an existing task for this object is generate DTMF is still running, the previous task is canceled. Calling insertDTMF with an empty tones parameter can be used to cancel any tones currently being send.

Editor Note: We need to add a callback that is set on the object that is called after the tones are sent. This is needed to allow the application to know when it can send new tones without canceling the tones that are currently being sent.

Editor Note: It seems we would want a callback or event for incoming tones. The proposal sent to the list had them played as audio to the speaker but I don’t see how that is useful.

ParameterTypeNullableOptionalDescription
tonesDOMString
durationlong
Return type: void

4. Peer-to-peer connections

A PeerConnection allows two users to communicate directly, browser-to-browser. Communications are coordinated via a signaling channel provided by script in the page via the server, e.g. using XMLHttpRequest.

Calling new PeerConnection(configuration, signalingCallback) creates a PeerConnection object.

The configuration string gives the address of a STUN or TURN server to use to establish the connection. [STUN] [TURN]

The allowed formats for this string are:

"TYPE 203.0.113.2:3478"

Indicates a specific IP address and port for the server.

"TYPE relay.example.net:3478"

Indicates a specific host and port for the server; the user agent will look up the IP address in DNS.

"TYPE example.net"

Indicates a specific domain for the server; the user agent will look up the IP address and port in DNS.

The "TYPE" is one of:

STUN
Indicates a STUN server
STUNS
Indicates a STUN server that is to be contacted using a TLS session.
TURN
Indicates a TURN server
TURNS
Indicates a TURN server that is to be contacted using a TLS session.

The signalingCallback argument is a method that will be invoked when the user agent needs to send a message to the other host over the signaling channel. When the callback is invoked, convey its first argument (a string) to the other peer using whatever method is being used by the Web application to relay signaling messages. (Messages returned from the other peer are provided back to the user agent using the processSignalingMessage() method.)

A PeerConnection object has an associated PeerConnection signaling callback, a PeerConnection ICE Agent, a PeerConnection readiness state and an SDP Agent. These are initialized when the object is created.

When the PeerConnection() constructor is invoked, the user agent must run the following steps. This algorithm has a synchronous section (which is triggered as part of the event loop algorithm). Steps in the synchronous section are marked with ⌛.

  1. Let serverConfiguration be the constructor’s first argument.

  2. Let signalingCallback be the constructor’s second argument.

  3. Let connection be a newly created PeerConnection object.

  4. Create an ICE Agent and let connection’s PeerConnection ICE Agent be that ICE Agent. [ICE]

  5. If serverConfiguration contains a U+000A LINE FEED (LF) character or a U+000D CARRIAGE RETURN (CR) character (or both), remove all characters from serverConfiguration after the first such character.

  6. Split serverConfiguration on spaces to obtain configuration components.

  7. If configuration components has two or more components, and the first component is a case-sensitive match for one of the following strings:

    • "STUN"
    • "STUNS"
    • "TURN"
    • "TURNS"

    ...then run the following substeps:

    1. Let server type be STUN if the first component of configuration components is 'STUN' or 'STUNS', and TURN otherwise (the first component of configuration components is "TURN" or "TURNS").

    2. Let secure be true if the first component of configuration components is "STUNS" or "TURNS", and false otherwise.

    3. Let host be the contents of the second component of configuration components up to the character before the first U+003A COLON character (:), if any, or the entire string otherwise.

    4. Let port be the contents of the second component of configuration components from the character after the first U+003A COLON character (:) up to the end, if any, or the empty string otherwise.

    5. Configure the PeerConnection ICE Agent’s STUN or TURN server as follows:

      • If server type is STUN, the server is a STUN server. Otherwise, server type is TURN and the server is a TURN server.
      • If secure is true, the server is to be contacted using TLS-over-TCP, otherwise, it is to be contacted using UDP.
      • The IP address, host name, or domain name of the server is host.
      • The port to use is port. If this is the empty string, then only a domain name is configured (and the ICE Agent will use DNS SRV requests to determine the IP address and port).
      • The long-term username for the STUN or TURN server is the ASCII serialization of the entry script’s origin; the long-term password is the empty string.

      If the given IP address, host name, domain name, or port are invalid, then the user agent must act as if no STUN or TURN server is configured.

  8. Let the connection’s PeerConnection signaling callback be signalingCallback.

  9. Set connection’s PeerConnection readiness state to NEW (0).

  10. Set connection’s PeerConnection ice state to NEW (0).

  11. Set connection’s PeerConnection sdp state to NEW (0).

  12. Let connection’s localStreams attribute be an empty read-only MediaStream array.

  13. Let connection’s remoteStreams attribute be an empty read-only MediaStream array.

  14. Return connection, but continue these steps asynchronously.

  15. Await a stable state. The synchronous section consists of the remaining steps of this algorithm. (Steps in synchronous sections are marked with ⌛.)

  16. ⌛ If the ice state is set to NEW, it must queue a task to start gathering ICE address and set the ice state to ICE_GATHERING.

  17. ⌛ Once the ICE address gathering is complete, if there are any streams in localStreams, the SDP Agent will send the initial the SDP offer. The initial SDP offer must contain both the ICE candidate information as well as the SDP to represent the media descriptions for all the streams in localStreams.

During the lifetime of the peerConnection object, the following procedures are followed:

  1. If a local media stream has been added and an SDP offer needs to be sent, and the ICE state is not NEW or ICE_GATHERING, and the SDP Agent state is NEW or SDP_IDLE, then send and queue a task to send an SDP offer and change the SPD state to SDP Waiting.

  2. If an SDP offer has been received, and the SDP state is NEW or SDP_IDLE, pass the ICE candidates from the SDP offer to the ICE Agent and change it state to ICE_CHECKING. Construct an appropriate SDP answer, update the remote streams, queue a task to send the SDP offer, and set the SDPAgent state to SDP_IDLE.

  3. At the point the sdpState changes from NEW to some other state, the readyState changes to NEGOTIATING.

  4. If the ICE Agent finds a candidates that froms a valid connection, the ICE state is changed to ICE_CONNECTED

  5. If the ICE Agent finishes checking all candidates, if a connection has been found, the ice state is changed to ICE_COMPLETED and if not connection has been found it is changed to ICE_FAILED.

  6. If the iceState is ICE_CONNECTED or ICE_COMPLETED and the SDP stat is SDP_IDLE, the readyState is set to ACTIVE.

  7. If the iceState is ICE_FAILED, a task is queued to calls the close method.

  8. The close method will cause the system to wait until the sdpStat is SDP_IDLE then it will send an SDP offer terminating all media and change the readyState to CLOSING as well as stop all ICE process and change the iceState to ICE_CLOSED. Once an SDP anser to this offer is received, the readyState will be changed to CLOSED.

User agents may negotiate any codec and any resolution, bitrate, or other quality metric. User agents are encouraged to initially negotiate for the native resolution of the stream. For streams that are then rendered (using a video element), user agents are encouraged to renegotiate for a resolution that matches the rendered display size.

Starting with the native resolution means that if the Web application notifies its peer of the native resolution as it starts sending data, and the peer prepares its video element accordingly, there will be no need for a renegotiation once the stream is flowing.

All SDP media descriptions for streams represented by MediaStream objects must include a label attribute ("a=label:") whose value is the value of the MediaStream object’s label attribute. [SDP] [SDPLABEL]

PeerConnections must not generate any candidates for media streams whose media descriptions do not have a label attribute ("a=label:"). [ICE] [SDP] [SDPLABEL]

When a user agent has reached the point in the media negotiation where a MediaStream can be created to represent incoming components, the user agent must run the following steps:

  1. Let connection be the PeerConnection expecting this media.

  2. Create a MediaStream object to represent the media stream. Set its label attribute to the value of the SDP Label attribute for that component’s media stream.

  3. Run the following steps for each component in the media stream.

    1. Create a MediaStreamTrack object track to represent the component.

    2. If track’s kind attribute equals "audio", add it to the MediaStream object’s audioTracks MediaStreamTrackList object.

    3. If track’s kind attribute equals "video", add it to the MediaStream object’s videoTracks MediaStreamTrackList object.

    The internal order in the MediaStreamTrackList objects on the receiving side should reflect the order on the sending side. One way to enforce this is to specify the order in the SDP.

  4. Queue a task to run the following substeps:

    1. If the connection’s PeerConnection readiness state is CLOSED (3), abort these steps.

    2. Add the newly created MediaStream object to the end of connection’s remoteStreams array.

    3. Fire a stream event named addstream with the newly created MediaStream object at the connection object.

When a user agent has negotiated media for a component that belongs to a media stream thas is already represented by an existing MediaStream object, the user agent must associate the component with that MediaStream object.

When a PeerConnection finds that a stream from the remote peer has been removed (its port has been set to zero in a media description sent on the signaling channel), the user agent must follow these steps:

  1. Let connection be the PeerConnection associated with the stream being removed.

  2. Let stream be the MediaStream object that represents the media stream being removed, if any. If there isn’t one, then abort these steps.

  3. By definition, stream is now finished.

    A task is thus queued to update stream and fire an event.

  4. Queue a task to run the following substeps:

    1. If the connection’s PeerConnection readiness state is CLOSED (3), abort these steps.

    2. Remove stream from connection’s remoteStreams array.

    3. Fire a stream event named removestream with stream at the connection object.

The task source for the tasks listed in this section is the networking task source.

If a PeerConnection object is consuming a MediaStream and a track is added to one of the stream’s MediaStreaMtrackList objects, by, e.g., the add() method being invoked, the PeerConnection object must add a media component for that track the next time the user agent provides a stable state. The user agent must also remove a media component in same way.

To prevent network sniffing from allowing a fourth party to establish a connection to a peer using the information sent out-of-band to the other peer and thus spoofing the client, the configuration information should always be transmitted using an encrypted connection.

4.1 PeerConnection

[Constructor (DOMString configuration, SignalingCallback signalingCallback)]
interface PeerConnection {
    void        processSignalingMessage (DOMString message);
    const unsigned short NEW = 0;
    const unsigned short NEGOTIATING = 1;
    const unsigned short ACTIVE = 2;
    const unsigned short CLOSING = 4;
    const unsigned short CLOSED = 3;
    readonly attribute unsigned short readyState;
    const unsigned short ICE_GATHERING = 0x100;
    const unsigned short ICE_WAITING = 0x200;
    const unsigned short ICE_CHECKING = 0x300;
    const unsigned short ICE_CONNECTED = 0x400;
    const unsigned short ICE_COMPLETED = 0x500;
    const unsigned short ICE_FAILED = 0x600;
    const unsigned short ICE_CLOSED = 0x700;
    readonly attribute unsigned short iceState;
    const unsigned short SDP_IDLE = 0x1000;
    const unsigned short SDP_WAITING = 0x2000;
    const unsigned short SDP_GLARE = 0x3000;
    readonly attribute unsigned short sdpState;
    DataChannel createDataChannel ([TreatNullAs=EmptyString] DOMString? label, optional DataChannelInit? dataChannelDict);
             attribute Function?      ondatachannel;
    void        addStream (MediaStream stream, MediaStreamHints hints);
    void        removeStream (MediaStream stream);
    readonly attribute MediaStream[]  localStreams;
    readonly attribute MediaStream[]  remoteStreams;
    void        close ();
             attribute Function?      onconnecting;
             attribute Function?      onopen;
             attribute Function?      onstatechange;
             attribute Function?      onaddstream;
             attribute Function?      onremovestream;
};

4.1.1 Attributes

iceState of type unsigned short, readonly

The iceState attribute must return the state of the PeerConnection ICE Agent ICE state, represented by a number from the following list:

PeerConnection . NEW (0)
The object was just created, and no networking has yet occurred.
PeerConnection . ICE_GATHERING (0x100)
The ICE Agent is attempting to establish a gather addresses.
PeerConnection . ICE_WAITING (0x200)
The ICE Agent is waiting for candidates from the other side before it can start checking.
PeerConnection . ICE_CHECKING (0x300)
The ICE Agent is checking candidates but has not yet found a connection.
PeerConnection . ICE_CONNECTED (0x400)
The ICE Agent has found a connection but is still checking other candidates to see if there is a better connection.
PeerConnection . ICE_COMPLETED (0x500)
The ICE Agent has finished checking and found a connection.
PeerConnection . ICE_FAILED (0x600)
The ICE Agent is finished checking all candidates and failed to find a connection.
PeerConnection . ICE_CLOSED (0x700)
The ICE Agent has shut down and is no longer responding to STUN requests.
localStreams of type array of MediaStream, readonly

Returns a live array containing the streams that the user agent is currently attempting to transmit to the remote peer (those that were added with addStream()).

Specifically, it must return the read-only MediaStream array that the attribute was set to when the PeerConnection’s constructor ran.

onaddstream of type Function, nullable
This event handler, of event handler event type addstream, must be supported by all objects implementing the PeerConnection interface.
onconnecting of type Function, nullable
This event handler, of event handler event type connecting, must be supported by all objects implementing the PeerConnection interface.
ondatachannel of type Function, nullable
This event handler, of type datachannel, must be supported by all objects implementing the PeerConnection interface.
onopen of type Function, nullable
This event handler, of event handler event type open, must be supported by all objects implementing the PeerConnection interface.
onremovestream of type Function, nullable
This event handler, of event handler event type removestream, must be supported by all objects implementing the PeerConnection interface.
onstatechange of type Function, nullable
This event handler, of event handler event type open, must be supported by all objects implementing the PeerConnection interface. It is called any time the readyState, iceState, or sdpState changes.
readyState of type unsigned short, readonly

The readyState attribute must return the PeerConnection object’s PeerConnection readiness state, represented by a number from the following list:

PeerConnection . NEW (0)
The object was just created, and no networking has yet occurred.
PeerConnection . NEGOTIATING (1)
The user agent is attempting to establish an connection with the ICE Agent and to negotiate codecs with the SDP Agent.
PeerConnection . ACTIVE (2)
The ICE Agent has found a connection the SDP Agent has performed a round of codec negotiation. It is possible for whatever media was negotiated to flow.
PeerConnection . CLOSING (4)
The PeerConnection object is terminating all media and is in the process of closing the Ice Agent and SDP Agent.
PeerConnection . CLOSED (3)
The connection is closed.
remoteStreams of type array of MediaStream, readonly

Returns a live array containing the streams that the user agent is currently receiving from the remote peer.

Specifically, it must return the read-only MediaStream array that the attribute was set to when the PeerConnection’s constructor ran.

This array is updated when addstream and removestream events are fired.

sdpState of type unsigned short, readonly

The sdpState attribute must return the state of the PeerConnection SDP Agent , represented by a number from the following list:

PeerConnection . NEW (0)
The object was just created, and no networking has yet occurred.
PeerConnection . SDP_IDLE (0x1000)
At least one SDP offer or answer has been exchange and the SDP Agent is ready to send an SDP offer or receive an SDP answer.
PeerConnection . SDP_WAITING (0x2000)
The SDP Agent has sent and offer and is waiting for a answer.
PeerConnection . SDP_GLARE (0x3000)
The SDP Agent received an offer while waiting for an answer and now much wait a random amount of time before retrying to send the offer.

4.1.2 Methods

addStream

Attempts to starting sending the given stream to the remote peer. The format for the MediaStreamHints objects is currently undefined by the specification.

When the other peer starts sending a stream in this manner, an addstream event is fired at the PeerConnection object.

When the addStream() method is invoked, the user agent must run the following steps:

  1. Let stream be the method’s first argument.

  2. Let hints be the method’s second argument.

  3. If the PeerConnection object’s PeerConnection readiness state is CLOSED (3), throw an INVALID_STATE_ERR exception.

  4. If stream is already in the PeerConnection object’s localStreams object, then abort these steps.

  5. Add stream to the end of the PeerConnection object’s localStreams object.

  6. Return from the method.

  7. Parse the hints provided by the application and apply them to the MediaStream, if possible.

  8. Have the PeerConnection add a media stream for stream the next time the user agent provides a stable state. Any other pending stream additions and removals must be processed at the same time.

ParameterTypeNullableOptionalDescription
streamMediaStream
hintsMediaStreamHints
Return type: void
close

When the close() method is invoked, the user agent must run the following steps:

  1. If the PeerConnection object’s PeerConnection readiness state is CLOSED (3), throw an INVALID_STATE_ERR exception.

  2. Destroy the PeerConnection ICE Agent, abruptly ending any active ICE processing and any active streaming, and releasing any relevant resources (e.g. TURN permissions).

  3. Set the object’s PeerConnection readiness state to CLOSED (3).

The localStreams and remoteStreams objects remain in the state they were in when the object was closed.

No parameters.
Return type: void
createDataChannel

Creates a new DataChannel object with the given label. The DataChannelInit dictionary can be used to configure properties of underlying channel such as data reliability. A corresponding DataChannel object is dispatched at the other peer if the channel setup was successful.

When the createDataChannel() method is invoked, the user agent must run the following steps.

  1. If the PeerConnection object’s PeerConnection readiness state is CLOSED (3), throw an INVALID_STATE_ERR exception.

  2. Let channel be a newly created DataChannel object.

  3. Initialize channel’s label attribute to the value of the first argument.

  4. Initialize channel’s reliable attribute to true.

  5. If the second argument is present and it contains a reliable dictionary member, then set channel’s reliable attribute to the dictionary member value.

  6. Return channel and continue these steps in the background.

  7. Create channel’s associated underlying data transport.

ParameterTypeNullableOptionalDescription
labelDOMString
dataChannelDictDataChannelInit
Return type: DataChannel
processSignalingMessage

When a message is relayed from the remote peer over the signaling channel is received by the Web application, pass it to the user agent by calling the processSignalingMessage() method.

The order of messages is important. Passing messages to the user agent in a different order than they were generated by the remote peer’s user agent can prevent a successful connection from being established or degrade the connection’s quality if one is established.

When the processSignalingMessage() method is invoked, the user agent must run the following steps:

  1. Let message be the method’s argument.

  2. Let connection be the PeerConnection object on which the method was invoked.

  3. If connection’s PeerConnection readiness state is CLOSED (3), throw an INVALID_STATE_ERR exception.

  4. If the first four characters of message are not "SDP" followed by a U+000A LINE FEED (LF) character, then abort these steps. (This indicates an error in the signaling channel implementation. User agents may report such errors to their developer consoles to aid debugging.)

    Future extensions to the PeerConnection interface might use other prefix values to implement additional features.

  5. Let sdp be the string consisting of all but the first four characters of message.

  6. Pass the sdp to the PeerConnection SDP Agent as a subsequent offer or answer, to be interpreted as appropriate given the current state of the SDP Agent. [ICE]

When a PeerConnection ICE Agent forms a connection to the the far side and enters the state ICE_CONNECTED, the user agent must queue a task that sets the PeerConnection object’s PeerConnection readiness state to ACTIVE (2) and then fires a simple event named open at the PeerConnection object.

ParameterTypeNullableOptionalDescription
messageDOMString
Return type: void
removeStream

Stops sending the given stream to the remote peer.

When the other peer stops sending a stream in this manner, a removestream event is fired at the PeerConnection object.

When the removeStream() method is invoked, the user agent must run the following steps:

  1. Let stream be the method’s argument.

  2. If the PeerConnection object’s PeerConnection readiness state is CLOSED (3), throw an INVALID_STATE_ERR exception.

  3. If stream is not in the PeerConnection object’s localStreams object, then abort these steps.

  4. Remove stream from the PeerConnection object’s localStreams object.

  5. Return from the method.

  6. Have the PeerConnectionremove the media stream for stream the next time the user agent provides a stable state. Any other pending stream additions and removals must be processed at the same time.

ParameterTypeNullableOptionalDescription
streamMediaStream
Return type: void

4.1.3 Constants

ACTIVE of type unsigned short
A connection has been formed and if any media streams were successfully negotiated, any relevant media can be streaming.
CLOSED of type unsigned short
The close() method has been invoked.
CLOSING of type unsigned short
The object is starting to shut down after the close() method has been invoked.
ICE_CHECKING of type unsigned short
The ICE Agent is checking candidates but has not yet found a connection that works.
ICE_CLOSED of type unsigned short
The ICE Agent is terminating and will no longer repined to STUN connectivity checks.
ICE_COMPLETED of type unsigned short
The ICE Agent has finished checking all candidates and a connection has been formed.
ICE_CONNECTED of type unsigned short
The ICE Agent has found at least one candidate that works but is still checking.
ICE_FAILED of type unsigned short
The ICE Agent has finished checking all candidates and no connection was worked.
ICE_GATHERING of type unsigned short
The ICE Agent is gather addresses that can be used.
ICE_WAITING of type unsigned short
THE ICE Agent has complete gathering addresses and is waiting for candidates to start checking.
NEGOTIATING of type unsigned short
The peerConenction object is attempting to get to the point wehre media can flow.
NEW of type unsigned short
The object was just created and its ICE and SDP Agent have not yet been started.
SDP_GLARE of type unsigned short
Both side sent SDP offers at the same time and the SDP Agent is waiting to be able to retransmit the SDP offer.
SDP_IDLE of type unsigned short
A valid offer anser pair has been exchanged and the SDP Agent is waiting for the next SDP transaction.
SDP_WAITING of type unsigned short
The SDP Agent has sent an SDP offer and is waiting for a response.
PeerConnection implements EventTarget;

All instances of the PeerConnection type are defined to also implement the EventTarget interface.

4.2 SignalingCallback

[Callback, NoInterfaceObject]
interface SignalingCallback {
    void handleEvent (DOMString message, PeerConnection source);
};

4.2.1 Methods

handleEvent
Def TBD
ParameterTypeNullableOptionalDescription
messageDOMString
sourcePeerConnection
Return type: void

4.3 Examples

When two peers decide they are going to set up a connection to each other, they both go through these steps. The STUN/TURN server configuration describes a server they can use to get things like their public IP address or to set up NAT traversal. They also have to send data for the signaling channel to each other using the same out-of-band mechanism they used to establish that they were going to communicate in the first place.

// the first argument describes the STUN/TURN server configuration
var local = new PeerConnection('TURNS example.net', sendSignalingChannel);
local.signalingChannel(...); // if we have a message from the other side, pass it along here

// (aLocalStream is some LocalMediaStream object)
local.addStream(aLocalStream); // start sending video

function sendSignalingChannel(message) {
  ... // send message to the other side via the signaling channel
}

function receiveSignalingChannel (message) {
  // call this whenever we get a message on the signaling channel
  local.signalingChannel(message);
}

local.onaddstream = function (event) {
  // (videoElement is some <video> element)
  videoElement.src = URL.createObjectURL(event.stream);
};

5. Peer-to-peer Data API

The Peer-to-peer Data API lets a web application send and receive generic application data peer-to-peer.

Open issues (this should not be considered as a complete list of open issues)

5.1 DataChannel

The DataChannel interface represents a bi-directional data channel between two peers. A DataChannel is created via a factory method on a PeerConnection object. The corresponding DataChannel object is then dispatched at the other peer if the channel setup was successful.

Each DataChannel has an associated underlying data transport that is used to transport actual data to the other peer. The transport properties of the underlying data transport, such as reliability mode, is configured by the peer taking the initiative to create the channel. The receiving peer cannot change any transport properties of a offered data channel. The actual wire protocol between the peers is out of the scope for this specification.

A DataChannel created with createDataChannel() must initially be in the CONNECTING (0) state. If the DataChannel object’s underlying data transport is successfully set up, the user agent must announce the DataChannel as open.

When the user agent is to announce a DataChannel as open, the user agent must queue a task to run the following steps:

  1. If the associated PeerConnection object’s PeerConnection readiness state is CLOSED (3), abort these steps.

  2. Let channel be the DataChannel object to be announced.

  3. Set channel’s readyState attribute to OPEN (1).

  4. Fire a simple event named open at channel.

When an underlying data transport has been established, the user agent, of the peer that did not initiate the creation process must queue a task to run the following steps:

  1. If the associated PeerConnection object’s PeerConnection readiness state is CLOSED (3), abort these steps.

  2. Let configuration be an information bundle with key-value pairs, received from the other peer as a part of the process to establish the underlying data channel.

  3. Let channel be a newly created DataChannel object.

  4. Initialize channel’s label attribute to value that corresponds to the "label" key in configuration.

  5. Initialize channel’s reliable attribute to true.

  6. If configuration contains a key named "reliable", set channel’s reliable attribute to the corresponding value.

  7. Set channel’s readyState attribute to OPEN (1).

  8. Fire a datachannel event named datachannel with channel at the PeerConnection object.

When the process of tearing down a DataChannel object’s underlying data transport is initiated, the user agent must run the following steps:

  1. If the associated PeerConnection object’s PeerConnection readiness state is CLOSED (3), abort these steps.

  2. Let channel be the DataChannel object which is about to be closed.

  3. If channel’s readyState is CLOSING (2) or CLOSED (3), then abort these steps.

  4. Set channel’s readyState attribute to CLOSING (2).

  5. Queue a task to run the following steps:

    1. Close channel’s underlying data transport.

      The data transport protocol will specify what happens to, e.g. buffered data, when the data transport is closed.
    2. Set channel’s readyState attribute to CLOSED (3).

    3. Fire a simple event named close at channel.

interface DataChannel {
    readonly attribute DOMString      label;
    readonly attribute boolean        reliable;
    const unsigned short CONNECTING = 0;
    const unsigned short OPEN = 1;
    const unsigned short CLOSING = 2;
    const unsigned short CLOSED = 3;
    readonly attribute unsigned short readyState;
    readonly attribute unsigned long  bufferedAmount;
    [TreatNonCallableAsNull]
             attribute Function?      onopen;
    [TreatNonCallableAsNull]
             attribute Function?      onerror;
    [TreatNonCallableAsNull]
             attribute Function?      onclose;
    void close ();
    [TreatNonCallableAsNull]
             attribute Function?      onmessage;
             attribute DOMString      binaryType;
    void send (DOMString data);
    void send (ArrayBuffer data);
    void send (Blob data);
};

5.1.1 Attributes

binaryType of type DOMString

FIXME: align behavior with WebSocket API

bufferedAmount of type unsigned long, readonly

FIXME: align behavior with WebSocket API

label of type DOMString, readonly

The DataChannel.label attribute represents a label that can be used to distinguish this DataChannel object from other DataChannel objects. The attribute must return the value to which it was set when the DataChannel object was created.

onclose of type Function, nullable
This event handler, of type close, must be supported by all objects implementing the DataChannel interface.
onerror of type Function, nullable
This event handler, of type error, must be supported by all objects implementing the DataChannel interface.
onmessage of type Function, nullable
This event handler, of type message, must be supported by all objects implementing the DataChannel interface.
onopen of type Function, nullable
This event handler, of type open, must be supported by all objects implementing the DataChannel interface.
readyState of type unsigned short, readonly

The DataChannel.readyState attribute represents the state of the DataChannel object. It must return the value to which the user agent last set it (as defined by the processing model algorithms). The attribute can have the following values: CONNECTING, OPEN, CLOSING or CLOSED.

reliable of type boolean, readonly

The DataChannel.reliable attribute returns true if the DataChannel is reliable, and false otherwise. The attribute must return the value to which it was set when the DataChannel was created.

5.1.2 Methods

close

Closes the DataChannel. It may be called regardless if the DataChannel object was created by this peer or the remote peer.

When the close() method is called, the user agent must initiate the process of tearing down the DataChannel object’s underlying data transport.

No parameters.
Return type: void
send

FIXME: align behavior with WebSocket API

ParameterTypeNullableOptionalDescription
dataDOMString
Return type: void
send

FIXME: align behavior with WebSocket API

ParameterTypeNullableOptionalDescription
dataArrayBuffer
Return type: void
send

FIXME: align behavior with WebSocket API

ParameterTypeNullableOptionalDescription
dataBlob
Return type: void

5.1.3 Constants

CLOSED of type unsigned short

The underlying data transport has been closed or could not be established.

CLOSING of type unsigned short

The process of closing down the underlying data transport has started.

CONNECTING of type unsigned short

The user agent is attempting to establish the underlying data transport. This is the initial state of a DataChannel object created with createDataChannel().

OPEN of type unsigned short

The underlying data transport is established and communication is possible. This is the initial state of a DataChannel object dispatched as a part of a DataChannelEvent.

dictionary DataChannelInit {
    boolean reliable;
};

5.1.4 Dictionary DataChannelInit Members

reliable of type boolean
-

5.2 Examples

This simple example shows how to create a DataChannel, register an event listener to handle incoming data, and how to send a message.

var chan = peerConn.createDataChannel("mylabel");

chan.onmessage = function (evt) {
    // use evt.data
};

chan.send("hello");

This simple example shows how to register an event listener to handle the case when a remote peer creates a new DataChannel.

peerConn.ondatachannel = function (evt) {
    var chan = evt.channel;

    chan.onmessage = function (evt) {
        // use evt.data
    };

    chan.onclose = function () {
        // remote side closed the data channel
    };
};

6. Garbage collection

A Window object has a strong reference to any PeerConnection objects created from the constructor whose global object is that Window object.

7. Event definitions

7.1 MediaStreamTrackEvent

The addtrack and removetrack events use the MediaStreamTrackEvent interface.

Firing a track event named e with a MediaStreamTrack track means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the MediaStreamTrackEvent interface with the track attribute set to track, must be created and dispatched at the given target.

[Constructor(DOMString type, optional MediaStreamTrackEventInit eventInitDict)]
interface MediaStreamTrackEvent : Event {
    readonly attribute MediaStreamTrack? track;
};
dictionary MediaStreamTrackEventInit : EventInit { MediaStreamTrack? track; };

7.1.1 Attributes

track of type MediaStreamTrack, readonly, nullable

The track attribute represents the MediaStreamTrack object associated with the event.

7.1.2 Dictionary MediaStreamTrackEventInit Members

track of type MediaStreamTrack, nullable

-

7.2 MediaStreamEvent

The addstream and removestream events use the MediaStreamEvent interface.

Firing a stream event named e with a MediaStream stream means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the MediaStreamEvent interface with the stream attribute set to stream, must be created and dispatched at the given target.

[Constructor(DOMString type, optional MediaStreamEventInit eventInitDict)]
interface MediaStreamEvent : Event {
    readonly attribute MediaStream? stream;
};
dictionary MediaStreamEventInit : EventInit { MediaStream? stream; };

7.2.1 Attributes

stream of type MediaStream, readonly, nullable

The stream attribute represents the MediaStream object associated with the event.

7.2.2 Dictionary MediaStreamEventInit Members

stream of type MediaStream, nullable

-

7.3 DataChannelEvent

The datachannel event use the DataChannelEvent interface.

Firing a datachannel event named e with a DataChannel channel means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the DataChannelEvent interface with the channel attribute set to channel, must be created and dispatched at the given target.

[Constructor(DOMString type, optional DataChannelEventInit eventInitDict)]
interface DataChannelEvent : Event {
    readonly attribute DataChannel? channel;
};
dictionary DataChannelEventInit : EventInit { DataChannel? channel; };

7.3.1 Attributes

channel of type DataChannel, readonly, nullable

The channel attribute represents the DataChannel object associated with the event.

7.3.2 Dictionary DataChannelEventInit Members

channel of type DataChannel, nullable
-

8. Event summary

This section is non-normative.

The following event fires on MediaStream objects:

Event name Interface Fired when...
ended Event The MediaStream finished as a result of all tracks in the MediaStream ending.

The following event fires on MediaStreamTrack objects:

Event name Interface Fired when...
muted Event The MediaStreamTrack object’s source is temporarily unable to provide data.
unmuted Event The MediaStreamTrack object’s source is live again after having been temporarily unable to provide data.
ended Event The MediaStreamTrack object’s source will no longer provide any data, either because the user revoked the permissions, or because the source device has been ejected, or because the remote peer stopped sending data, or because the stop() method was invoked.

The following event fires on MediaStreamTrackList objects:

Event name Interface Fired when...
addtrack MediaStreamTrackEvent A new MediaStreamTrack has been added to this list.
removetrack MediaStreamTrackEvent A MediaStreamTrack has been removed from this list.

The following events fire on PeerConnection objects:

Event name Interface Fired when...
connecting Event The ICE Agent has begun negotiating with the peer. This can happen multiple times during the lifetime of the PeerConnection object.
open Event The ICE Agent has finished negotiating with the peer.
datachannel_ DataChannelEvent The other peer successfully created a new DataChannel object.
addstream MediaStreamEvent A new stream has been added to the remoteStreams array.
removestream MediaStreamEvent A stream has been removed from the remoteStreams array.

The following events fire on DataChannel objects:

Event name Interface Fired when...
open Event The DataChannel object’s underlying data transport has been established (or re-established).
error Event -
close Event The DataChannel object’s underlying data transport has was closed.
message MessageEvent A message was successfully received.

9. application/html-peer-connection-data

This registration is for community review and will be submitted to the IESG for review, approval, and registration with IANA.

Type name:
application
Subtype name:
html-peer-connection-data
Required parameters:
No required parameters
Optional parameters:
No optional parameters
Encoding considerations:
This MIME type defines a binary protocol format which uses UTF-8 for text encoding.
Security considerations:

This format is used for encoding UDP packets transmitted by potentially hostile Web page content via a trusted user agent to a destination selected by a potentially hostile remote server. To prevent this mechanism from being abused for cross-protocol attacks, all the data in these packets is masked so as to appear to be random noise. The intent of this masking is to reduce the potential attack scenarios to those already possible previously.

However, this feature still allows random data to be sent to destinations that might not normally have been able to receive them, such as to hosts within the victim’s intranet. If a service within such an intranet cannot handle receiving UDP packets containing random noise, it might be vulnerable to attack from this feature.

Interoperability considerations:
Rules for processing both conforming and non-conforming content are defined in this specification.
Published specification:
This document is the relevant specification.
Applications that use this media type:
This type is only intended for use with SDP. [SDP]
Additional information:
Magic number(s):
No sequence of bytes can uniquely identify data in this format, as all data in this format is intentionally masked to avoid cross-protocol attacks.
File extension(s):
This format is not for use with files.
Macintosh file type code(s):
This format is not for use with files.
Person & email address to contact for further information:
Daniel C. Burnett <dburnett@voxeo.com>
Intended usage:
Common
Restrictions on usage:
No restrictions apply.
Author:
Daniel C. Burnett <dburnett@voxeo.com>
Change controller:
W3C

Fragment identifiers cannot be used with application/html-peer-connection-data as URLs cannot be used to identify streams that use this format.

10. Change Log

This section will be removed before publication.

To Do Items

Need a way to indicate the type of the SDP when passing SDP strings.

Changes since 21 April 2012

  1. Moved MediaStream and related definitions to getUserMedia.

Changes since 12 January 2012

  1. Clarified what relation of Stream, Track, and Channel.

Changes since 17 October 2011

  1. Tweak the introduction text and add a reference to the IETF RTCWEB group.
  2. Changed the first argument to getUserMedia to be an object.
  3. Added a MediaStreamHints object as a second argument to PeerConnection.addStream.
  4. Added AudioMediaStreamTrack class and DTMF interface.

Changes since 23 August 2011

  1. Separated the SDP and ICE Agent into separate agents and added explicit state attributes for each.
  2. Removed the send method from PeerConenction and associated callback function.
  3. Modified MediaStream() constructor to take a list of MediaStreamTrack objects instead of a MediaStream. Removed text about MediaStream parent and child relationship.
  4. Added abstract.
  5. Moved a few paragraphs from the MediaStreamTrack.label section to the MediaStream.label section (where they belong).
  6. Split MediaStream.tracks into MediaStream.audioTracks and MediaStream.videoTracks.
  7. Removed a sentence that implied that track access is limited to LocalMediaStream.
  8. Updated a few getUserMedia()-examples to use MediaStreamOptions.
  9. Replaced calls to URL.getObjectURL() with URL.createObjectURL() in example code.
  10. Fixed some broken getUserMedia() links.
  11. Introduced state handling on MediaStreamTrack (removed state handling from MediaStream).
  12. Reintroduced onended on MediaStream to simplify checking if all tracks are ended.
  13. Aligned the MediaStreamTrack ended event dispatching behavior with that of MediaStream.
  14. Updated the LocalMediaStream.stop() algorithm to implicitly use the end track algorithm.
  15. Replaced an occurrence the term finished track with ended track (to align with rest of spec).
  16. Moved (and extended) the explanation about track references and media sources from LocalMediaStream to MediaStreamTrack.
  17. Removed section "Obtaining local multimedia content".
  18. Updated getUserMedia() calls in examples (changes in Media Capture TF spec).
  19. Introduced MediaStreamTrackList interface with support for adding and removing tracks.
  20. Updated the algorithm that is run when PeerConnection receives a stream (create new stream when negotiated instead of when data arrives).
  21. Removed some left-overs from the old Data Stream API.
  22. Initial import of new Data API.
  23. Renamed "underlying data channel" to "underlying data transport". Fixed closing procedures. Fixed some typos.

A. Acknowledgements

The editors wish to thank the Working Group chairs, Harald Alvestrand and Stefan Håkansson, for their support.

B. References

B.1 Normative references

[GETUSERMEDIA]
D. Burnett, A. Narayanan. getusermedia: Getting access to local devices that can generate multimedia streams 22 December 2011. W3C Editors draft (Work in progress.) URL: http://dev.w3.org/2011/webrtc/editor/getusermedia.html
[ICE]
J. Rosenberg. Interactive Connectivity Establishment (ICE): A Protocol for Network Address Translator (NAT) Traversal for Offer/Answer Protocols. April 2010. Internet RFC 5245. URL: http://tools.ietf.org/html/rfc5245
[RFC2119]
S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. March 1997. Internet RFC 2119. URL: http://www.ietf.org/rfc/rfc2119.txt
[SDP]
J. Rosenberg, H. Schulzrinne. An Offer/Answer Model with the Session Description Protocol (SDP). June 2002. Internet RFC 3264. URL: http://tools.ietf.org/html/rfc3264
[SDPLABEL]
O. Levin, G. Camarillo. The Session Description Protocol (SDP) Label Attribute. August 2006. Internet RFC 4574. URL: http://tools.ietf.org/html/rfc4574
[STUN]
J. Rosenberg, R. Mahy, P. Matthews, D. Wing. Session Traversal Utilities for NAT (STUN). October 2008. Internet RFC 5389. URL: http://tools.ietf.org/html/rfc5389
[TURN]
P. Mahy, P. Matthews, J. Rosenberg. Traversal Using Relays around NAT (TURN): Relay Extensions to Session Traversal Utilities for NAT (STUN). April 2010. Internet RFC 5766. URL: http://tools.ietf.org/html/rfc5766
[WEBIDL]
Cameron McCormack. Web IDL. 27 September 2011. W3C Working Draft. (Work in progress.) URL: http://www.w3.org/TR/2011/WD-WebIDL-20110927/

B.2 Informative references

No informative references.