WebRTC 1.0: Real-time Communication Between Browsers

4.1 Introduction

The MediaStream interface, as defined in the [GETUSERMEDIA] specification, typically represents a stream of data of audio and/or video. A MediaStream may be extended to represent a stream that either comes from or is sent to a remote node (and not just the local camera, for instance). The extensions required to enable this capability on the MediaStream object will be described in this document.

A MediaStream as defined in [GETUSERMEDIA] may contain zero or more MediaStreamTrack objects. A MediaStreamTrack sent to another peer will appear as one and only one MediaStreamTrack to the recipient. A peer is defined as a user agent that supports this specification.

Channels are the smallest unit considered in the MediaStream specification. Channels are intended to be encoded together for transmission as, for instance, an RTP payload type. All of the channels that a codec needs to encode jointly must be in the same MediaStreamTrack and the codecs should be able to encode, or discard, all the channels in the track.

The concepts of an input and output to a given MediaStream apply in the case of MediaStream objects transmitted over the network as well. A MediaStream created by a RTCPeerConnection object (later described in this document) will take as input the data received from a remote peer. Similarly, a MediaStream from a local source, for instance a camera via [GETUSERMEDIA] will have an output that represents what is transmitted to a remote peer if the object is used with a RTCPeerConnection object.

The concept of duplicating MediaStream objects as described in [GETUSERMEDIA] is also applicable here. This feature can be used, for instance, in a video-conferencing scenario to display the local video from the user’s camera and microphone in a local monitor, while only transmitting the audio to the remote peer (e.g. in response to the user using a "video mute" feature). Combining tracks from different MediaStream objects into a new MediaStream is useful in certain cases.

4.2 Interface definitions

4.3 AudioMediaStreamTrack

The AudioMediaStreamTrack is a specialization of a normal MediaStreamTrack that only carries audio and is extended to have the capability to send and/or receive DTMF codes.

interface AudioMediaStreamTrack : MediaStreamTrack {
    readonly attribute boolean canInsertDTMF;
    void insertDTMF (DOMString tones, optional long duration);
};

5.1 RTCPeerConnection

The general operation of the RTCPeerConnection is described in [RTCWEB-JSEP].

enum RTCSdpType {
    "offer",
    "pranswer",
    "answer"
};

[Constructor (optional RTCSessionDescriptionInit descriptionInitDict)]
interface RTCSessionDescription {
             attribute RTCSdpType? type;
             attribute DOMString?  sdp;
};
dictionary RTCSessionDescriptionInit {
    RTCSdpType type;
    DOMString  sdp;
};

callback RTCSessionDescriptionCallback = void (RTCSessionDescription sdp);

callback RTCStatsCallback = void (RTCStatsElement[] statsElements, MediaStreamTrack? selector);

callback RTCVoidCallback = void ();

callback RTCPeerConnectionErrorCallback = void (DOMString errorInformation);

enum RTCPeerState {
    "new",
    "have-local-offer",
    "have-local-pranswer",
    "have-remote-pranswer",
    "active" (also could be called "open", "stable")",
    "closed"
};

enum RTCGatheringState {
    "new",
    "gathering",
    "complete"
};

enum RTCIceState {
    "starting",
    "checking",
    "connected",
    "completed",
    "failed",
    "disconnected",
    "closed"
};

[Constructor (optional RTCIceCandidateInit candidateInitDict)]
interface RTCIceCandidate {
             attribute DOMString?      candidate;
             attribute DOMString?      sdpMid;
             attribute unsigned short? sdpMLineIndex;
};
dictionary RTCIceCandidateInit {
    DOMString      candidate;
    DOMString      sdpMid;
    unsigned short sdpMLineIndex;
};

dictionary RTCIceServer {
    DOMString          url;
    nullable DOMString credential;
};

dictionary RTCConfiguration {
    RTCIceServer[] iceServers;
};

dictionary RTCIdentityAssertion {
    DOMString idp;
    DOMString name;
};

dictionary RTCStatsElement {
    RTCStatsReport local;
    RTCStatsReport remote;
};

interface RTCStatsReport {
    readonly attribute long timestamp;
    any getValue (DOMString statName);
};

typedef MediaStream[] MediaStreamArray;

[Constructor (RTCConfiguration configuration, optional MediaConstraints
        constraints)]
interface RTCPeerConnection : EventTarget  {
    void        createOffer (RTCSessionDescriptionCallback successCallback, optional RTCPeerConnectionErrorCallback failureCallback, optional MediaConstraints constraints);
    void        createAnswer (RTCSessionDescriptionCallback successCallback, optional RTCPeerConnectionErrorCallback? failureCallback = null, optional MediaConstraints constraints = null);
    void        setLocalDescription (RTCSessionDescription description, optional RTCVoidCallback successCallback, optional RTCPeerConnectionErrorCallback failureCallback);
    readonly attribute RTCSessionDescription localDescription;
    void        setRemoteDescription (RTCSessionDescription description, optional RTCVoidCallback successCallback, optional RTCPeerConnectionErrorCallback failureCallback);
    readonly attribute RTCSessionDescription remoteDescription;
    readonly attribute RTCPeerState          readyState;
    void        updateIce (optional RTCConfiguration? configuration = null, optional MediaConstraints? constraints = null);
    void        addIceCandidate (RTCIceCandidate candidate);
    readonly attribute RTCGatheringState     iceGatheringState;
    readonly attribute RTCIceState           iceState;
    readonly attribute MediaStreamArray      localStreams;
    readonly attribute MediaStreamArray      remoteStreams;
    DataChannel createDataChannel ([TreatNullAs=EmptyString] DOMString label, optional DataChannelInit dataChannelDict);
             attribute EventHandler          ondatachannel;
    void        addStream (MediaStream stream, optional MediaConstraints constraints);
    void        removeStream (MediaStream stream);
    void        setIdentityProvider (DOMString provider, optional DOMString protocol, optional optional DOMString username);
    void        getIdentityAssertion ();
    readonly attribute RTCIdentityAssertion? peerIdentity;
    void        getStats (MediaStreamTrack? selector, RTCStatsCallback successCallback, optional RTCPeerConnectionErrorCallback failureCallback);
    void        close ();
             attribute EventHandler          onnegotationneeded;
             attribute EventHandler          onicecandidate;
             attribute EventHandler          onopen;
             attribute EventHandler          onstatechange;
             attribute EventHandler          onaddstream;
             attribute EventHandler          onremovestream;
             attribute EventHandler          ongatheringchange;
             attribute EventHandler          onicechange;
             attribute EventHandler          onidentityresult;
};

6.1 Constraints

TOOD: Need to change the naming and declaration of these constraints to match the constraints draft once that is a bit further along. The names here now are likely not quite right but they serve as a place holder.

The following new constraints are defined that can be used with a RTCPeerConnection object:

OfferToReceiveVideo

This is a enum type constraint that can take the values "true" and "false". The default is a non mandatory "true" for a RTCPeerConnection object that has a video stream at the point in time when the constraints are being evaluated and is non mandatory "false" otherwise.

In some cases, a RTCPeerConnection may wish to receive video but it is not going to send any video. The RTCPeerConnection needs to know if it should signal to the remote side if it wishes to receive video or not. This constraint allows an application to indicate its preferences for receiving video when creating an offer.

OfferToReceiveAudio

This is a enum type constraint that can take the values "true" and "false". The default is a non mandatory "true".

In some cases, a RTCPeerConnection may wish to receive audio but it is not going to send any audio. The RTCPeerConnection needs to know if it should signal to the remote side if it wishes to receive audio. This constraints allows an application to indicate its preferences for receiving audio when creating an offer.

VoiceActivityDetection

This is a enum type constraint that can take the values "true" and "false". The default is a non mandatory "true".

Many codecs and system are capable of detecting "silence" and changing their behavior in this case by doing things such as not transmitting any media. In many cases, such as when dealing with sounds other than spoken voice or emergency calling, it is desirable to be able to turn off this behavior. This constraints allows the application to provide information about if it wishes this type of processing enable or disabled.

IceTransports

This is a enum type constraint that can take the values "none", "relay", and "all". The default is a non mandatory "all".

This constraint indicates which candidates the ICE engine is allowed to use. The value "none" means the ICE engine must not send or receive any packets at this point. The value "relay" indicates the ICE engine must only use media relay candidates such as candidates passing through a TURN server. This can be used to reduce leakage of IP addresses in certain use cases. The value of "all" indicates all values can be used.

RequestIdentity

This is a enum type constraint that can take the values "yes", "no", and "ifconfigured". The default is a non mandatory "ifconfigured".

This constraint indicates whether an identity should be be requested. The constraint may be used with either of the createOffer(), createAnswer() calls or with the constructor.The value "yes" means that an identity must be requested. The value "no" means that no identity is to be requested. The value "ifconfigured" means that an identity will be requested if either the user has configured an identity in the browser or if the setIdentityProvider() call has been made by the JS. As this is the default value, an identity will be requested if and only if the user has configured an IdP in some way. Note that as long as DTLS-SRTP is in used, fingerprints will be sent regardless of the value of this constraint.

TODO items - need to register with IANA.

11.1 DataChannel

The DataChannel interface represents a bi-directional data channel between two peers. A DataChannel is created via a factory method on a RTCPeerConnection object. The corresponding DataChannel object is then dispatched at the other peer if the channel setup was successful.

Each DataChannel has an associated underlying data transport that is used to transport actual data to the other peer. The transport properties of the underlying data transport, such as reliability mode, are configured by the peer taking the initiative to create the channel. The other peer cannot change any transport properties of a offered data channel. The actual wire protocol between the peers is out of the scope for this specification.

A DataChannel created with createDataChannel() must initially be in the connecting state. If the DataChannel object’s underlying data transport is successfully set up, the user agent must announce the DataChannel as open.

When the user agent is to announce a DataChannel as open, the user agent must queue a task to run the following steps:

1. If the associated RTCPeerConnection object’s RTCPeerConnection readiness state is closed (3), abort these steps.

2. Let channel be the DataChannel object to be announced.

3. Set channel’s readyState attribute to open.

4. Fire a simple event named open at channel.

When an underlying data transport has been established, the user agent, of the peer that did not initiate the creation process must queue a task to run the following steps:

1. If the associated RTCPeerConnection object’s RTCPeerConnection readiness state is closed (3), abort these steps.

2. Let configuration be an information bundle with key-value pairs, received from the other peer as a part of the process to establish the underlying data channel.

3. Let channel be a newly created DataChannel object.

4. Initialize channel’s label attribute to value that corresponds to the "label" key in configuration.

5. Initialize channel’s reliable attribute to true.

6. If configuration contains a key named "reliable", set channel’s reliable attribute to the corresponding value.

7. Set channel’s readyState attribute to open.

8. Fire a datachannel event named datachannel with channel at the RTCPeerConnection object.

When the process of tearing down a DataChannel object’s underlying data transport is initiated, the user agent must run the following steps:

1. If the associated RTCPeerConnection object’s RTCPeerConnection readiness state is closed, abort these steps.

2. Let channel be the DataChannel object which is about to be closed.

3. If channel’s readyState is closing or closed, then abort these steps.

4. Set channel’s readyState attribute to closing.

5. Queue a task to run the following steps:

6. 1. Close channel’s underlying data transport.

      Note

      The data transport protocol will specify what happens to, e.g. buffered data, when the data transport is closed.

   2. Set channel’s readyState attribute to closed (3).

   3. Fire a simple event named close at channel.

interface DataChannel : EventTarget {
    readonly attribute DOMString        label;
    readonly attribute boolean          reliable;
    readonly attribute DataChannelState readyState;
    readonly attribute unsigned long    bufferedAmount;
             attribute EventHandler     onopen;
             attribute EventHandler     onerror;
             attribute EventHandler     onclose;
    void close ();
             attribute EventHandler     onmessage;
             attribute DOMString        binaryType;
    void send (DOMString data);
    void send (ArrayBuffer data);
    void send (Blob data);
};

dictionary DataChannelInit {
    boolean reliable;
};

enum DataChannelState {
    "connecting",
    "open",
    "closing",
    "closed"
};

11.2 Examples

var signalingChannel = createSignalingChannel();
var pc;
var configuration = "...";
var channel;

// call start(true) to initiate
function start(isInitiator) {
    pc = new PeerConnection(configuration);

    // send any ice candidates to the other peer
    pc.onicecandidate = function (evt) {
        signalingChannel.send(JSON.stringify({ "candidate": evt.candidate }));
    };

    // let the "negotiationneeded" event trigger negotiation
    pc.onnegotiationneeded = function () {
        pc.createOffer(localDescCreated);
    }

    if (isInitiator) {
        // create data channel and setup chat
        channel = pc.createDataChannel("chat");
        setupChat();
    } else {
        // setup chat on incoming data channel
        pc.ondatachannel = function (evt) {
            channel = evt.channel;
            setupChat();
        };
    }
}

function localDescCreated(desc) {
    pc.setLocalDescription(desc, function () {
        signalingChannel.send(JSON.stringify({ "sdp": pc.localDescription }));
    });
}

signalingChannel.onmessage = function (evt) {
    if (!pc)
        start(false);

    var message = JSON.parse(evt.data);
    if (message.sdp)
        pc.setRemoteDescription(new SessionDescription(message.sdp), function () {
            if (pc.remoteDescription.type == "offer")
                createAnswer(localDescCreated);
        });
    else
        pc.addIceCandidate(new IceCandidate(message.candidate));
};

function setupChat() {
    channel.onopen = function () {
        // e.g. enable send button
        enableChat(channel);
    };

    channel.onmessage = function (evt) {
        showChatMessage(evt.data);
    };
}

function sendChatMessage(msg) {
    channel.send(msg);
}

11.3 Garbage Collection

A DataChannel object must not be garbage collected if its

• readyState is connecting and at least one event listener is registered for open events, message events, error events, or close events.

• readyState is open and at least one event listener is registered for message events, error events, or close events.

• readyState is closing and at least one event listener is registered for error events, or close events.

• underlying data transport is established and data is queued to be transmitted.

12.1 Example

Consider the case where the user is experiencing bad sound and the application wants to determine if the cause is is it packet loss. The sound track is audio track 0 of remote stream 0 of pc1. The following example code might be used:

var baseline, now;
var selector = pc.remoteStreams[0].audioTracks[0];

pc.getStats(selector, function (stats) {
    baseline = stats;
});

// ... wait a bit
setTimeout(function () {
    pc.getStats(selector, function (stats) {
        now = stats;
        processStats();
    });
}, aByte);

function processStats() {
    // Real code would:
    // - Check that timestamp of “local stats” and “remote stats”
    //   are reasonably consistent.
    // - Sum up over all the elements rather than just accessing
    //   element zero.

    var packetsSent = now[0].remote.getValue("packetsSent") -
            baseline[0].remote.getValue("packetsSent");

    var packetsReceived = now[0].local.getValue("packetsReceived") -
            baseline[0].local.getValue("packetsReceived");

    // if fractionLost is > 0.3, we have probably found the culprit
    var fractionLost = (packetsSent - packetsReceived) / packetsSent;
}

13.1 Peer-Connection/IdP Communications

In order to communicate with the IdP, the browser must instantiate an isolated interpreted context [TODO: What's the technical term?], such as an invisible IFRAME. The initial contents of the context are loaded from a URI derived from the IdP's domain name. [RTCWEB-SECURITY-ARCH; Section XXX].

For purposes of generating assertions, the IdP shall be chosen as follows:

1. If the setIdentityProvider() method has been called, the IdP provided shall be used.
2. If the setIdentityProvider() method has not been called, then the browser shall use an IdP configured into the browser. If more than one such IdP is configured, the browser should provide the user with a chooser interface.

In order to verify assertions, the IdP domain name and protocol shall be equal to the "domain" and "protocol" fields of the identity assertion.

The context must have a MessageChannel named window.TBD which is "entangled" to the PeerConnection and is unique to that subcontext. This channel is used for messaging between the PeerConnection and the IdP. All messages sent via this channel are strings, specifically the JSONified versions of JS structs.

All messages sent from the PeerConnection to the IdP context must have an origin of rtcweb://peerconnection/. The fact that ordinary Web pages cannot set their origin values arbitrarily is an essential security feature, as it stops attackers from requesting WebRTC-compatible identity assertions from IdPs. For this reason, the origin must be included in the identity assertion and verified by the consuming PeerConnection.

13.2 Requesting Assertions

The identity assertion request process involves the following steps.

1. The PeerConnection instantiates an IdP context as described in the previous section.
2. The IdP serves up the IdP JS code to the IdP context.
3. Once the IdP is loaded and ready to receive messages it sends a "READY" message ([RTCWEB-SECURITY-ARCH; Section 5.6.5.2]. Note that this does not imply that the user is logged in, merely that enough IdP state is booted up to be ready to handle PostMessage calls.
4. The IdP sends a "SIGN" message (Section 5.6.5.2.2) to the IdP proxy context. This message includes the material the PeerConnection desires to be bound to the user's identity.
5. If the user is not logged in, at this point the IdP will initiate the login process. For instance, it might pop up a dialog box inviting the user to enter their (IdP) username and password.
6. Once the user is logged in (potentially after the previous step), the IdP proxy generates an identity assertion (depending on the authentication protocol this may involve interacting with the IDP server).
7. Once the assertion is generated, the IdP proxy sends a response (Section 5.6.5.2.2) containing the assertion to the PeerConnection over the message channel.
8. The PeerConnection stores the assertion for use with future offers or answers. If the identity request was triggered by a createOffer() or createAnswer(), then the assertion is inserted in the offer/answer.

13.3 Verifying Assertions

13.4 Examples

The identity system is designed so that applications need not take any special action in order for users to generate and verify identity assertions; if a user has configured an IdP into their browser, then the browser will automatically request generate assertions and the other side will automatically verify them and display the results. However, applications may with to exercise tighter control over the identity system as shown by the following examples.

pc.setIdentityProvider("example.com", "default", "alice@example.com");

pc.onidentityresult = function(result) {
  console.log("IdP= " + pc.peerIdentity.idp +
              " identity=" + pc.peerIdentity.name);
};

14.1 RTCPeerConnectionIceEvent

The onicecandidate event of the RTCPeerConnection uses the RTCPeerConnectionIceEvent interface.

Firing a RTCPeerConnectionIceEvent event named e with an RTCIceCandidate candidate means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the RTCPeerConnectionIceEvent interface with the candidate attribute set to the new ICE candidate must be created and dispatched at the given target.

[Constructor(DOMString type, RTCPeerConnectionIceEventInit
      eventInitDict)]
interface RTCPeerConnectionIceEvent : Event {
    readonly attribute RTCIceCandidate candidate;
};
dictionary RTCPeerConnectionIceEventInit : EventInit {
    RTCIceCandidate candidate;
};

14.2 MediaStreamEvent

The addstream and removestream events use the MediaStreamEvent interface.

Firing a stream event named e with a MediaStream stream means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the MediaStreamEvent interface with the stream attribute set to stream, must be created and dispatched at the given target.

[Constructor(DOMString type, MediaStreamEventInit eventInitDict)]
interface MediaStreamEvent : Event {
    readonly attribute MediaStream? stream;
};
dictionary MediaStreamEventInit : EventInit {
    MediaStream stream;
};

14.3 DataChannelEvent

The datachannel event use the DataChannelEvent interface.

Firing a datachannel event named e with a DataChannel channel means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the DataChannelEvent interface with the channel attribute set to channel, must be created and dispatched at the given target.

[Constructor(DOMString type, DataChannelEventInit eventInitDict)]
interface DataChannelEvent : Event {
    readonly attribute DataChannel channel;
};
dictionary DataChannelEventInit : EventInit {
    DataChannel channel;
};