This specification defines HTML form enhancements that provide access to the audio, image and video capture capabilities of the device.

This document is a the first part of the split of the previous version of this document, focused on the integration of media capture in HTML forms based on an extension to the FileAPI. The second part of the split focused on programmatic access to the capture devices will be published separately.

The Working Group is looking for feedback on the general approach of this new version, and will coordinate with the HTML and Web Applications Working Group to ensure the proper progress of this document.

Issues and editors notes in the document highlight some of the points on which the group is still working and would particularly like to get feedback.

Introduction

The HTML Form Based Media Capturing specification defines a new interface for media files, a new parameter for the accept attribute of the HTML input element in file upload state, and recommendations for providing optimized access to the microphone and camera of a hosting device.

Providing streaming access to these capabilities is outside of the scope of this specification.

The Working Group is investigating the opportunity to specify streaming access via the proposed <device> element.

This specification defines conformance criteria that apply to a single product: the user agent that implements the interfaces that it contains.

Implementations that use ECMAScript to implement the APIs defined in this specification must implement them in a manner consistent with the ECMAScript Bindings defined in the Web IDL specification [[!WEBIDL]], as this specification uses that specification and terminology.

Security and Privacy Considerations

This specification builds upon the security and privacy protections provided by the [[!HTML5]] <input type="file"> and the [[!FILE-API]] specifications; in particular, it is expected that any offer to start capturing content from the user’s device would require a specific user interaction on an HTML element that is entirely controlled by the user agent.

In addition to the requirements already highlighted in the [[!HTML5]] and [[!FILE-API]] specifications, implementors should take care of additional leakage of privacy-sensitive data from captured media. For instance, embedding the user’s location in a captured media metadata (e.g. EXIF) might transmit more private data than the user might be expecting.

Capture aware file-select control

This section is normative.

[[!HTML5]] links <input type="file"> to the File interface. This specification defines a refined MediaFile interface to be used when the accept attribute take certain values — this will require coordination with the HTML5 Working Group.

If an input element in the File Upload state [[!HTML5]] contains accept attribute with values image/*, audio/*, or video/*, the user agent can invoke a file picker that allows respectively the user to take a picture, record a sound file, or record a video in addition to selecting an existing file from the file system.

See the User Interface Examples appendix for the illustration.

In case the user chooses to capture video, audio, or image content, the user agent creates media files on the fly as specified in [[HTML5]].

If the user selects files of whose MIME types match image/*, audio/*, or video/* (on the filesystem or via a successful media capture), the relevant files in the files attribute [[HTML5]] MUST implement the MediaFile interface.

<input type="file" accept="image/*" id="capture"> 

The capture attribute

This section is normative.

The capture attribute may be added to the input element to provide user agents with a hint of that by the default a file picker should be in media capturing mode.

The capture attribute is an enumerated attribute that can take one of the following values: camera, camcorder, microphone, filesystem (ASCII case-insensitive). These values indicate which source the file picker interface should preferably present to the user by default. Both the invalid and missing default value are filesystem.

What to do if there is no accept attribute? What if the accept attribute is set to a value that the pre-hinted device cannot support? See wa href="http://lists.w3.org/Archives/Public/public-device-apis/2011Apr/0013.html">related discussion./p>

attribute DOMString capture
One of camera, camcorder, microphone, filesystem

For example, the following code indicates that the user is expected to upload an image from the device camera:

<input type="file" accept="image/*" capture="camera" id="capture"> 

A possible rendering of a file picker taking this parameter into account is offered in the User Interface Examples appendix.

WebIDL interfaces

Example

After the user successfully captured or selected an existing media file, the format properties of the file can be retrieved as follow:

var captureInput = document.getElementById('capture');
// Accessing the file object from the input element with id capture
var file = captureInput.files[0];
if (file) {
  // getting format data asynchronously
  file.getFormatData(displayFormatData, errorHandler);
}

// success callback when getting format data
function displayFormatData(formatData) {  
  var mainType = file.type.split("/")[0]; // "image", "video" or "audio"
  var mediaDescriptionNode = document.createElement("p");
  if (mainType === "image") {
    mediaDescriptionNode.appendChild(document.createTextNode("This is an image of dimensions " + formatData.width + "x" + formatData.height);
  } else {
    mediaDescriptionNode.appendChild(document.createTextNode("Duration: " + formatData.duration  + "s");
  }
  captureInput.parentNode.insertBefore(mediaDescriptionNode, captureInput);
}

// error callback if getting format data fails
function errorHandler(error) {
  alert("Couldn’t retrieve format properties for the selected file (error code " + error.code + ")");
}

MediaFileData interface

MediaFileData encapsulates format information of a media file.

The relationship between this MediaFileData interface and the properties made available through the API for Media Resource 1.0 [[MEDIAONT-API]] needs further investigation.

attribute DOMString codecs
The type attribute of the Blob interface (inherited from the File interface) is not sufficient to determine the format of the content since it only specifies the container type. The codecs attribute represents the actual format that the audio and video of the content. The codecs attribute MUST conform to the [[!RFC4281]]. For example, a valid value for H.263 video and AAC low complexity would be codecs="s263, mp4a.40.2".

This could be turned into a list of DOMString rather than keeping it as a comma-separated values list; this needs some care with regard to the RFC ref.

attribute unsigned long bitrate
The codecs attribute only specifies the profile and level of the encoded content which doesn't specify the actual bitrate. It only specifies the maximum encoded bitrate, thus this bitrate attribute is the average bitrate of the content. In the case of an image this attribute has value 0.
attribute unsigned long height
The height attribute represents height of the image or video in pixels. In the case of a sound clip this attribute has value 0.
attribute unsigned long width
The width attribute represents width of the image or video in pixels. In the case of a sound clip this attribute has value 0.
attribute float duration
The duration attribute represents length of the video or sound clip in seconds. In the case of an image this attribute has value 0.

Some of the proposed attributes of the MediaFileData interface could possibly be integrated as parameters of the MIME type, or as MIME options object.

MediaFile interface

MediaFile encapsulates a single photo, video or sound from the device. It inherits from File [[!FILE-API]].

void getFormatData (in MediaFileDataSuccessCallback successCallback, in optional MediaFileDataErrorCallback errorCallback)
The getFormatData() method takes one or two arguments. When called, it returns immediately and then asynchronously attempts to obtain the format data of the given media file. If the attempt is successful, the successCallback is invoked with a new MediaFileData object, reflecting the format data of the file. If the attempt fails, the errorCallback is invoked with a new MediaFileDataError object, reflecting the reason for the failure.

MediaFileDataSuccessCallback interface

void onSuccess()
MediaFileData formatData
The MediaFileData object describing the relevant properties of the given media file.

MediaFileDataErrorCallback interface

void onError()
MediaFileDataError error
The MediaFileDataError object describing the error encountered while retrieving the format data.

MediaFileDataError interface

The MediaFileDataError interface encapsulates all errors in the retrieval of format data associated with a MediaFile object.

const unsigned short UNKNOWN_ERROR = 0
An unknown error occurred.
const unsigned short TIMEOUT_ERROR = 1
The requested method timed out before it could be completed.
readonly attribute unsigned short code
An error code assigned by an implementation when an error has occurred in retrieving format data.

User Interface Examples

A media capture file picker might render as:

A File picker with camera support