Copyright © @@@@ W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark and document use rules apply.
This document specifies use cases and requirements as an input for the development of the "Ontology for Media Resource 1.0" and the "API for Media Resource 1.0". The ontology will be a simple ontology to support cross-community data integration of information related to media resources on the Web. The API will provide read access and potentially write access to media resources, relying on the definitions from the ontology.
The main scope of this document are videos. Metadata for other media resources like audio or images will be taken into account if it is applicable for videos as well.
1 Introduction
2 Purpose of this draft publication
3 Purpose of the Ontology and the API
4 Terminology
5 Use Cases
5.1 Interoperability between media resources across Cultural Heritage Institutions
5.2 Recommendation across different media types
5.3 Life Log
5.4 Access via web client to metadata in heterogeneous formats
5.5 User generated Metadata
5.6 Use cases: to be done
6 Requirements
6.1 Requirement r01: Providing methods for getting metadata information stored in different formats
6.2 Requirement r02: Providing methods for setting metadata information stored in different formats
6.3 Requirement r03: Providing in the API a means for supporting structured annotations
6.4 Requirement r04: Providing a means to access user-defined metadata
6.5 Requirement r05: Providing the ontology as a simple set of properties
6.6 Requirement r06: Specifying an internal or external format for the ontology
6.7 Requirement r07: Introducing several abstraction levels in the ontology
6.8 Requirement r08: Being able to apply the ontology / API for collections of metadata
6.9 Requirement r09: Supporting the provenance information of metadata properties
6.10 Requirement r10: Being able to describe fragments of media resources
6.11 Requirement r11: Providing the ontology in slices of conformance
6.12 Requirement r12: Providing support for controlled vocabularies for the values of different properties
6.13 Requirement r13: Allowing for different return types for the same property
6.14 Requirement r14: Providing support for policy information
6.15 Requirement r15: Providing support for discovery of named and track fragments
A References
B References (Non-Normative)
C Change Log (Non-Normative)
D Acknowledgements (Non-Normative)
Anticipating the increase in online video and audio in the upcoming years, we can foresee that it will become progressively more difficult for viewers to find the content using current search tools. In addition, video services on the web that allow for upload of video, need to display selected information about the media documents which could be facilitated by a uniform access to selected metadata across a variety of file formats.
Unlike hypertext documents, it is more complex and sometimes impossible to deduce meta information about a medium, such as its title, author, or creation date from its content. There has been a proliferation of media metadata formats for the document's authors to express this metadata information. For example, an image could potentially contain EXIF, IPTC and XMP information. There are also several metadata solutions for media related content, including MPEG-7, Yahoo! MEDIA RSS, Google Videositemaps, VODCSV, TVAnytime and EBU P-Meta. Many of these formats have been extensively discussed in the deliverables XGR Vocabularies and XGR Image Annotation of the W3C Multimedia Semantics Incubator Group , which provide a major input to this Working Group.
The "Ontology for Media Resource 1.0" will address the intercompatiblity problem by providing a common set of properties to define the basic metadata needed for media resources and the semantic links between their values in different existing vocabularies. It will help circumventing the current proliferation of video metadata formats by providing full or partial translation and mapping from properties in formats to a common set of properties in the ontology. The ontology will be accompanied by an API that provides uniform access to all elements defined by the ontology.
This document specifies the use cases and requirements that are motivating the development of the "Ontology for Media Resource 1.0". The scope is mainly video media resources, but we take also other media resources into account if their metadata information is related to video.
The development of the requirements has three major inputs: Use cases, analysis of existing standards, and a description of canonical media processes.
This initial version of this document contains only a small set of use cases and requirements. Nevertheless it is being published to gather wide feedback on the general direction of the Working Group. Hence, we would like to encourage especially feedback on 6 Requirements, the requirements which we are planning to implement, or others which we are planning not to take into account.
Currently, there is an additional section under development, describing a top-down modeling approach to describe the media annotation problem. The Working Group is considering to publish that section in an updated version of this document.
The following figure visualizes the purpose of the ontology, the purpose of the API and their relation to applications.
The ontology will define mappings from properties in formats to a common set of properties. The API then will define methods to access heterogeneous metadata, using such mappings. An example: the property createDate
from XMP XMP can be mapped to the property DateCreated
from IPTC IPTC.
An important aspect of the above figure is that everything visualized above the API is left to applications. For example.
languages for simple or complex queries
analysis of user preferences (like "preferring movies with actor X and suitable for children")
other mechanisms for accessing metadata
The ontology and the API provide merely a basic, simple means of interoperability for such applications.
The keywords MUST, MUST NOT, SHOULD and SHOULD NOT are to be interpreted as defined in RFC 2119.
Summary: Accessing media collections of different cultural heritage institutions (libraries, museums, archives, etc.) on the Web.
Related requirements:
Requirement r03: Providing in the API a means for supporting structured annotations
Requirement r04: Providing a means to access user-defined metadata
Requirement r07: Introducing several abstraction levels in the ontology
Requirement r08: Being able to apply the ontology / API for collections of metadata
Requirement r09: Supporting the provenance information of metadata properties
Requirement r10: Being able to describe fragments of media resources
Description / Example:
The collections of cultural heritage institutions (libraries, museums, archives, etc.) are increasingly digitised and made available on the Web. These collections range from text to image, video and audio (music and radio collections, for example). A comprehensive, professionally created documentation is usually available, however, often using domain specific or even proprietary metadata models. This hinders accessing these collections in an homogeneous or centralized way and linking them across collections.
For example, Jane is a TV journalist searching for material about some event in contemporary history. She is interested in television and radio broadcasts from this event, along with photos and newspaper articles. All these resources come from different collections, and some are in different languages. A homogeneous way of accessing them across the Web would improve her work.
Summary: Accessing heterogeneous media resources metadata as the input to the creation of recommendations which is based on user preferences.
Related requirements:
Requirement r01: Providing methods for getting metadata information stored in different formats
Requirement r03: Providing in the API a means for supporting structured annotations
Requirement r05: Providing the ontology as a simple set of properties
Requirement r15: Providing support for discovery of named and track fragments
Description / Example:
People nowadays are able to enjoy large number of programs from different content providers (broadcasting companies, Internet video website, etc.). To achieve better user experience, reduce the user's experience of being overloaded, and hence retain users, some systems provide recommendations based on the user's history, ratings, or stated preferences. However, different content providers usually have their specific or proprietary metadata models, which is one of the key problems faced by recommendation service providers. A common ontology spanning different metadata sets can allow recommendation systems to return a better, larger, and more relevant selection than when the metadata systems are unrelated.
Company A is an IPTV add-value service provider. One of their services is to recommend programs that users might like, based on their watching history or explicit rating of programs. In this system, users are able to watch regular TV programs with electronic program guide (EPG) format metadata, videos such as from YouTube, with website-specific metadata, etc. In order to perform uniform and effective recommendation in the absence of a common set of vocabularies, they would need to design own integrated media annotation model.
Use case summary: combining heterogeneous metadata from life logs, to allow searching personal life log information, potentially enriched with geolocation information.
Related requirements:
Requirement r01: Providing methods for getting metadata information stored in different formats
Requirement r05: Providing the ontology as a simple set of properties
Requirement r15: Providing support for discovery of named and track fragments
Description / Example:
With modern devices, a person can capture his or her experience, including all sorts of daily events, by creating images, audios and videos files, and publish them on the Web. These are called "Life Logs". These Life Logs contain various information such as time, location, creator's profile, relations between different people, and even emotion. If accessed via an ontology providing links between the different metadata used to describe these various information, a user could easily and efficiently search for his or her personal Life Log information, including emotional information ( this type of information can be described using a vocabulary like Emotions ML 1.0), or geolocation information on the Web (which can be described using the Geolocation API specification). Other people's Life Logs contents could also be searched and accessed via this ontology.
Use case summary: Accessing metadata in heterogeneous formats for web developers
Related requirements:
Requirement r01: Providing methods for getting metadata information stored in different formats
Requirement r05: Providing the ontology as a simple set of properties
Requirement r15: Providing support for discovery of named and track fragments
Description / Example:
John is developing a JavaScript library for accessing metadata of media resources (e.g. video) in various formats. These resources are available within a database, such as that of a search engine indexing the internet or other web-accessible content (e.g. a corporate repository, library, etc.). His library can be used to make queries of the media resources like:
"Find me all media resources which have been created by a specified person"
"Find me all media resources which have been created this year"
"Find me all videos which are not longer than a specified time"
"Extract all user added tags from all media resources available"
This use case is related to many other use cases. Nevertheless it is mentioned separately since, at the difference from other requirements, its implementation requires only a small set of requirements. The difference from this use case to the Cultural Heritage use case is that the former is very strongly tied to the requirement of a read-only client side API.
Use case summary: Adding or linking to external metadata by different users.
Related requirements:
Requirement r01: Providing methods for getting metadata information stored in different formats
Requirement r02: Providing methods for setting metadata information stored in different formats
Requirement r04: Providing a means to access user-defined metadata
Description / Example:
John wants to publish comments on the last movies he has seen on http://example.cheap-vod.com/ . For each movie, he uses the description metadata field to provide a personal summary of the movie (with incentive to see or avoid the movie according to his own opinions), and the ranking metadata. John is also not satisfied with the genre classification of the website, so he uses the genre metadata field to provide his appreciation of the genre with regard to a better scheme. He then publishes these metadata on his blog (may be in the form of a podcast), but only links to the videos themselves.
Jane, a friend of John's and another cheap-vod customer, can now configure her cheap-vod account or her browser, to have John's metadata added to or replacing the original metadata embedded in each file.
Now Jane wants to study more particularly the characters of the movie. For making this easier, she defines one custom metadata field for each of the main characters, and sets these fields to "yes" or "no" for each sequence, to indicate if they contain that character or not. For example:
<http://example.library.myschool.edu/rose.ogv#some_fragment_identifier> dc:title "Meeting Tom Baxter" ; dc:description "Cecilia sees the movie several times when...." ; custom:cecilia "yes" ; custom:tom "yes" ; custom:gil "no" ; custom:monk "no".
In this context, the ontology would enhance the interoperability between different users.
This sections describes requirements for the ontology and the API. The Working Group has agreed to implement the following requirements. For the other requirements, there is no agreement yet, and the Working Group is asking reviewers of this document for feedback about their implementation.
6.1 Requirement r01: Providing methods for getting metadata information stored in different formats
6.3 Requirement r03: Providing in the API a means for supporting structured annotations
6.5 Requirement r05: Providing the ontology as a simple set of properties
6.10 Requirement r10: Being able to describe fragments of media resources
6.11 Requirement r11: Providing the ontology in slices of conformance
The requirements which the Working Group currently does not have agreement to take into account are the following:
6.2 Requirement r02: Providing methods for setting metadata information stored in different formats
6.4 Requirement r04: Providing a means to access user-defined metadata
6.6 Requirement r06: Specifying an internal or external format for the ontology
6.7 Requirement r07: Introducing several abstraction levels in the ontology
6.8 Requirement r08: Being able to apply the ontology / API for collections of metadata
6.9 Requirement r09: Supporting the provenance information of metadata properties
6.13 Requirement r13: Allowing for different return types for the same property
6.14 Requirement r14: Providing support for policy information
6.15 Requirement r15: Providing support for discovery of named and track fragments
Description: The API MUST provide methods for getting metadata stored in different formats related to media resources. Metadata can be in one of different formats, either as separate document or embedded in media resources.
Rationale: This is a core requirements. Its implementation is necessary for nearly all use cases.
Target (API and / or ontology): API
Description: The API MUST provide methods for setting metadata stored in different formats related to media resources. Metadata can be in one of different formats, either as separate document or embedded in media resources.
Rationale: The implementation of this requirement is mainly necessary for use cases which involve change of media resources by users.
Target (API and / or ontology): API
Note:
The implementation of this requirement may impose several problems, like: how to set information in formats which have more detailed information than our ontology, or how to implement the setting process in the API (e.g. what protocol to use). Due to such problems and since there seem to be no implementations achieving this functionality, we might not take this requirement into account.
Description: The API MUST provide a means to support structured metadata to media resources, like the name of the creator being structured in "first name" and "last name".
Rationale: There are existing, widely used formats like XMP which are defined in a structured manner. To be able to support meta information for media resources, including such formats, the API needs to have a means to achieve this.
Target (API and / or ontology): API
Description: It MUST be possible to access user-defined metadata to media resources. "user-defined metadata" means metadata that is not defined in a standardized format, but which is being created entirely by the user.
Rationale: The ability to access user-defined metadata is necessary for the use case user generated metadata.
Target (API and / or ontology): API which needs to provide a method to add user-defined metadata, and the ontology which needs to provide an extensibility mechanism.
Note:
"Accessing user-defined metadata" may mean setting or getting such metadata. We have not decided whether we will be able to support the process of setting metadata, see issues mentioned at Requirement r02: Providing methods for setting metadata in media resources in different formats.
Description: the ontology MUST be available as a simple set of properties, to hide complexity for whose who do not need it.
Rationale: In use cases like access via web client to metadata in heterogeneous formats it is important to hide the potentially complex ontology from the web developer. This will foster ease of use and wide spread adoption.
Target (API and / or ontology): API and ontology
Description: The ontology MUST be provided not only in prose description but also as an internal or external format.
Rationale: to be able foster interoperability between applications, a common format for the ontology will be helpful. To avoid the need to process this format for all implementations, the specification(s) will provide separate slices of conformance, see Requirement r11: providing the ontology in slices of conformance.
Target (API and / or ontology): Mainly the ontology, but possibly also the API, if we require it to process this format.
Description: The ontology MUST provide several abstraction levels.
Rationale: Several metadata standards like FRBR or CIDOC allow referring to multimedia resources on several abstraction levels, in order to separate e.g. a movie, a DVD which contains the movie and a specific copy of the DVD. Especially for collections of multimedia resources, knowledge about such abstraction levels is helpful, as a means for accessing the resources on each level.
Target (API and / or ontology): ontology and potentially API, if we want to provide access to metadata and multimedia resources on several abstraction levels.
Description: It MUST be possible to access collections of metadata.
Rationale: For processing collections of multimedia resources, access to collections of metadata referring potentially to more than one resource is necessary. As an example for the need for this requirement and a related requirement see Requirement r07: Introducing several abstraction levels in the ontology.
Target (API and / or ontology): API and ontology
Description: The ontology MUST support provenance information of metadata properties.
Rationale: Metadata is being dealt with by for example producers of metadata (e.g. a video camera), changers (e.g. a person which modifies initially created metadata) and consumers (e.g. an application which processes metadata to make it accessible for search). If several pieces of metadata, created by machines or people in different roles, are in conflict (e.g. contradictory creation dates), a description of provenance (i.e. roles of the metadata creators) can be useful for conflict resolution (e.g. "metadata produced by the changer has precedence over metadata produced by the creator").
Target (API and / or ontology): ontology
Description: It MUST be possible to relate metadata to fragments of media resources.
Rationale: Processes like search may be specific to fragments of media resources, e.g. a search for all kiss scenes in a movie. The implementation of this requirement provides the means to implement such processes.
Note:
This requirement will be implemented by the Media Fragments Working Group.
Target (API and / or ontology): none of these
Description: The ontology MUST be provided in a prose description and MAY be provided in different serializations (RDF, XML). The yet to be produced general conformance description MUST require implementations to take the prose description into account. Additional conformance descriptions, being specific to a serialization, MAY be provided.
Rationale: Existing metadata formats use a wide range of serializations like RDF and XML. To foster a widespread adoption of the ontology, we do not want to be specific to one serialization, but rather state that following the prose description is sufficient for an implementation. If there is a interest in the Working Group to create one or more serializations, we may provide additional types of conformance for them.
Target (API and / or ontology): ontology
Description: It MUST be possible to take information from controlled vocabularies for certain properties into account.
Rationale: Media archives often make use of controlled vocabularies (e.g. classifications, thesauri, ontologies) for certain properties. Providing access to knowledge about which vocabulary is actually being in use for a media resource, is an important requirement for such archives.
Target (API and / or ontology): ontology (for describing properties which need a slot for specifying a controlled vocabulary) and the API ( for getting information about which vocabulary is being used for a media resource)
Description: It MUST be possible to provide different return types for the same property.
Rationale: Some properties are defined with the same name and functionality (e.g. conveying information about the creator of a media resource), but use different value types (e.g. string
versus URI
). This raises the question whether the API should be specific to only one return type, or allow for undefined/unformatted return values for the same property.
Target (API and / or ontology): API
Description: The ontology MUST provide support for linking policy information related to the media resource.
Rationale: Specific types of policy information are license, rights and access. If an implementation supports policy information, the set of properties must include a link to policy description, represented using e.g. ODRL or P3P, in order to express the binding of the policy information to the media resource being described. Ideally, the link to the policy information should be embedded in the media resource.
Source: Input from PLING IG
Target (API and / or ontology): Ontology and API
Description: The ontology MUST provide properties to query the list tracks that exist in a media resource as well as the list of named fragments.
Rationale: The Media Fragments WG is interested in technologies that enable track and named fragments discovery, i.e., the UA needs a way to know for which tracks are available in a particular media resource, and which named fragments have been annotated.
Source: Input from Media Fragments WG
Target (API and / or ontology): Ontology and API
Date | Change |
2009-01-19 | Initial publication. |
2009-03-16 | Integrated comments from the Media Fragments Working Group, and Raphaël Troncy. See editing summary. |
2009-03-16 | Editing of the Cultural Heritage Institutions use case. |
2009-03-19 | Integrated comments from Jean-Pierre Evain. |
2009-04-02 | Removed the mobile use case. |
2009-04-29 | Integrated comments from David Singer, except the "More structural comments". |
2009-04-29 | Added a health warning to the status section about ongoing terminology discussions. |
2009-12-24 | Added r14 and r15 to 6 Requirements. |
2009-12-24 | Revised r09 with concrete description. |
2009-12-24 | Changed the term of Media Object with Media Resource. |
This document is the work of the W3C Media Annotations Working Group.
Members of the Working Group are (at the time of writing, and by alphabetical order): Werner Bailer (JOANNEUM RESEARCH), Tobias Bürger (University of Innsbruck), Eric Carlson (Apple, Inc.), Pierre-Antoine Champin ((public) Invited expert), Ashish Chawla ((public) Invited expert), Jaime Delgado (Universitat Politècnica de Catalunya), Jean-Pierre EVAIN ((public) Invited expert), Philip Jägenstedt (Opera Software), Ralf Klamma ((public) Invited expert), WonSuk Lee (Electronics and Telecommunications Research Institute (ETRI)), Véronique Malaisé (Vrije Universiteit), Erik Mannens (IBBT), Hui Miao (Samsung Electronics Co., Ltd.), Thierry Michel (W3C/ERCIM), Frank Nack (University of Amsterdam), Soohong Daniel Park (Samsung Electronics Co., Ltd.), Silvia Pfeiffer (W3C Invited Experts), Chris Poppe (IBBT), Víctor Rodríguez (Universitat Politècnica de Catalunya), Felix Sasaki (Potsdam University of Applied Sciences), David Singer (Apple, Inc.), Florian Stegmaier ((public) Invited expert), John Strassner ((public) Invited expert), Joakim Söderberg (ERICSSON), Thai Wey Then (Apple, Inc.), Ruben Tous (Universitat Politècnica de Catalunya), Raphaël Troncy (CWI), Vassilis Tzouvaras (K-Space), Davy Van Deursen (IBBT).
The people who have contributed to discussions on public-media-annotation@w3.org are also gratefully acknowledged.