Web Services Architecture

1.4.1 Agents and Services

A Web service is an abstract notion that must be implemented by a concrete agent. (See Figure 1-1) The agent is the concrete piece of software or hardware that sends and receives messages, while the service is the resource characterized by the abstract set of functionality that is provided. To illustrate this distinction, you might implement a particular Web service using one agent one day (perhaps written in one programming language), and a different agent the next day (perhaps written in a different programming language) with the same functionality. Although the agent may have changed, the Web service remains the same.

1.4.2 Requesters and Providers

The purpose of a Web service is to provide some functionality on behalf of its owner -- a person or organization, such as a business or an individual. The provider entity is the person or organization that provides an appropriate agent to implement a particular service. (See Figure 1-1: Basic Architectural Roles.)

A requester entity is a person or organization that wishes to make use of a provider entity's Web service. It will use a requester agent to exchange messages with the provider entity's provider agent.

(In most cases, the requester agent is the one to initiate this message exchange, though not always. Nonetheless, for consistency we still use the term "requester agent" for the agent that interacts with the provider agent, even in cases when the provider agent actually initiates the exchange.)

In order for this message exchange to be successful, the requester entity and the provider entity must first agree on both the semantics and the mechanics of the message exchange. (This is a slight simplification that will be explained further in 3.3 Using Web Services.)

1.4.3 Service Description

The mechanics of the message exchange are documented in a Web service description (WSD). (See Figure 1-1) The WSD is a machine-processable specification of the Web service's interface, written in WSDL. It defines the message formats, datatypes, transport protocols, and transport serialization formats that should be used between the requester agent and the provider agent. It also specifies one or more network locations at which a provider agent can be invoked, and may provide some information about the message exchange pattern that is expected. In essence, the service description represents an agreement governing the mechanics of interacting with that service. (Again this is a slight simplification that will be explained further in 3.3 Using Web Services.)

1.4.4 Semantics

The semantics of a Web service is the shared expectation about the behavior of the service, in particular in response to messages that are sent to it. In effect, this is the "contract" between the requester entity and the provider entity regarding the purpose and consequences of the interaction. Although this contract represents the overall agreement between the requester entity and the provider entity on how and why their respective agents will interact, it is not necessarily written or explicitly negotiated. It may be explicit or implicit, oral or written, machine processable or human oriented, and it may be a legal agreement or an informal (non-legal) agreement. (Once again this is a slight simplification that will be explained further in 3.3 Using Web Services.)

While the service description represents a contract governing the mechanics of interacting with a particular service, the semantics represents a contract governing the meaning and purpose of that interaction. The dividing line between these two is not necessarily rigid. As more semantically rich languages are used to describe the mechanics of the interaction, more of the essential information may migrate from the informal semantics to the service description. As this migration occurs, more of the work required to achieve successful interaction can be automated.

1.4.5 Overview of Engaging a Web Service

There are many ways that a requester entity might engage and use a Web service. In general, the following broad steps are required, as illustrated in Figure 1-1: (1) the requester and provider entities become known to each other (or at least one becomes know to the other); (2) the requester and provider entities somehow agree on the service description and semantics that will govern the interaction between the requester and provider agents; (3) the service description and semantics are realized by the requester and provider agents; and (4) the requester and provider agents exchange messages, thus performing some task on behalf of the requester and provider entities. (I.e., the exchange of messages with the provider agent represents the concrete manifestation of interacting with the provider entity's Web service.) These steps are explained in more detail in 3.4 Web Service Discovery. Some of these steps may be automated, others may be performed manually.

Figure 1-1. The General Process of Engaging a Web Service

2.2.1 Concepts

A concept is expected to have some correspondence with any realizations of the architecture. For example, the message concept identifies a class of object (not to be confused with Objects and Classes as are found in Object Oriented Programming languages) that we expect to be able to identify in any Web services context. The precise form of a message may be different in different realizations, but the message concept tells us what to look for in a given concrete system rather than prescribing its precise form.

Not all concepts will have a realization in terms of data objects or structures occurring in computers or communications devices; for example the person or organization refers to people and human organizations. Other concepts are more abstract still; for example, message reliability denotes a property of the message transport service — a property that cannot be touched but nonetheless is important to Web services.

Each concept is presented in a regular, stylized way consisting of a short definition, an enumeration of the relationships with other concepts, and a slightly longer explanatory description. For example, the concept of agent includes as relating concepts the fact that an agent is a computational resource, has an identifier and an owner. The description part of the agent explains in more detail why agents are important to the archicture.

2.2.2 Relationships

Relationships denote associations between concepts. Grammatically, relationships are verbs; or more accurately, predicates. A statement of a relationship typically takes the form: concept predicate concept. For example, in agent, we state that:

An agent is

a computational resource

This statement makes an assertion, in this case about the nature of agents. Many such statements are descriptive, others are definitive:

A message has

a message sender

Such a statement makes an assertion about valid instances of the architecture: we expect to be able to identify the message sender in any realization of the architecture. Conversely, any system for which we cannot identify the sender of a message is not conformant to the architecture. Even if a service is used anonymously, the sender has an identifier but it is not possible to associate this identifier with an actual person or organization.

2.2.3 Concept Maps

Many of the concepts in the architecture are illustrated with concept maps. A concept map is an informal, graphical way to illustrate key concepts and relationships. For example the diagram:

Figure 2-1. Concept Map

shows three concepts which are related in various ways. Each box represents a concept, and each arrow (or labeled arc) represents a relationship.

The merit of a concept map is that it allows rapid navigation of the key concepts and illustrates how they relate to each other. It should be stressed however that these diagrams are primarily navigational aids; the written text is the definitive source.

2.2.4 Model

A model is a coherent subset of the architecture that typically revolves around a particular aspect of the overall architecture. Although different models share concepts, it is usually from different points of view; the major role of a model is to explain and encapsulate a significant theme within the overall Web services architecture.

For example, the Message Oriented Model focuses and explains Web services strictly from a message passing perspective. In particular, it does not attempt to relate messages to services provided. The Service Oriented Model, however, lays on top of and extends the Message Oriented Model in order to explain the fundamental concepts involved in service - in effect to explain the purpose of the messages in the Message Oriented Model.

Each model is described separately below, in terms of the concepts and relationships inherent to the model. The ordering of the concepts in each model section is alphabetical; this should not be understood to imply any relative importance. For a more focused viewpoint the reader is directed to the Stakeholder's perspectives section which examines the architecture from the perspective of key stakeholders of the architecture.

The reason for choosing an alphabetical ordering is that there is a large amount of cross-referencing between the concepts. As a result, it is very difficult, if not misleading, to choose a non-alphabetic ordering that reflects some sense of priority between the concepts. Furthermore, the optimal ordering depends very much on the point of view of the reader. Hence, we devote the Stakeholders perspectives section to a number of prioriterized readings of the architecture.

2.2.5 Conformance

Unlike language specifications, or protocol specifications, conformance to an architecture is necessarily a somewhat imprecise art. However, the presence of a concept in this enumeration is a strong hint that, in any realization of the architecture, there should be a corresponding feature in the implementation. Furthermore, if a relationship is identified here, then there should be corresponding relationships in any realized architecture. The consequence of non-conformance is likely to be reduced interoperability: The absence of such a concrete feature may not prevent interoperability, but it is likely to make such interoperability more difficult.

A primary function of the Architecture's enumeration in terms of models, concepts and relationships is to give guidance about conformance to the architecture. For example, the architecture notes that a message has a message sender; any realization of this architecture that does not permit a message to be associated with its sender is not in conformance with the architecture. For example, SMTP could be used to transmit messages. However, since SMTP (at present) allows forgery of the sender's identity, SMTP by itself is not sufficient to discharge this responsibility.

2.3.1 Message Oriented Model

The Message Oriented Model focuses on those aspects of the architecture that relate to messages and the processing of them. Specifically, in this model, we are not concerned with any semantic significance of the content of a message or its relationship to other messages. However, the MOM does focus on the structure of messages, on the relationship between message senders and receivers and how messages are transmitted.

The MOM is illustrated in the Figure 2-7:

Figure 2-7. Message Oriented Model

2.3.2 The Service Oriented Model

The Service Oriented Model (SOM) focuses on those aspects of the architecture that relate to service and action.

The primary purpose of the SOM is to explicate the relationships between an agent and the services it provides and requests. The SOM builds on the MOM, but its focus is on action rather than message.

The concepts and relationships in the SOM are illustrated in Figure 2-8:

Figure 2-8. Service Oriented Model

2.3.3 The Resource Oriented Model

The Resource Oriented Model focuses on those aspects of the architecture that relate to resources. Resources are a fundamental concept that underpins much of the Web and much of Web services; for example, a Web service is a particular kind of resource that is important to this architecture.

The ROM focuses on the key features of resources that are relevant to the concept of resource, independent of the role the resource has in the context of Web services. Thus we focus on issues such as the ownership of resources, policies associated with resources and so on. Then, by virtue of the fact that Web services are resources, these properties are inherited by Web services.

We illustrate the basic concepts and relationships in the ROM in Figure 2-9:

Figure 2-9. Resource Oriented Model

2.3.4 The Policy Model

The Policy Model focuses on those aspects of the architecture that relate to policies and, by extension, security and quality of service.

Security is fundamentally about constraints; about constraints on the behavior on action and on accessing resources. Similarly, quality of service is also about constraints on service. In the PM, these constraints are modeled around the core concept of policy; and the relationships with other elements of the architecture. Thus the PM is a framework in which security can be realized.

However, there are many other kinds of constraints, and policies that are relevant to Web services, including various application-level constraints.

The concepts and relationships in the PM are illustrated in Figure 2-10:

Figure 2-10. Policy Model

2.4.1 The is a relationship

2.4.2 The describes relationship

2.4.3 The has a relationship

2.4.4 The owns relationship

2.4.5 The realized relationship

3.1.1 Distributed Systems

A distributed system consists of diverse, discrete software agents that must work together to perform some tasks. Furthermore, the agents in a distributed system do not operate in the same processing environment, so they must communicate by hardware/software protocol stacks over a network. This means that communications with a distributed system are intrinsically less fast and reliable than those using direct code invocation and shared memory. This has important architectural implications because distributed systems require that developers (of infrastructure and applications) consider the unpredictable latency of remote access, and take into account issues of concurrency and the possibility of partial failure [Dist Comp].

Distributed object systems are distributed systems in which the semantics of object initialization and method invocation are exposed to remote systems by means of a proprietary or standardized mechanism to broker requests across system boundaries, marshall and unmarshall method argument data, etc. Distributed objects systems typically (albeit not necessarily) are characterized by objects maintaining a fairly complex internal state required to support their methods, a fine grained or "chatty" interaction between an object and a program using it, and a focus on a shared implementation type system and interface hierarchy between the object and the program that uses it.

A Service Oriented Architecture (SOA) is a form of distributed systems architecture that is typically characterized by the following properties:

• Logical view: The service is an abstracted, logical view of actual programs, databases, business processes, etc., defined in terms of what it does, typically carrying out a business-level operation.

• Message orientation: The service is formally defined in terms of the messages exchanged between provider agents and requester agents, and not the properties of the agents themselves. The internal structure of an agent, including features such as its implementation language, process structure and even database structure, are deliberately abstracted away in the SOA: using the SOA discipline one does not and should not need to know how an agent implementing a service is constructed. A key benefit of this concerns so-called legacy systems. By avoiding any knowledge of the internal structure of an agent, one can incorporate any software component or application that can be "wrapped" in message handling code that allows it to adhere to the formal service definition.

• Description orientation: A service is described by machine-processable meta data. The description supports the public nature of the SOA: only those details that are exposed to the public and important for the use of the service should be included in the description. The semantics of a service should be documented, either directly or indirectly, by its description.

• Granularity: Services tend to use a small number of operations with relatively large and complex messages.

• Network orientation: Services tend to be oriented toward use over a network, though this is not an absolute requirement.

• Platform neutral: Messages are sent in a platform-neutral, standardized format delivered through the interfaces. XML is the most obvious format that meets this constraint.

3.1.2 Web Services and Architectural Styles

Distributed object systems have a number of architectural challenges. [Dist Comp] and others point out:

• Problems introduced by latency and unreliability of the underlying transport.

• The lack of shared memory between the caller and object.

• The numerous problems introduced by partial failure scenarios.

• The challenges of concurrent access to remote resources.

• The fragility of distributed systems if incompatible updates are introduced to any participant.

These challenges apply irrespective of whether the distributed object system is implemented using COM/CORBA or Web services technologies. Web services are no less appropriate than the alternatives if the fundamental criteria for successful distributed object architectures are met. If these criteria are met, Web services technologies may be appropriate if the benefits they offer in terms of platform/vendor neutrality offset the performance and implementation immaturity issues they may introduce.

Conversely, using Web services technologies to implement a distributed system doesn't magically turn a distributed object architecture into an SOA. Nor are Web services technologies necessarily the best choice for implementing SOAs -- if the necessary infrastructure and expertise are in place to use COM or CORBA as the implementation technology and there is no requirement for platform neutrality, using SOAP/WSDL may not add enough benefits to justify their costs in performance, etc.

In general SOA and Web services are most appropriate for applications:

• That must operate over the Internet where reliability and speed cannot be guaranteed;

• Where there is no ability to manage deployment so that all requesters and providers are upgraded at once;

• Where components of the distributed system run on different platforms and vendor products;

• Where an existing application needs to be exposed for use over a network, and can be wrapped as a Web service.

3.1.3 Relationship to the World Wide Web and REST Architectures

The World Wide Web operates as a networked information system that imposes several constraints: Agents identify objects in the system, called resources, with Uniform Resource Identifiers (URIs). Agents represent, describe, and communicate resource state via representations of the resource in a variety of widely-understood data formats (e.g. XML, HTML, CSS, JPEG, PNG). Agents exchange representations via protocols that use URIs to identify and directly or indirectly address the agents and resources. [Web Arch]

An even more constrained architectural style for reliable Web applications known as Representation State Transfer (REST) has been proposed by Roy Fielding and has inspired both the W3C Technical Architecture Group's architecture document [Web Arch] and many who see it as a model for how to build Web services [Fielding]. The REST Web is the subset of the WWW (based on HTTP) in which agents provide uniform interface semantics -- essentially create, retrieve, update and delete -- rather than arbitrary or application-specific interfaces, and manipulate resources only by the exchange of representations. Furthermore, the REST interactions are "stateless" in the sense that the meaning of a message does not depend on the state of the conversation.

We can identify two major classes of Web services:

• REST-compliant Web services, in which the primary purpose of the service is to manipulate XML representations of Web resources using a uniform set of "stateless" operations; and

• arbitrary Web services, in which the service may expose an arbitrary set of operations.

Both classes of Web services use URIs to identify resources and use Web protocols (such as HTTP and SOAP 1.2) and XML data formats for messaging. (It should be noted that SOAP 1.2 can be used in a manner consistent with REST. However, SOAP 1.2 can also be used in a manner that is not consistent with REST.)

3.2.1 XML

XML solves a key technology requirement that appears in many places. By offering a standard, flexible and inherently extensible data format, XML significantly reduces the burden of deploying the many technologies needed to ensure the success of Web services.

The important aspects of XML, for the purposes of this Architecture, are the core syntax itself, the concepts of the XML Infoset [XML Infoset], XML Schema and XML Namespaces.

XML Infoset is not a data format per se, but a formal set of information items and their associated properties that comprise an abstract description of an XML document [XML 1.0]. The XML Infoset specification provides for a consistent and rigorous set of definitions for use in other specifications that need to refer to the information in a well-formed XML document.

Serialization of the XML Infoset definitions of information may be expressed using XML 1.0 [XML 1.0]. However, this is not an inherent requirement of the architecture. The flexibility in choice of serialization format(s) allows for broader interoperability between agents in the system. In the future, a binary encoding of the XML infoset may be a suitable replacement for the textual serialization. Such a binary encoding may be more efficient and more suitable for machine-to-machine interactions.

3.2.2 SOAP

SOAP 1.2 provides a standard, extensible, composable framework for packaging and exchanging XML messages. In the context of this architecture, SOAP 1.2 also provides a convenient mechanism for referencing capabilities (typically by use of headers).

[SOAP 1.2 Part 1] defines an XML-based messaging framework: a processing model and an exensibility model. SOAP messages can be carried by a variety of network protocols; such as HTTP, SMTP, FTP, RMI/IIOP, or a proprietary messaging protocol.

[SOAP 1.2 Part 2] defines three optional components: a set of encoding rules for expressing instances of application-defined data types, a convention for representing remote procedure calls (RPC) and responses, and a set of rules for using SOAP with HTTP/1.1.

While SOAP Version 1.2 [SOAP 1.2 Part 1] doesn't define "SOAP" as an acronym anymore, there are two expansions of the term that reflect these different ways in which the technology can be interpreted:

1. Service Oriented Architecture Protocol: In the general case, a SOAP message represents the information needed to invoke a service or reflect the results of a service invocation, and contains the information specified in the service interface definition.

2. Simple Object Access Protocol: When using the optional SOAP RPC Representation, a SOAP message represents a method invocation on a remote object, and the serialization of in the argument list of that method that must be moved from the local environment to the remote environment.

3.2.3 WSDL

WSDL 2.0[WSDL 2.0 Part 1] is a language for describing Web services.

WSDL describes Web services starting with the messages that are exchanged between the requester and provider agents. The messages themselves are described abstractly and then bound to a concrete network protocol and message format.

Web service definitions can be mapped to any implementation language, platform, object model, or messaging system. Simple extensions to existing Internet infrastructure can implement Web services for interaction via browsers or directly within an application. The application could be implemented using COM, JMS, CORBA, COBOL, or any number of proprietary integration solutions. As long as both the sender and receiver agree on the service description, (e.g. WSDL file), the implementations behind the Web services can be anything.

3.4.1 Manual Versus Autonomous Discovery

The discovery process described above is not specific about who or what within the requester entity actually performs the discovery. Under manual discovery, a requester human uses a discovery service (typically at design time) to locate and select a service description that meets the desired functional and other criteria. Under autonomous discovery, the requester agent performs this task, either at design time or run time. Although the steps are similar in either case, the constraints and needs are significantly different, such as:

• Interface requirements. The requirements for something that is intended for human interaction are very different from the requirements for something that is intended for machine interaction.

• Need for standardization. There is far less need to standardize an interface or protocol that humans use than those that machines are intended to use.

• Trust. People do not necessarily trust machines to make decisions that may have significant consequences. This is explained more fully in 3.6.4.5 Trust and Discovery.

In the case of autonomous discovery, the need for machine-processable semantics is greatly increased.

One situation in which autonomous discovery is often needed is when the requester agent has been interacting with a particular provider agent, but for some reason needs to refresh its choice of provider agent, either because the previous provider agent is no longer available, or other reasons.

3.4.2 Discovery: Registry, Index or Peer-to-Peer?

At present, there are three leading viewpoints on how a discovery service should be conceived: as a registry, as an index, or as a peer-to-peer system. What are the differences? For what purpose is one better than the other?

3.4.3 Federated Discovery Services

Although the registry viewpoint is a more centralized approach to discovery than the index approach, there will arise situations where multiple registries exist on the Web. It is expected that multiple indexes will also exist. In such an environment, web service requesters that need to use a discovery service may need to obtain information from more than one registry or index. Federation refers to the ability to consolidate the results of queries that span more than a single registry or index, and make them appear more like a single service.

A registry or index may contain information about other registries or indexes to help support federation. For example, a registry dedicated to air travel services may know about another registry dedicated to rail travel services. A third registry for general travel services may contain information about some travel services, but may look to other registries for certain categories of services. A search of the general travel registry may return a referral to the requester pointing them to the rail travel registry. Federation of results in this scenario, as contrasted to the referral, would require the general travel registry to submit a query to the rail travel registry on behalf of the requester. The general travel registry would then merge the results of the query to the rail travel registry with the results of a query to its own registry. The general, rail, and air travel registries may need to share a common taxonomy or ontology to avoid forwarding inappropriate queries to other registries. In this scenario, we assume the general travel registry examined the query from the requester and therefore did not forward the query to the air travel registry.

The general travel registry could have discovered the rail travel registry using a spider or index approach. An indexing engine could have come across a registry, and based on the information it harvested from the registry classified it as a rail travel registry. An alternative approach would be for the rail travel registry to publish information to the general travel registry and using the shared taxonomy could classify itself as a registry for rail travel services.

Note that each registry or index may provide a web service for discovery, so it may be appropriate to use a choreography or orchestration description language to describe the exchanges among these services needed for federation.

3.4.4 Functional Descriptions and Discovery

As mentioned at the beginning of 2.3.3.1 Discovery, Web services discovery requires the ability to search for appropriate Web services based on functional descriptions ("FD" in Figure 3-2) or other criteria. Because these functional descriptions need to be machine processable, written by many provider entities and read by many requester entities, an appropriate language for representing functional descriptions should at least be:

• Web friendly (based on URIs and globally scalable)

• Unambiguous

• Capable of expressing any existing or future functionality

• Capable of expressing existing and new vocabularies and relationships between functionalities

This is an area that needs further standardization work. One such effort is OWL-S.

3.5.1 Message semantics and visibility

The extent to which the shared agreement about the form, structure and meaning of a message is shared beyond just the agents involved with the message governs the overall visibility of the message semantics. The emphasis on messages, rather than on the actions that are caused by messages, means that SOAs have good visibility: third parties may inspect the flow of messages and have a some assurance as to the services being invoked and the roles of the various parties. This, in turn, means that intermediaries, such as firewalls, are in a better situation for performing their functions. A firewall can look at the message traffic, and at the structure of the message, and make predictable and reasonable decisions about security.

In REST-compliant SOAs, additional visibility comes from the uniform interface semantics, essentially those of the HTTP protocol: an intermediary can inspect the URI of the resource being manipulated, the TCP/IP address of the requester, and the interface operation requested (e.g. GET, PUT, DELETE) and determine whether the requested operation should be performed. The TCP/IP and HTTP protocols have a widely supported set of conventions (e.g. known ports) to support intermediaries, and firewalls, proxies, caches, etc. are almost universal today.

Visibility, however, goes beyond firewalls. In this architecture, instead of emphasising a REST-style uniform interface, we emphasize messages' structure in terms of envelopes, headers and bodies. We enhance visibility architecturally by fostering agreements on particular forms of headers. For example, by having well-known standards that describe the form and interpretation of authentication tokens in headers, we can simultaneously reduce the cost of performing authentication and increase the overall visiblity of the message's semantics: if the authentication aspect of a message can be specified in a standard way then it is easier for a larger number of interested parties to process the message. Furthermore, increased visibility can reduce the cost of entry into a marketplace.

Other potential examples of standardized headers include support for message reliability, support for message correlation, support for process flow and service composition and support for choreography.

This argument can be extended from obvious infrastructure-related processing of messages to more application-related processing of the message. For example, by capturing customer identification in a well-understood header, then all applications capable of processing that header will be able to extract the customer information of a message independently of the intended final disposition of the message.

This, in turn, suggests an extremely powerful architectural approach to message processing: different stakeholders in an organization, represented by different applications processing different aspects of messages, can collaborate with a minimal pre-ordained design.

3.5.2 Semantics of the Architectural Models

The different models in the architecture focus on different aspects of the interoperability issues between Web service agents. The Message Oriented Model focuses on how Web service agents (requester and provider agents) may interact with each other using a message oriented communication model. The format of messages as XML infosets and the structuring of messages in terms of envelopes, headers and bodies, as described in that model, acts to lay a foundation for the standard comprehension of messages exchanged between Web service agents.

The Service Oriented Model builds on the basics of message communication by adding the concept of action and service. Essentially, the service model allows us to interpret messages as requests for actions and as responses to those requests. Furthermore, it allows an interpretation of the different aspects of messages to be expressed in terms of different expectations, in well understood ways, of the different parts of the message: in effect, an incremental and layered approach to service is possible using well understood headers.

The Resource Oriented Model extends this further by adding the concept of resource. Resources are important internally to the architecture (a Web service is best understood as a resource in the context of Web service management and in terms of policy management) and externally: resources are an important metaphor for interpreting the interaction between a requester entity and a provider entity.

3.5.3 The Role of Metadata

An important part of the Service Oriented Architecture approach is the extensive use of metadata. This is important for several reasons: it fosters interoperability by requiring increased precision in the documentation of Web services and it permits tools to give a higher degree of automation to the development of Web services (and hence lowers the cost of deploying same).

The metadata associated with a Web service can be regarded as a partial machine-readable description of the semantics of the Web service. In particular using technologies such as WSDL, a Web service can be described in a machine readable document as to the forms of expected messages, the datatypes of elements of messages and using a choreography description language the expected flows of messages between Web service agents.

However, current technologies used for describing Web services are probably not yet sufficient to meet interoperability requirements on a global scale. We see the following areas where increased and richer meta-data would further enhance interoperability:

• It should be possible to identify the real-world entities referenced by elements of messages. For example, when using a credit card to arrange for the purchase of goods or services, the element of the message that contains the credit card information is fundamentally a reference to a real-world entity: the account of the card holder.

  The appropriate technology for this is standardized ontology languages, such as OWL.

• It should be possible to identify the expected effects of any actions undertaken by Web service requester and provider agents. That this cannot be captured by datatyping can be illustrated with the example of a Web service for withdrawing money from an account as compared to depositing money (more accurately, transferring from an account to another account, or vice versa). The datatypes of messages associated with two such services may be identical, but with dramatically different effects: instead of being paid for goods and services, the risk is that one's account is drained instead.

  We expect that a richer model of services, together with technologies for identifying the effects of actions, is required. Such a model is likely to incorporate concepts such as contracts (both legally binding and technical contracts) as well as ontologies of action.

• Finally, a Web service program may "understand" what a particular message means in terms of the expected results of the message, but, unless there is also an understanding of the relationship between the requester entity and the provider entity, the provider agent may not be able to accurately determine whether the requested actions are warranted.

  For example, a provider agent may receive a request to transfer money from one account to another. The request may be valid in the sense that the datatypes of the message are correct, and that the semantic markers associated with the message lead the provider agent to correctly interpret the message as a transfer request. However, the transaction still may not be valid, or fully comprehensible, unless the provider agent can properly identfy the relationship of the requester agent's owner (i.e., the requester entity) to the requested action. Currently, such concerns are often treated simply as security considerations, which they are, in an ad hoc fashion. However, when one considers issues such as delegated authority, proxy requests, and so on, it becomes clear that a simple authentication model cannot accurately capture the requirements.

  We expect that a model that formalizes concepts such as institutions, roles (in business terms), "regulations" and regulation formation will be required. With such a model we should be able to capture not only simple notions of authority, but more subtle distinctions such as the authority to delegate an action, authority by virtue of such delegation, authority to authorize and so on.

3.6.1 Security policies

From the perspective of this architecture, there are three fundamental concepts related to security: the resources that must be secured; the mechanisms by which these resources are secured (i.e., policy guards); and policies, which are machine-processable documents describing constraints on these resources.

Policies can be logically broken down into two main types: permission policies and obligatory policies. A permission policy concerns those actions and accesses that entities are permitted to perform and an obligation policy concerns those actions and states that entities are required to perform. These are closely related, and dependent: it is not consistent to be obliged to perform some action that one does not have permission to perform. A given policy document is likely to contain a mix of obligation and permission policy statements.

The two kinds of policies have different enforcement mechanisms: a permission guard is a mechanism that can be used to verify that a requested action or access is permitted; an audit guard can only verify after the fact that an obligation has not been met. The precise form of these guards is likely to vary, both with the resources being controlled and with the implementation technologies deployed.

The architecture is principally concerned with the existence of guards and their role in the architecture. In a well engineered system it may be possible to construct guards that are not directly visible to either the requester or provider agents. For example, the unauthorized access threat may be countered by a mechanism that validates the identity of potential agents who wish access the controlled resource. That mechanism is, in turn, controlled by the policy document which expresses what evidence must be offered by which agents before the access is permitted.

A permission guard acts as a guard enabling or disabling access to a resource or action. In the context of SOAP, for example, one important role of SOAP intermediaries is that of permission guards: the intermediary may not, in fact, forward a message if some security policy is violated.

Not all guards are active processes. For example, confidentiality of information is encouraged by encryption of messages. As noted above, it is potentially necessary to encrypt not only the content of SOAP messages but also the identities of the sender and receiver agents. The guard here is the encryption itself; although this may be further backed up by other active guards that apply policy.

3.6.2 Message Level Security Threats

Traditional network level security mechanisms such as Transport Layer Security (SSL/TLS), Virtual Private Networks (VPNs), IPSec (Internet Protocol Security) and Secure Multipurpose Internet Mail Exchange (S/MIME) are point-to-point technologies. Although traditional security technologies are used in Web services security, however, they are not sufficient for providing end-to-end security context, as Web services require more granularities. In general, Web services use a message-based approach that enables complex interactions that can include the routing of messages between and across various trust domains.

Web services face traditional security challenges. A message might travel between various intermediaries before it reaches its destination. Therefore, message-level security is important as opposed to point-to-point, transport-level, security. In Figure 3-3 below, the requester agent is communicating with the ultimate receiver through the use of one or more intermediaries. The security context of the SOAP message is end-to-end. However, there may be a need for the intermediary to have access to some of the information in the message. This is illustrated as a security context between the intermediary and the original requester agent, and the intermediary and the ultimate receiver.

Figure 3-3. End-to-End Security

The threats listed below addresses message security.

3.6.3 Web Services Security Requirements

There are many security challenges for adopting Web services. At the highest level, the objective is to create an environment, where message level transactions and business processes can be conducted securely in an end-to-end fashion. There is a need to ensure that messages are secured during transit, with or without the presence of intermediaries. There may also be a need to ensure the security of the data in storage.

The requirements for providing end-to-end security for Web services are summarized in the next sub-sections.

3.6.4 Security Consideration of This Architecture

Organizations that implement Web services must be able to conduct business in a secure fashion. This implies that all aspects of Web services including routing, management, publication, and discovery should be performed in a secure manner. Web services implementers must be able to utilize security services such as authentication, authorization, encryption and auditing.

Web services messages can flow through firewalls, and can be tunneled through existing ports and protocols. Web services security requires the use of appropriate corporate wide policies that may need to be integrated with external cross-enterprise policy and trust resolution. Organizations may need to implement the capabilities that are listed next.

3.6.4.6 Secure Messaging

Secure Messaging ensures privacy, confidentiality and integrity of interactions. Digital signatures techniques can be used to help ensure non-repudiation.

Techniques that ensure channel security can be used for securing messages. However, such techniques are applicable in a few limited cases. Examples include a static direct connection between a requester agent and a provider agent. For some applications, such mechanisms can be appropriate. However, in the general case, message security techniques such as encryption and signing of the message payload can be used in routing and reliable messaging.

Figure 3-4. Secure Discovery

3.6.5 Privacy Considerations

Privacy as related to behavior, habits and actions are expressed in terms of policies that the owners of data — typically the users of Web services — have, together with mechanisms necessary to ensure that the owners' rights are respected.

Privacy policies are typically much more of the obligatory form than access control policies. A policy that requires a provider agent to properly propagate P3P tags, for example, represents an obligation on the provider entity. However, it is not possible to prevent a rogue provider agent from leaking private information. Thus, it should be possible to monitor the public actions of the Web service to verify that the P3P tags are propagated appropriately.

Many privacy-related constraints are concerned with maintaining certain kinds of state. For example, a provider entity may have a constraint that any P3P tags associated with a use of one of its Web services are appropriately propagated to third parties. Such a constraint cannot easily expressed in terms of the allowed actions that the provider agent may perform. It is an obligation to ensure that the publicly observable condition (the proper use of P3P tags) is always maintained (presumably maintained in private also). Similarly, a provider agent may link the possible actions that a requester agent may perform to the requester agent maintaining a particular level of secure access (e.g., administrative tasks may only be performed if the request is using secure communications).

3.8.1 Message reliability

Reliability at the level of messages is often referred to as reliable messaging. In any distributed system there are fundamental limits to the reliability of communication between agents on a public network. However, in practice there are techniques that we can use to greatly increase the reliability of messages, and in those cases where communication fails then we can gain some feedback as to what went wrong.

In more detail, we identify two properties of message sending that are important: the sender of the message would like to be able to determine whether a given message has been received by its intended receiver and that the message has been received exactly once.

Knowing if a message has been received correctly allows the sender to take compensating action in the event the message has not been received. At the very least, the sender may attempt to resend a message that has not been received.

The general goal of reliable messaging is to define mechanisms that make it possible to achieve these objectives with a high probability of success in the face of inevitable but unpredictable network, system and software failures.

This goal may also be examined with respect to whether one wishes to confirm only the receipt of a message, or perhaps also to confirm the validity of that message. Three questions may be asked about message validity:

1. Was the message received the same as the one sent? This may be determined by such techniques as byte counts, check sums, digital signatures.

2. Does the message conform to the formats specified by the agreed upon protocol for the message? Typically determined by automatic systems using syntax constraints (e.g. XML well formed) and structural constraints (validate against one or more XML schemas or WSDL message definitions).

3. Does the message conform to the business rules expected by the receiver? For this purpose additional constraints and validity checks related to the business process are typically checked by application logic and/or human process managers.

Of these, the first is considered to be part of reliable messaging, the last is partly addressed by Web service choreography, but is more closely related to the business expectations of the parties.

The Web services architecture does not itself give specific support for reliable messaging, or for reporting in the event of failure. However, it does give guidance as to how this may be accomplished. The headers and body structure of messages can be utilized: by providing standardized headers to support message auditing then message reliability infrastructures can be deployed in ways that do not need to impact applications and services.

In effect, we can augment message traffic as necessary with specific headers and intermediaries that implement specific semantics for message reliability and reporting in the case that message communication fails. Recall that the architecture does not itself mandate a specific means of message delivery. In fact, we envisage many potential modes of communication, including HTTP, SMTP, JMS based message transports. A given message may even involve multiple kinds of message transport. However, since all messages are structured according to SOAP, we can incorporate overall message reliability within the SOAP message structure.

Message reliability is most often achieved via an acknoweldgement infrastructure, which is a set of rules defining how the parties to a message should communicate with each other concerning the receipt of that message and its validity. WS-Reliability and WS-ReliableMessaging are examples of specifications for an acknowledgement infrastructure that leverage the SOAP Extensibility Model. In cases where the underlying transport layer already provides reliable messaging support (e.g. a queue-based infrastructure), the same level of reliability can be achieved in SOAP by defining a binding that relies on the underlying properties of the transport.

3.8.2 Service reliability

As with message reliability, we are not in a position to be able to offer guarantees that service provider agents and/service requester agents will always perform flawlessly; again, especially in the context of a distributed system over a public network where the different agents may be owned by different people and subject to different policies and management it is not possible to engineer complete service reliability. However, as with message communication we can deploy techniques that greatly enhance reliability and reduce the cost of failure. The principal technique here is one of transactional context management.

Transaction management allows conversations between agents to be managed so that all the parties involved have a greater degree of confidence that the transactions between them progress satisfactorily, and in the event of failure the failure may be identified and transactions either cancelled, rolled back or compensated for.

The architecture does not give specific advice on how to implement transactional reliability. However, again as with message reliability, the combination of the flexible and extensible message structures and the concept of multiple processing of messages (via intermediaries implementing service roles) gives us guidance.

One way to incorporate transactional support would be to use standardized headers containing information such as transactional bracket markers and context information that are added to messages exchanged between service requester agents and service provider agents in such a way that intermediaries can process messages and monitor transactions in a way that only minimally impacts existing applications. Specialized transactional intermediaries could process messages' transaction-specific headers (such as beginning of transaction, commitment, roll-back and so on) and mark messages that they process with the results; so that applications can respond appropriately.

Related to transactional monitoring is the monitoring of service choreographies. A significant aspect of the specification of the interface of a service is the pattern of message traffic that one might see. For simple cases, this pattern is often very straightforward; however, for most realistic cases, the choreography of services can be very complex. Monitoring that messages are arriving in the order expected is potentially a significant tool in the deployment of reliable services.

Again, as with transactional monitoring, one approach would be to deploy specialized intermediary processes whose specific function is to ensure that the choreographic as well as the static (i.e., message structure) requirements of service usage are being met. This is especially important when the provider agent of a service is not in the same ownership domain as the requester agent.

The key architectural property being used here is the potential deployment of third party services that monitor and process messages in specific role-oriented ways that neither the requesters of services nor the providers of serives needs to be unduly concerned with. This is possible because the architecture does not require messages to be consumed by single agents — nor conversely to be produced by single agents — but allows multiple agents to collaborate in the processing of a given message. Each service role establishes a specific functionality, often encoded in specific headers of the messages.

3.8.3 Reliability and management

The reliability of the individual requester and provider agents is out of scope of this architecture as we do not comment on the realization of Web services. In some cases reliability at this level can be enhanced by provider entities adopting deployment platforms that have strong management capabilities. Note that platform manageability represents a different perspective than the notion of management identified in Service Management (below), which focuses on the manageability of services from a peer or business-partner perspective.

3.10.1 When Something Goes Wrong

What happens when a transaction goes awry for reasons other than the loss of a message or security violations? Although it is possible, and useful, to automate safeguards both at the protocol and application level, experience indicates that there is a virtually limitless variety of ways that business transactions can fail. Informal interviews with current EDI practitioners have indicated that in practice the 80-20 of what EDI people actually do is involved with the issue of finding out what has actually happened in transactions when something has gone wrong. For example, such an interaction may start with a phone call that goes something like, "Why haven't you paid us?" and continues, "We think we have paid you". In these cases there is often a good faith desire on both sides to figure out what has happened and comply with the requirements of the transaction, but the information that people are working with may differ and coming to a common understanding can take some work.

3.10.2 The Need for Tracking

In current EDI operations, many of the questions that must be answered in these cases can be handled in an automated fashion by the vendor of the proprietary network used in the transactions. For example, if company A asks if the invoice to company B was delivered, the vendor can access its records, probably from a central repository, and respond, "The message was delivered to company B's mailbox on Dec 24 but they have not as yet downloaded the message". Queries of this sort are relatively easy to satisfy in this environment because the vendor is in control of all aspects of the communication. In a Web services scenario, where the transactions take place in a distributed environment, with no central authority, some other means must replace the current automated queries to the EDI vendor or this important tracking capability will be lost.

One possibility would be to provide some kind of uniform tracking interface. The basic requirement here is for companies that are cooperating in a business transaction to find out at any time what is the status and history of the transaction. Significant complexity is added by the fact that multiple companies may be involved. That is, company A may initiate a transaction by sending a message to B, but the process may then involve messages between B and C. In some cases the interactions between B and C may be known to A (as opposed to being part of B's internal process that is opaque to A). It is not immediately clear whether this should be handled by A querying both B and C, or if a responding to a query from A to B should carry with it the obligation to query C and return the results. This is presumably an issue which must be ironed out in the creation of the specification(s) for the uniform interface.

3.10.3 Examples of Tracking

As illustrations, here are some of the typical queries that A might send to B or C. Web Services Usage Scenarios [WSUS] contains additional examples.

1. (Query to B) Did you receive and process message M from A?

2. (Query to B) Please return copies of all messages associated with Transaction T.

3. (Query to B) Please return copies of all messages between A and B in a given time range.

4. (Query to C) Please return all messages associated with transactions involving A during a given time frame (including messages between B and C related to transactions in which A is involved).

5. (Query to B) Please return copies of messages between B and other companies involved with a transaction (or all transactions in a date range).

Of course, in all cases, the party performing the query must be authorized to receive the information.

Current EDI practices may automate some of these queries; others may involve manual processing. In general, however, there are significant cost savings to be realized by automating as much of the process as possible.

3.10.4 Requirements for Effective Tracking

In order to help automate the tracking process, there are various requirements, some of which are probably achievable using current or planned specifications and others of which may require new ones:

1. A uniform, interoperable interface for tracking queries, so that company A can send a standard query to all of its business partners. This interface should be associated with the functional Web services interfaces. For example, such an interface might be implemented as part of a management interface.

2. Standard identifiers for transactions and individual messages that are necessary to define the queries. Note that some of these queries involve identifiers of participants in a transaction other than sender and receiver of a particular message. There are clearly aspects of this requirement that are related to the choreography domain.

3. Policies controlling whether party A is authorized to make tracking queries to B. There may be several variants of such policies: e.g. a can query B about messages directly between A and B but not messages between B and C associated with transactions involving A and so on. It may be possible to establish these policies using mechanisms currently available or under development in the security or policy domain, or there may be transactional aspects to these policies that are not currently being considered.

4. A method to establish the trust relationships necessary to implement the policies in 3.

3.10.5 Tracking and URIs

One of the important connections between Web services architecture and Web architecture as a whole, is the common use of URIs. Although URIs are important to many aspects of Web services, it is particularly worth noting their potential role and benefit in indentifying and tracking transactions in Web services.

As a simple example to illustrate this benefit, suppose URIs are used as transaction identifiers. Each time a new transaction is initiated, a new URI is generated to unambiguously identify that transaction, much like a primary key in a database. However, while a database key may only be unambiguous within a particular database, a URI is globally unambiguous, which means that it can be conveniently transmitted to others without loss or confusion of meaning.

Furthermore, a URI may be dereferenceable: If the URI also represents the location of a document (or a dynamic query into a database), it could act as a convenient link for determining the status or history of that transaction, provided the user is authorized to access such information. (Security mechanisms will need to ensure that a tracking URI cannot be dereferenced without proper authority and privacy controls, but the use of URIs is largely orthogonal to this requirement.)

The potential value of this dual use of URIs — both as globally unambiguous identifiers and as universally dereferenceable links — is one of the most fundamental and important insights in the architecture of the Web. Because the Web services architecture builds on the Web architecture, Web services can leverage the benefits of clarity, simplicity, universality and convenience that this use of URI offers.

This is not to say that Web services tracking must be done using URIs in this way. Indeed, there are other ways tracking can be performed, and any engineering design must take many factors into consideration. Rather, the point is to illuminate the fact that, because Web services architecture is based on Web architecture, Web services have the possibility of taking advantage of this use of URIs.