Web Services Addressing 1.0 Test Suite

Introduction

One of the deliverables specified in the Charter of the Web Services Addressing Working Group is a test suite intended to promote implementation of the Candidate Recommendation, and to assess interoperability between these implementations. The Working Group Charter also states that the Working Group is expected to demonstrate four interoperable implementations during the Call for Implementations step. It is anticipated that some, but not all of the testcases held within the test suite will be used to practically test implementations during an interoperability Workshop and thereby satisfy the entrance criteria for the Web Service Addressing specifications to enter the Proposed Recommendation stage.

The test suite will be maintained by the Working Group and remain available for regression testing beyond publication of the Web Addressing Recommendation, however it is not expected that the test suite will form a part of the Web Addressing Recommendation.

Conformance

Conformance to the Web services Addressing Test Suite does not by itself enable a party to claim conformance with the Web Services Addressing specification. Not passing an individual test may be used as an indication that an implementation is likely not to conform to the Web Services Addressing specification.

A Web Services Addressing implementation that passes all of the tests specified in this document may claim conformance to the Web Services Addressing Test Suite (Release-Date).

Structure

The Test Suite contains:

features
The feature list forms part of the Exit Criteria used to transition a Web Service Addressing specification from Candidate Recommendation to Proposed Recommendation.
documents
Example messages and other documents including WSDL 1.1 and WSDL 2.0
message exchanges
The concrete pattern followed by a successful execution of the test case.
testcases
The collection of test cases including assertions applied to each captured message.
endpoints
Public endpoints available during CR testing..
logs
Captured messages exchanged by implementations.
observer
A processor which validates captured log files against the testcases assertions.
report
The result of running the observer against a set of captured log files.

The report and log files are currently restricted to W3C membership.

Operations

The testsuite currently consists of two simple operations:

notify
Text is sent in a message.
No reply is expected.
An empty text string or a text string starting with the string 'fault' triggers a fault.
echo
Text sent in a message is echoed in a reply message.
An empty text string or a text string starting with the string 'fault' triggers a fault.

It's suggested that implementers are optionally able to send the testcase identifier as the text of the message to assist debugging. In the cases where the message is intended to cause a fault, the testcase identifier can be sent in a string beginning with 'fault'.

A WSDL author may elect to describe the echo operation as either a single In-Out operation, or as a pair of one-way operations without impacting the actual messages exchanged.

Test Case Metadata

This Test Suite comprises of a collection of testcases. Each test case is categorized using the following test case metadata:

Identifier
An unambiguous xml:id value to enable the test case to be referenced within a specific version of the test suite.
Title
A friendly name for the test case.
Description
A description of the nature and characteristics of the test case to assist reviewers to understand the purpose of a test and provide those executing the test with any additional information that will be helpful should the test fail.
Features
A list of testable features the test case exhibits. The feature list forms part of the Exit Criteria used to transition a Web Service Addressing specification from Candidate Recommendation to Proposed Recommendation.
Message Exchange
The concrete message exchange pattern followed by a successful execution of the test case. The following nodes should be used when prodiving testcases:

node address
none http://www.w3.org/2005/08/addressing/none
anonymous http://www.w3.org/2005/08/addressing/anonymous
A http://example.org/node/A
B http://example.org/node/B
Documents
One or more example documents, including WSDL 1.1 and WSDL 2.0 documents as well as example messages and EPRs. The example messages may be used to build 'canned' interactions for some testcases.
XPath Expressions
Each test case may be accompanied a series of XPath 2.0, tests which may be asserted against a log of captured messages by an independent observer of the messages being exchanged.

The following namespace prefixes are known to the observer when evaluating XPath tests:

prefix namespace
wsa http://www.w3.org/2005/08/addressing
wsaw http://www.w3.org/2005/03/addressing/wsdl
soap11 http://schemas.xmlsoap.org/soap/envelope/
soap12 http://www.w3.org/2003/05/soap-envelope
wsdl11 http://schemas.xmlsoap.org/wsdl/
wsdl20 http://www.w3.org/2005/08/wsdl
xs http://www.w3.org/2001/XMLSchema
customer http://example.org/customer
notify http://example.org/notify
echo http://example.org/echo
alertcontrol http://example.org/alertcontrol

The following wsa:Action values are used by the testsuite:

Action Message
http://example.org/action/notify notify request
http://example.org/action/echoIn echo request
http://example.org/action/echoOut echo response
http://example.org/action/fault testsuite fault
http://www.w3.org/2005/08/addressing/fault Web Services Addressing 1.0 fault

Contributing

The Test Suite consists entirely of voluntary contributions of tests from interested parties. We encourage all members of the community to consider contributing any tests they may have developed to the Test Suite.

Tests may be submitted to the Suite in the form of an Email sent to public-ws-addressing-tests@w3.org An archive is available at http://lists.w3.org/Archives/Public/public-ws-addressing-tests Submitted testcases should be released to the Test Suite by completing the Test Case Grant II Form.

When contributing materials to the Test Suite, it is helpful if each individual test case cites the metadata such as the existing features and message exchanges exhibited in a format suitable for adding to testcases.xml. The submission should include complete examples of the messages exchanged as XML documents.

Where possible the tests should re-use existing operations and message exchanges, however the submission may also include details of any new scenarios which the test case exhibits, in which case providing any additional metadata such as WSDL 1.1 and WSDL 2.0 documents will be of value.

Status

Much of the Test Suite was developed before and during an Interoperability Event held 17-18 January in Vancouver, CA. The testcase, feature and scenario identifiers will remain stable and should an item be removed, the identifier won't be reused to avoid confusion.

Releases

The Test Suite is periodically made available in the form of a single file:


Paul Downey, David Illsley, Jonathan Marsh, Editors
Bob Freund, Chair
Hugo Haas, Philippe Le Hégaret, W3C Team contacts
$Date: 2007/07/09 20:33:14 $