Here are some initial thoughts on what we can test. Please comment. This document is based on Suggested Contents of Test Suite.
Refer to Policies for Contribution of Test Cases to W3C for the official guidelines.
The Test Suite is stored in CVS under the test-suite module. It contains the following test case folders which are described below:
WSDL is a language so we can test whether a document belongs to the language or not (actually, we need to apply this to a collection of documents since we have <import> and <include>). We can encode many, but not all of the language rules in XML Schema. The XML Schema is a normative part of the WSDL spec. However, there are rules that cannot be expressed in XML Schema and we need to record them. We will mine the WSDL spec and create a Test Assertion Document (TAD). Each assertion will be given a unique, stable, identifier.
Our test suite should consist of two buckets: Good and Bad (I am avoiding the use of the term Valid since XML already uses that term to indicate compliance with a schema).
The Good bucket consists of documents that are in the language. We should aim for adequate test coverage since people will look to this bucket for examples of usage of the capabilities of WSDL. I suggest we approach the test coverage problem as follows. We should analyse the XML Schema and extract a list of every element name, attribute name, and enumerated value, and we should aim that each appears in at least one Good test case.
The Bad bucket consists of documents that are not in the language. I suggest that our test coverage aims to have each Test Assertion be violated by at least one test case in the Bad bucket.
The above WSDL document tests address the syntax of the language. The semantics of the WSDL language include descriptions of messages. The WSDL specification includes binding rules that define the content of concrete messages as they are transmitted over some transport protocol, e.g. SOAP over HTTP.
Our test suite should include buckets of Good and Bad message instances for each normative binding. Each test case in these buckets consists of a triple that contains a Good WSDL document, a message definition for some bound operation defined in the document, and a message instance. The test case is Good if the WSDL document correctly describes the message instance, and is Bad otherwise.
We need to mine the WSDL binding specs and extract a TAD for each binding. In the case where the message body is XML, there may be an XML Schema which can be used to validate the message, and we should add this to the test case.
The Good bucket should contain test cases that illustrate the use of each main aspect of the binding. The Bad bucket should contains test cases that in total violate each Test Assertion.
The WSDL spec also defines Message Exchange Patterns (MEP) that define an interaction with the Web service. This is part of the semantics of WSDL. A test case consists of a triple that contains a Good WSDL document, a bound operation defined in the document, and a set of message instances and their roles as defined by the MEP of the operation. A test case is Good if the message instances conform to the operation, and is Bad otherwise.
We need to mine the WSDL MEP spec and extract a TAD.
Our suite should contain Good test cases for each normative MEP and Bad test cases that in total violate each Test Assertion for each MEP.
This area needs further clarification in the spec. We need to mine the spec and extract a TAD.
A test case consists of a pair that contains a Good WSDL document, and some parameterization of the capabilities of a processor. This parameterization remains to be determined, but could consist of a list of "understood" namespaces, or an assertion that all namespaces are understood.
The Good bucket should contain WSDL documents that illustrate the major ways that extensions and be added to documents and be marked with wsdl:required. The Bad bucket should contain test cases that in total violate each Test Assertion.