W3C

Guidelines for XML Query Test Suite Submissions

Licensing Issues and Acceptance

Tests submissions must use the Grant II: Grant of License for Contributed test Cases Published Outside a W3C Recommendation. This license grant, for non-participant test case Contributors, is described Policies for Contribution of Test Cases to W3C.
The XML Query Working Group and XSL Working Group will, at their sole discretion, determine what tests are incorporated in the test suite.

Conformance to Schemas and File Format

Submissions must be valid with respect to "XQTSCatalog.xsd" schema file.
All submissions must be in a ZIP, tar or gzip file formats.
Please add your test cases to the "XQTSCatalogSubmission.xml" document that we have provided.

Test Structure

Entries in the catalogue file should contain citations of document sections, using both the section number and section name, whenever possible. This practice is highly recommended. A typical test catalogue entry is illustrated below.

<test-case name="test001" FilePath="Axes" ... >
...
<query name="test001.xq" date="2003-02-25">
<description>Child Element Test</description>
</query>
<input-file role="principal-data" context="default" variable="input-context">TreeCompass</input-file>
<output-file role="principal" compare="XML">test001.xml</output-file>
</test-case>

Tests for which the <input-file> element in the catalog file references the "emptydoc" file are not required to make any variable mapping back to the <input-file> element. In other words the following declaration is not required to be present in the test case:

(: insert-start :)
declare variable $input-context external;
(: insert-end :)
Submitters should indicate any potential problems with the tests.
Negatives tests will be accepted. However, they must be identified as such in the catalog file. Please see the Error Tests Section below for futher information. A typical negative test is illustrated below:

(: Name: test001 :)
(: Test Description: count function with missing argument :)
fn:count()

Variable names, function names, etc., should not contain any copyrighted information or any company name or any other text identifying a company. However test submitters are allowed to use a company name as the value of the "featureOwner" attribute of the <test-group> element.
Submitted tests should be minimal in nature and independent of one another.
Submitted tests must at least include a “Name” comment and a “Description” comment. These comments must appear as the first two lines of the file.
Submitters should reuse existing input files whenever possible.
Tests involving dates, times and dateTime should use an explicit timezone (whether optional or required). These tests should use the "Z" whenever possible. Tests designed to evaluate implicit timezone should omit the timezone component.
Submitted tests should be insensitive to boundary-space, that is, in direct element constructors there should be no boundary space between open/close tags and enclosed expression.
Tests involving the decimal and integer data types should use no more than 18 digits.
Test involving the data types xs:date, xs:time, xs:dateTime, xs:gYear, xs:gYearMonth, the maximum value of the year component should be no more than 4 digits, and the maximum number of fractional second digits must be no more than 3.
For tests involving the int, long, short and byte data types, please see the Schema specifications for limits on those data types.
For tests cases that are sensitive to typed values, submittors will provide a result for both the typed and untyped data.
For test cases where typed data is imperative, submittors will need to use constructor functions instead of relying on typed data.
Submitted tests for a particular section must include only functionality defined by that section and by the provided foundation features. The XML Query Working Group defines the following features as foundation features:

Feature Name	Comments
string literals (quoted strings)
integer literals
URI literals
Parentheses
empty-sequence constructor ()
concatenation (comma operator) for sequence
expression comments
/ and //	for path steps
/ alone	for root node
all axes	for both expanded and abbreviated forms
node tests (e.g., text() )
Predicates
element constructors	including text-node content for those elements
Namespace declaration in prolog	Including default
+, - with two operands
fn:true(), fn:false()
fn:not()
and, or
string equality with XPath 1.0 semantics
numeric equality and inequality
node compare (these operators == and != backing up ‘node-equal’)
accessor functions: fn:data, fn:node-kind
fn:empty
fn:count
fn:position, fn:last
fn:string-length
fn:string-to-codepoints
“is” predicate
The Constructors: xs:string, xs:anyURI, xs:integer, xs:decimal, xs:float, xs:double, xs:boolean, xs:date, xs:dateTime, xs:time

Collations and URI’s

When no collation is specified, the Unicode-Codepoint collation is assumed. Tests should be written with that in mind.
URI’s must be based on the www.example.com domain.

Static Typing

Submitted tests should obey the Static Typing rules, except for tests intended to evaluate violations of the Static Typing rules.
Submitted tests must not rely on any static typing extensions.

Optional Features and vendor Specific Features

Submitted tests that evaluate non-optional features should not use optional features.
Submitted tests should not include any vendor specific extensions (pragmas).

Error Tests

Error conditions that can be mapped to the conditions specified in the appendices of the XML Query specifications should include the error code stated for that condition. The following example from the catalog file illustrates one of such conditions:

<test-case name="Literals036" FilePath="Expressions/PrimaryExpr/Literals/" scenario="parse-error" Creator="John Doe">
<description>Test for invalid decimal literal</description>
<spec-citation spec="XQuery" section-number="3.1.1" section-title="Literals" section-pointer="id-literals"/>
<query name="Literals036.xq" date="2005-02-03"/>
<input-file role="principal-data" variable="input-context">emptydoc</input-file>
<expected-error>XPST0003</expected-error>
</test-case>

Any error condition for which there is not a predefined code, the "*" symbol should be used.

External Variable Tests

External variable tests are designed to run queries, whose external variable values are defined by outsite agents. The main catalog will define an <input-query> element. The results of the input query (s) are bind to external variables defined in the main query. The <input-query> element require three attributes, namely:

name - specifies the name of the input query (no extension required).
variable - the name of the variable to which the results of the query should be binded.
date - the date the query was written.

The following table illustrates a typical entry for an external variable test case:

  <test-case is-XPath2="false" name="extvardeclwithtype-17" ...>
   <description>External Variable used to evaluate a boolean expression ... .</description>
   <spec-citation spec="XQuery" section-number="4.14" ... />
   <query name="extvardeclwithtype-17" date="2006-02-09"/>
   <input-file role="principal-data" variable="input-context">emptydoc</input-file>
   <input-query variable="x" name="extvardeclwithtypetobind-17" date="2006-02-09"/>
   <output-file role="principal" compare="Text">extvardeclwithtype-17.txt</output-file>
  </test-case>

The following are the input query and the main query that corresponds to the entry above:

Input Query

   (: Name: extvardeclwithtypetobind-17 :)
   (: Description: Binding boolean expression for extvardeclwithtype-17.:)

   let $var := fn:true() or fn:true()
   return $var

Main Query

     (: Name: extvardeclwithtype-17 :)
     (: Description: Evaluates an external variable used in a boolean expression:)
     (: Both queries perform the operation. :)

     declare variable $x as xs:boolean external;
     $x or fn:false()

Note the name(s) of the external variable(s) defined in the main query must match the names of the variable(s) defined in the input queries ("x" in this case). The main query must be a stand alone query, it should have no input infosets.

Module Import Tests

Module Import tests are accepted as optional features of the XML Query Specifications as outlined in sections 4.2 and 5.2.5. The following example from the catalog file illustrates a typical entry for a module import test.

<test-case name="modules-simple" FilePath="Modules/ModuleImport/" scenario="standard" Creator="Jane Doe">
<description>Import simple library module</description>
<spec-citation spec="XQuery" section-number="4.11" section-title="Module Import" section-pointer="id-module-imports"/>
<spec-citation spec="XQuery" section-number="4.2" section-title="Module Declaration" section-pointer="id-module-declaration"/>
<query name="modules-simple" date="2005-12-05"/>
<module namespace="http://www.w3.org/TestModules/test1">test1-lib</module>
<input-file role="principal-data" variable="input-context">emptydoc</input-file>
<output-file role="principal" compare="XML">modules-simple.xml</output-file>
</test-case>

Note the presence of a required <module> element and the required namespace attribute. There must be one <module> element for every module that is imported by the main query. There also must be one <module> element for each library module that the imported module may in turn have. The content of the <module> element is an IDREF type that must match an ID attribute from a <module> element defined in the sections area of the catalog file as a child of the <sections> element. The following example illustrates the matching ID definition for the example above.

<module ID="test1-lib" FileName="TestSources/test1-lib" Creator="Jane Doe">
<description last-mod="2005-12-05">Simple library module</description>
</module>

Note that the file "TestSources/test1-lib" does not have an extension. The extension for this file is defined at "test-suite/@XQueryFileExtension".

The main query (and any imported library module with subsequent imports) must have an import statement with a prefix and a namespace URI. The namespace URI must match the namespace URI defined in the catalog file. The module import statements must be between the "(: insert-start :)" and the "(: insert-end :)" comments. Please refer to the "Guidelines for Running The XML Query Test Suite" for further usage of these comments. The following example shows the query that corresponds to the example above.

(: insert-start :)
import module namespace test1="http://www.w3.org/TestModules/test1"
declare variable $input-context external;
(: insert-end :)

And the corresponding relevant section from the library module.

module namespace test1="http://www.w3.org/TestModules/test1";

It is highly recommended (but not required) that any library module contain the "lib" string as part of its name.

XML and Namespaces

Submitted tests should be independent of XML versions “1.0” and “1.1” and XML Namespaces versions “1.0” and “1.1” features unless the purpose is to evaluate those features.

XML Encoding

Submitted tests should use UTF-8 encoding.

Related Specifications

Submitted tests should be based exclusively on the XQuery specifications and the Functions and Operators specifications. Some tests may rely on the Formal Semantics, Data Model and Serialization documents, however those specifications are not being directly tested.

XPath Compatibility

The test-group element in the catalog may have the optional attribute is-XPath2="no". If this attribute is not present, when it is assumed that all descendent test cases may or may not be XPath 2.0 tests.
The test-case element in the catalog may have the optional attribute is-XPath2="yes|no". If this attribute is not present, then it is assumed that this test case may or may not be an XPath 2.0 test. If any test-group ancester of this element has is-XPath2="no", then this test-case element cannot have is-XPath2="yes". Over time, all test-case elements will have this attribute defined.

Results and Serialization

Tests in the XQTTF test suite rely on the serialized form of the test result in order to check correctness. Thus, any feature whose effect is not captured as part of the serialized result of a query, should be explicitly checked as part of the test query itself. For example, typing information is not present in the serialized results of an XQuery expression.

Expected Results

Submitters must provide expected results for all submitted tests.
When serializing content of a typed element where the type is atomic, any lexical representation of the value may be serialized. To minimize the number of expected results per test case, the input value used to construct the typed element should be identical to the result if this input value was cast to xs:string. If this is true, then only one expected result with the serialized value in its original lexical form is required for this test case. However, if the intent of the testcase is to test other lexical representations, then two expected results must be created where one result would contain the value if it was cast to xs:string, and the other in its original lexical form.
Many tests involve operations on floats/doubles and coverting those results to strings. Even as one explicit value is given, the task force realizes that other values may also be acceptable. In such cases submitters are encouraged to sumbit values that may differ. The task force will eventually determine if such values are within the acceptable range.

Webmaster · Last modified: $Date: 2006/06/12 00:47:57 $ by $Author: aeisenbe2 $

Copyright © 1994-2005 W3C^® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark, document use and software licensing rules apply. Your interactions with this site are in accordance with our public and Member privacy statements.