QT3 Test Suite Result Summary

This document contains the results of running the QT3 Test Suite on fifteen implementations of XQuery and XPath.

The latest version of this test suite is QT3 1.0. Implementations that have used older versions of the test suite are noted.

When results are listed as number/number/number, then indicate pass/failed/total. Passed and failed together may not equal total, due to tests not run or not reported.

The latest version of our files is available at http://dev.w3.org/cvsweb/2011/QT3-test-suite/. The "catalog" and "query" links found with each test case are links to this version.


Summarized Results:

Total number of test cases: 26770

Total number of implementation reports: 15

Results by Specification

XPath 2.0 (14835 tests)

Product Version Tests passed Tests failed Tests run % passed
Saxon-EE 9.5.1.1 14812 0 14812 100.00
XmlPrime 3.0.0.0 14819 0 14819 100.00

XPath 3.0 (18655 tests)

Product Version Tests passed Tests failed Tests run % passed
Saxon-EE 9.5.1.1 18586 0 18586 100.00
XmlPrime 3.0.0.0 18602 0 18602 100.00

XQuery 1.0 (21752 tests)

Product Version Tests passed Tests failed Tests run % passed
Saxon-EE 9.5.1.1 21696 0 21696 100.00
XmlPrime* 3.0.0.0 21740 1 21741 100.00
XmlPrime 3.0.0.0 21729 1 21730 100.00
Zorba 2.9 21586 75 21661 99.65
*Implementation results on XQueryX Syntax.

XQuery 3.0 (26247 tests)

Product Version Tests passed Tests failed Tests run % passed
BaseX 7.7 25493 100 25593 99.61
EMC/xDB xDB_main_r2411365 24168 411 24579 98.33
Saxon-EE* 9.5.1.4 25495 10 25505 99.96
Saxon-EE 9.5.1.1 26159 0 26159 100.00
XmlPrime 3.0.0.0 26178 1 26179 100.00
XmlPrime* 3.0.0.0 26193 2 26195 99.99
Zorba 2.9 25901 181 26082 99.31
*Implementation results on XQueryX Syntax.

Results for Optional Features

calendar

calendar=CB not satisfied (2 tests)

default-language

default-language=en satisfied (10 tests)

directory-as-collection-uri

directory-as-collection-uri satisfied (1 test)

format-integer-sequence

format-integer-sequence=١ satisfied (2 tests)

format-integer-sequence=ﯴ not satisfied (1 test)

format-integer-sequence=① satisfied (1 test)

format-integer-sequence=⑴ satisfied (1 test)

format-integer-sequence=⒈ satisfied (1 test)

format-integer-sequence=Α satisfied (1 test)

format-integer-sequence=α satisfied (1 test)

format-integer-sequence=一 satisfied (1 test)

higherOrderFunctions

higherOrderFunctions satisfied (1522 tests)

infoset-dtd

infoset-dtd satisfied (32 tests)

language

language=en satisfied (32 tests)

language=xib not satisfied (2 tests)

language=de satisfied (1 test)

limits

limits=year_lt_0 satisfied (2 tests)

moduleImport

moduleImport satisfied (21 tests)

namespace-axis

namespace-axis satisfied (35 tests)

schema-location-hint

schema-location-hint satisfied (3 tests)

schemaImport

schemaImport satisfied (589 tests)

schemaValidation

schemaValidation satisfied (141 tests)

serialization

serialization satisfied (34 tests)

serialization not satisfied (2 tests)

staticTyping

staticTyping satisfied (74 tests)

typedData

typedData satisfied (1 test)

typedData not satisfied (1 test)

unicode-normalization-form

unicode-normalization-form=NFD satisfied (1 test)

unicode-normalization-form=NFKD satisfied (1 test)

unicode-normalization-form=NFKC satisfied (1 test)

unicode-normalization-form=FULLY-NORMALIZED satisfied (2 tests)

unicode-normalization-form=FULLY-NORMALIZED not satisfied (2 tests)

xml-version

xml-version=1.0 satisfied (10 tests)

xml-version=1.1 satisfied (14 tests)

xml-version=1.0:4- satisfied (2 tests)

Two tests were passed by only one implementation

xml-version=1.0:5+ 1.1 satisfied (2 tests)

xpath-1.0-compatibility

xpath-1.0-compatibility satisfied (5 tests)

xsd-version

xsd-version=1.1 satisfied (73 tests)

xsd-version=1.0 satisfied (9 tests)

Results for New 3.0 Facilities

Abstract and blocked elements in substitution groups (20 tests)

Add fn:analyze-string function (32 tests)

Add fn:available-environment-variables function (11 tests)

Add fn:base-uri#0 overload (23 tests)

Add fn:data#0 overload (5 tests)

Add fn:document-uri#0 overload (20 tests)

Add fn:filter function (32 tests)

Add fn:fold-left function (20 tests)

Add fn:fold-right function (20 tests)

Add fn:for-each function (11 tests)

Add fn:format-integer (64 tests)

Add fn:function-arity function (28 tests)

Add fn:function-lookup function (560 tests)

Add fn:function-name function (32 tests)

Add fn:has-children function (37 tests)

Add fn:head function (6 tests)

Add fn:innermost function (58 tests)

Add fn:map-pair function (47 tests)

Add fn:nilled#0 overload (11 tests)

Add fn:node-name#0 overload (9 tests)

Add fn:outermost function (58 tests)

Add fn:parse-xml-fragment function (23 tests)

Add fn:parse-xml function (14 tests)

Add fn:path function (20 tests)

Add fn:round#2 overload (10 tests)

Add fn:serialize function (14 tests)

Add fn:string-join#1 overload (6 tests)

Add fn:tail function (6 tests)

Add fn:unparsed-text-lines (54 tests)

Add fn:uri-collection function (7 tests)

Add math:acos function (9 tests)

Add math:asin function (9 tests)

Add math:atan2 function (10 tests)

Add math:atan function (9 tests)

Add math:cos function (9 tests)

Add math:exp10 function (8 tests)

Add math:exp function (9 tests)

Add math:log10 function (9 tests)

Add math:log function (9 tests)

Add math:pi function (5 tests)

Add math:pow function (34 tests)

Add math:sin function (9 tests)

Add math:sqrt function (9 tests)

Add math:tan function (11 tests)

Add q flag to regex functions (8 tests)

Add regex non-capturing groups (5 tests)

Allow casting from string to a list type (36 tests)

Allow casting from variable string to QName or NOTATION (12 tests)

Allow casting to a union type (37 tests)

"allowing empty" in FLWOR expressin (21 tests)

Ann fn:environment-variable function (12 tests)

Annotation assertions in function tests (18 tests)

Change error codes returned by fn:collection (2 tests)

Change fn:namespace-uri-for-prefix to return empty sequence if prefix="" or () and there is no default namespace (8 tests)

Change rules for fn:error in the light of try/catch (20 tests)

Change rules for fn:resolve-uri function (9 tests)

Computed namespace constructors (42 tests)

Consistency constraints on reserved namespaces (44 tests)

Context item declaration (41 tests)

Count clause in FLWOR expression (10 tests)

Cycles among global variables are now detected as a dynamic error (23 tests)

Cycles in module import are not prohibited provided functions and variables have no circular dependencies (11 tests)

Decimal format declarations (31 tests)

Default values for external variables (39 tests)

Dynamic function calls (598 tests)

EQNames (22 tests)

Error codes for errors casting to namespace-sensitive types (20 tests)

Error code XQST0118 for start tag / end tag mismatch (7 tests)

Forwards references to global variables are allowed (6 tests)

Function and variable annotations (51 tests)

Function item coercion (6 tests)

"group by" in FLWOR expression (30 tests)

HTML output method including HTML5 (46 tests)

Inline function expressions (49 tests)

In regex, "." does not match CR (8 tests)

Let expressions now in XPath (21 tests)

Move fn:format-date from XSLT (95 tests)

Move fn:format-dateTime from XSLT (80 tests)

Move fn:format-number from XSLT (78 tests)

Move fn:format-time from XSLT (39 tests)

Move fn:generate-id from XSLT (25 tests)

Move fn:unparsed-text-available from XSLT (51 tests)

Move fn:unparsed-text from XSLT (52 tests)

Named function references (595 tests)

Output declarations and serialization parameter files (34 tests)

Partial function application (38 tests)

Private functions (31 tests)

Private variables (11 tests)

Removed restrictions on FLWOR expressions, eg multiple where clauses (10 tests)

Sequence type subtype relationships moved from Formal Semantics (21 tests)

Simple mapping operator ('!') (17 tests)

String concatenation operator ('||') (92 tests)

Switch expression (22 tests)

Try/Catch (167 tests)

Tumbling and sliding windows (117 tests)

Type derivation involving union types (19 tests)

Typeswitch cases can now name a set of types (9 tests)

Union types as Sequence Type (21 tests)

Validate by type (17 tests)

Validation checks that NOTATION values are declared (7 tests)

Version declaration specifies XQuery 3.0 (26 tests)

Wildcard syntax Q{uri}* (6 tests)

XML and XHTML output methods (0 tests)

XSD 1.1 support (12 tests)


Generated by Michael Kay on 23 January 2014