This document contains the results of running the QT3 Test Suite on six implementations of XQuery and XPath.
The latest version of this test suite is QT3 3.1. Implementations that have used older versions of the test suite are noted.
The report excludes:
- Tests added to the test suite in the last few weeks (because many of the result submissions do not include these tests)
- Tests in the catalog test set that failed because of problems in the test catalog
- Tests in the fn:transform test set (because a late specification change means that a significant number of these tests need to be rewritten, and because many of the tests depend on optional features which have not been codified.)
When results are listed as number/number/number, then indicate pass/failed/total. Passed and failed together may not equal total, due to tests not run or not reported.
The latest version of our files is available at http://dev.w3.org/cvsweb/2011/QT3-test-suite/. The "catalog" and "query" links found with each test case are links to this version.
Total number of test cases: 30462
Total number of implementation reports: 6
Results by Specification
Product Version Tests passed Tests failed Tests run % passed Saxon-EE 9.7.0.11 21790 14 21804 99.94 XmlPrime 4.0.0.16079 21716 29 21745 99.87
Product Version Tests passed Tests failed Tests run % passed BaseX 8.5 beta 27966 291 28257 98.97 Saxon-EE 9.7.0.11 30269 11 30280 99.96 XmlPrime 4.0.0.26293 30284 28 30312 99.91
Product Version Tests passed Tests failed Tests run % passed XmlPrime 4.0.0.27704 30281 31 30312 99.90
Sub-Sections:
Optional Features | Combination of Optional Features | New 3.1 Specification Facilities
Results for Optional Features
(Top of page)higherOrderFunctions
higherOrderFunctions satisfied (2220 tests)
moduleImport
moduleImport satisfied (13 tests)
schemaImport
schemaImport satisfied (755 tests)
Tests passed by fewer than two implementations:
Test name Passes Explanation substitution-020 1 Test requires an XSD 1.1 processor substitution-021 1 Test requires an XSD 1.1 processor substitution-022 1 Test requires an XSD 1.1 processor substitution-023 1 Test requires an XSD 1.1 processor substitution-024 1 Test requires an XSD 1.1 processor substitution-025 1 Test requires an XSD 1.1 processor schemaValidation
schemaValidation satisfied (156 tests)
serialization
serialization satisfied (194 tests)
serialization not satisfied (2 tests)
staticTyping
staticTyping satisfied (75 tests)
staticTyping not satisfied (1 test)
typedData
typedData satisfied (4 tests)
typedData not satisfied (2 tests)
Results for Optional Features in Combination
(Top of page)Test with no dependencies on optional features (27237 tests)
Tests passed by fewer than two implementations:
Test name Passes Explanation Serialization-adaptive-37 1 Test requires an XSD 1.1 processor XML10-4ed-Excluded-char-1 0 Test designed for products that don't yet support XML 5th Edition XML11-1ed-Included-char-1 0 Test designed for products that don't yet support XML 5th Edition cbcl-cast-gYear-003a 0 Test requires an XSD 1.1 processor cbcl-cast-gYearMonth-004 1 Test requires an XSD 1.1 processor collection-004 0 Test designed for processors that allow collection stability to be disabled collection-008 0 Test designed for processors that allow collection stability to be disabled collection-010 1 Test assumes a form of collection URI that products are not required to support d1e78807 1 Test uses fn:transform d1e78807b 1 Test uses fn:transform eqname-023 1 XPath-only test requiring support for the namespace axis (deprecated feature) fn-lower-case-19 1 Test results vary depending on Unicode version fn-normalize-unicode-11 0 Test results are specific to Unicode 5.2 fn-normalize-unicode-11-6-2 1 Test results are specific to Unicode 6.2 fn-upper-case-19 1 Test results vary depending on Unicode version re00288 1 Test results vary depending on Unicode version re00370 1 Test results vary depending on Unicode version re00480 1 Test results vary depending on Unicode version same-key-008 1 Test dependencies in metadata misdocumented? same-key-025 1 Test dependencies in metadata misdocumented? xs-error-017 1 Test requires an XSD 1.1 processor xs-error-018 1 Test requires an XSD 1.1 processor xs-error-019 1 Test requires an XSD 1.1 processor xs-error-020 1 Test requires an XSD 1.1 processor xs-error-021 1 Test requires an XSD 1.1 processor xs-error-022 1 Test requires an XSD 1.1 processor xs-error-023 1 Test requires an XSD 1.1 processor xs-error-025 1 Test requires an XSD 1.1 processor xs-error-027 1 Test requires an XSD 1.1 processor xs-error-028 1 Test requires an XSD 1.1 processor xs-error-029 1 Test requires an XSD 1.1 processor xs-error-031 1 Test requires an XSD 1.1 processor xs-error-045 1 Test requires an XSD 1.1 processor xs-error-046 1 Test requires an XSD 1.1 processor Tests depending on: not (serialization) (2 tests)
Tests depending on: not (staticTyping) (1 tests)
Tests depending on: not (typedData) (2 tests)
Tests depending on: higherOrderFunctions (2141 tests)
Tests depending on: higherOrderFunctions + schemaImport (53 tests)
Tests depending on: higherOrderFunctions + schemaValidation (7 tests)
Tests depending on: higherOrderFunctions + staticTyping (17 tests)
Tests depending on: higherOrderFunctions + typedData (2 tests)
Tests depending on: moduleImport (13 tests)
Tests depending on: schemaImport (584 tests)
Tests passed by fewer than two implementations:
Test name Passes Explanation substitution-020 1 Test requires an XSD 1.1 processor substitution-021 1 Test requires an XSD 1.1 processor substitution-022 1 Test requires an XSD 1.1 processor substitution-023 1 Test requires an XSD 1.1 processor substitution-024 1 Test requires an XSD 1.1 processor substitution-025 1 Test requires an XSD 1.1 processor Tests depending on: schemaImport + schemaValidation (117 tests)
Tests depending on: schemaImport + staticTyping (1 tests)
Tests depending on: schemaValidation (32 tests)
Tests depending on: serialization (194 tests)
Tests depending on: staticTyping (57 tests)
Tests depending on: typedData (2 tests)
Results for New 3.1 Facilities
(Top of page)Version declaration specifies XQuery 3.1 (6 tests)
String templates (44 tests)
Empty enclosed expression in content (17 tests)
Array Square Constructor (39 tests)
Array Curly Constructor (20 tests)
Exponent separator (15 tests)
Arrow operator (=>) (44 tests)
New built in type numeric (22 tests)
The error err:FOTY0013 when atomizing a function, map or array (5 tests)
Added the Unary Lookup operator ('?') for maps and arrays (40 tests)
Added the Postfix Lookup operator ('?') for maps and arrays (133 tests)
General features of Arrays: Atomization, instanceof, mixed content (67 tests)
Add map constructor syntax (107 tests)
General features of Map: map type, instanceof (92 tests)
Extend the fn:boolean function to handle arrays and maps (5 tests)
Extend the fn:string function to handle arrays and maps (7 tests)
Add fn:string-join#1 overload Changed the first argument of fn:string-join to xs:anyAtomicType* (5 tests)
Add fn:data#0 overload Extend the fn:data function to handle arrays and maps (7 tests)
Extend the fn:deep-equal function to handle arrays and maps (33 tests)
Add fn:parse-ietf-date function (112 tests)
Add fn:sort function (53 tests)
fn:error can be empty sequence - FOER0000 (5 tests)
Move fn:format-number from XSLT fn:format-number formatting using scientific notation (76 tests)
Add fn:apply function (24 tests)
Add fn:contains-token function (45 tests)
Add fn:default-language function (7 tests)
Add fn:json-doc function (97 tests)
Add fn:json-to-xml function (100 tests)
Add fn:parse-json function (160 tests)
Add fn:transform function (5 tests)
Tests passed by fewer than two implementations:
Test name Passes Explanation d1e78807 1 Test uses fn:transform d1e78807b 1 Test uses fn:transform Add fn:collation-key function (32 tests)
Add fn:load-xquery-module (91 tests)
Add fn:random-number-generator (52 tests)
Introduced a one argument version of the fn:trace function (7 tests)
Stricter rules for the dynamic types of fn:max (6 tests)
Stricter rules for the dynamic types of fn:min (6 tests)
Add a one arugment fn:tokenize function (20 tests)
Add fn:xml-to-json function (132 tests)
Unicode Collation Algorithm (188 tests)
Add map:for-each function (22 tests)
Add map:get function (39 tests)
Add map:entry function (16 tests)
Add map:contains (35 tests)
Add base64Binary-greater-than() operator (32 tests)
Add base64Binary-less-than() operator (32 tests)
Add op:hexBinary-greater-than() operator (32 tests)
Add op:hexBinary-less-than() operator (32 tests)
Add op:same-key() operator (27 tests)
Tests passed by fewer than two implementations:
Test name Passes Explanation same-key-008 1 Test dependencies in metadata misdocumented? same-key-025 1 Test dependencies in metadata misdocumented? Add the map function map:merge (48 tests)
Add the map function map:remove (26 tests)
Add the map function map:put (27 tests)
Add the map:size function (26 tests)
Add the map:keys function (22 tests)
Add the map:find function (14 tests)
Add the array:append function (16 tests)
Add the array:filter function (16 tests)
Add the array:flatten function (17 tests)
Add the array:fold-left function (17 tests)
Add the array:fold-right function (18 tests)
Add the array:for-each function (12 tests)
Add the array:for-each-pair function (15 tests)
Add the array:get function (15 tests)
Add the array:put function (14 tests)
Add the array:head function (15 tests)
Add the array:insert-before function (17 tests)
Add the array:join function (20 tests)
Add the array:reverse function (11 tests)
Add the array:remove function (21 tests)
Add the array:size function (21 tests)
Add the array:sort function (43 tests)
Add the array:subarray function (28 tests)
Add the array:tail function (13 tests)
Add the collation URI for the HTML ASCII-case-insensitive comparison rules (18 tests)
Changed formatting of milliseconds (9 tests)
Add JSON output method (119 tests)
The allow-duplicate-names parameter to the JSON output method (10 tests)
Adaptive serialization method (84 tests)
Tests passed by fewer than two implementations:
Test name Passes Explanation Serialization-adaptive-37 1 Test requires an XSD 1.1 processor General features of the serialization added in 3.1 spec (13 tests)
Generated by O'Neil Delpratt on 6 December 2016