Having downloaded the XML Query Update Test Suite (XQUTS), constructed your test harness, and executed the test cases, we ask that you send us your results. As we will be generating reports from all of the results that are submitted, we ask that you send us your results in an XML document with a very specific format.
We have included several documents in the XQUTS zip file that you downloaded:
YourDirectory
ReportingResults
Build.xml
Results.xml
XQUTSReport.html
XQUTSReportSimple.html
XQTSResult.xsd
XQUTSResults.xsl
XQUTSSampleResult.xml
We ask that the document that you send us begin with a test-suite-result element and validate using the schema defined by XQTSResult.xsd. We have provided a sample result file for you to look at, XQUTSSampleResult.xml. This document has the following structure:
<?xml version="1.0" encoding="UTF-8"?>
<test-suite-result
xmlns='http://www.w3.org/2005/02/query-test-XQTSResult'
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<implementation ...>
<organization ... />
<submittor ... />
<description> ... </description>
<implementation-defined-items> ... </implementation-defined-items>
<features> ... </features>
<context-properties> ... </context-properties>
</implementation>
<test-run ... >
<transformation> ... </transformation>
<comparison> ... </comparison>
<otherComments> ... </otherComments>
</test-run>
<test-case ... />
.
.
.
</test-suite-result>
You can provide information about your implementation using the implementation element. You can tell us about your organization and who is submitting these results.
The implementation element also allows you to tell us which features of XQuery Update your implementation supports, the values of implementation-defined items, and whether you have augmented or overwritten properties in the static and dynamic context. The names of all of these items are provided in the feature, implementation-defined-item, and context-property elements of XQUTSCatalog.xml, respectively.
You might wish information about the results you submit to be anonymous. If you set the anonymous-result-column attribute to 'true', then the column that contains your results will have the heading 'Anonymous'. If the anonymous attribute in the organization element is set to 'true', then the reports that we generate will not contain any header information that identifies your implementation.
You may provide information about how you transformed the queries and how you compared your results with the expected results in the test-run element. These elements, along with the otherComments element, allow you to provide free text within one or more p elements.
For each test case in XQUTSCatalog.xml, you should report your result in a test-case element. Results can be pass, fail, not applicable, not tested, or cannot tell. You can provide a comment with each result, if you desire. This looks something like the following:
<test-case name="id-insert-expr-001" result='pass' /> <test-case name="id-insert-expr-002" result='fail' /> <test-case name="id-insert-expr-003" result='not tested' comment='This was hard.' />
We have provided the XQUTSResults.xsl style sheet, so that you can generate your own report from the XML result document that you have produced. We have provided the Build.xml Ant script that invokes the transformation for you. The Results.xml document contains a result element for each test result that is to be included in the report. The result element contains the URL of the XML document that contains the test results. XQUTSReport.html and XQUTSReportSimple.html were produced from the sample report document that we provided.
Copyright © 2008-2009 W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark, document use and software licensing rules apply. Your interactions with this site are in accordance with our public and Member privacy statements.