Copyright © 2008 W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark, document use rules apply.
This is a supporting document for authors of tests in the Mobile Web Test Suites Working Group. It defines the key properties for Mobile web tests. It proposes various guidelines for tests authors to follow when writing tests.
These guidelines were produced by members of the Mobile Web Test Suite working group which is part of the Mobile Web Initiative (see About).
Comments on, and discussions of this document can be sent on the (archived) public mailing list public-mwts@w3.org (see instructions). W3C Members can also send comments directly to the Mobile Web Test Suites working group.
These guidelines represent the current thinking of the working group and as such may be updated, replaced or rendered obsolete by other W3C documents at any time. Its publication does not imply endorsement by the W3C membership or the Mobile Web Test Working Group.
Patent disclosures relevant to the Mobile Web Test Suites Working Group may be found on the Working Group's public patent disclosure page.
Tests should be simple and evaluate a specific feature of the specifications.
Tests should be in conformance to the specifications it is targeting. No test should be dependent on any device and/or browser in particular.
Test users should be able to run the tests without having a deep understanding of the specifications.
Test cases should not provide a lot of technical explantions in teh text given the limited screen space available.
The tests should be self explanatory of what the expected results should be. Whenever alternate expected results are possible, it should also be self explanatory.
Test should contain the minimum number of lines necessary to evaluate the feature.
Scrolling and pagination should be keep to a minimum unless the test is specifically addressing those features.
Tests should be positive, that is conformant to the addressed specifications, unless they are specifically designed to catch negative/error conditions.
Whenever a test is addressing an error condition, it should clearly indicate so.
Whenever possible a test should contain a link back to the specific area of the specifications it is evaluating.
Whenever possible tests should not contain background colors as part of the test as some colors (such as red and green) are usually used to indicate pass/failure.
Whenever possible execution results should use green coloring to indicate success.
Whenever possible execution results should use red coloring to indicate failure.
Whenever possible execution results should use yellow coloring to indicate uncertain results.
Tests should use UTF-8 or UTF-16 encoding as much as possible unless the feature under test is encoding.
Tests should be written as to avoid any special fonts unless the purpose of the test is to evaluate special fonts.
In general test should be short in nature. However For tests that must be long (e.g. scrolling tests), it is important to make it clear that the filler text is not relevant, otherwise the tester may think he is missing something and therefore waste time reading the filler text. Good text for use in these situations is, quite simply, "This is filler text. This is filler text. This is filler text.". If it looks boring, it's working!
Test writers should avoid test, whose purpose and expected results are not obvious. For instance a test for which half of a sentence is normal text and the other half is half bold is not very obvious. Exception will be tests, whose nature and purpose will require such combinations.
Test writers should provide an easy, non-obstrusive navigation across test cases.
Test writers should provide a test harness, that loads one test case at a time rather than many at once.
In general test writers should avoid cryptic test names. It is suggested that tests follow a test-topic-xxx.ext where:
where: