Interactive Conformance Testing for X3D
Introduction
As the Virtual Reality Modeling Language (VRML)
specification (ISO/IEC 14772-1:1997) continues to evolve, new VRML software
testing requirements are evolving with it. Some of these requirements
include testing new VRML browser rendering features, VRML file encoding
formats and the ability of a VRML web browser to correctly interpret “profiled”
VRML input files. The NIST X3D Test Suite (XTS) provides a conformance
testing tool for measuring a browser’s conformance in rendering of
XML encoded VRML ( called Extensible 3D, or X3D ) files that conform to
the VRML97 profile.
1. Overview
NIST has been a part of the VRML Conformance Testing
effort since SIGGRAPH ’96. During that meeting, the groundwork was laid
for NIST’s involvement and contribution to the Web3D Consortium’s
effort to bring ubiquitous 3D to the World Wide Web. NIST’s contributions
include the VRML Test Suite (VTS), providing UTF8 encoded test files and
associated test documentation covering the testing of the VRML97 base profile
and the VIPER VRML97 reference parser, used to determine conformance of
VRML97 files generated by authoring tools. In addition, NIST has
co-chaired the Conformance and Interoperability Working Group within the
Web3D Consortium; helping to identify conformance and interoperability
problems with VRML browser implementations, and share them with the Web3D
Consortium.
To assist the Web3D Consortium in its goal of an open,
standards-based ubiquitous 3D solution for the internet, NIST continues
to provide conformance testing assistance for VRML through the XTS.
2. Testing Methodology
The XTS, like the NIST VRML Test Suite and other
graphics test suites, is a “falsification-based” conformance test
suite for X3D browsers. Falsification testing is based upon the definition
of test assertions ( called “semantic requirements” ) that establish pre-conditions
for an X3D browser test, and a defined set of post-conditions that
must exist after a test is complete. These semantic requirements must be:
-
Independent - If one semantic implies another we keep only the stronger
of the two, since the other is redundant.
-
Complete - The requirements should require everything that the standard
requires.
-
Consistent - The requirements should not contradict each other.
-
Specific - Each requirement must have some testable consequence,
perhaps in conjunction with other requirements. Broad generalities are
to be avoided in favor of lower-level concrete assertions.
-
Simple - A requirement should not be a long list of requirements;
to a reasonable extent, each requirement should state an atomic rule about
conforming implementations.
Tests are categorized by X3D node group, node and
finally by field. X3D browser test files are designed to fulfill
all of the semantic requirements defined for each individual test.
Documentation is provided for each test. This documentation includes:
-
A list of all semantic requirements for the test
-
A hyper-link to the actual specification reference for the test
-
A complete normative description of the test, including all pre
and expected post-conditions
-
A visual representation of expected test results ( images or mpeg
movies )
3. The X3D Test Suite Application
The XTS is a web-based application that allows anyone
on the internet to browse through the list of X3D browser tests by node
group, node and field. It is a simple, intuitive interface that provides
full documentation for all test in a single unified environment. Currently,
actual test source code (XML files) are presented to the user for downloading
and testing locally on his/her machine. Once sample X3D browser implementations
are available, they will be added to the testing environment to provide
a "real-time" testing environment, similar to the current NIST VRML Test
Suite. For more specific information on how to use the XTS, consult the
online help available with the application.
If you have any questions or comments, please contact
michael.kass@nist.gov