An XTM conformance test suite

<< 2006-08-02 19:24 >>

One thing that's really needed is a conformance test suite for XTM 2.0, which can be used by implementors to verify that they've actually gotten their implementations right. Of course, it could also be used by customers to verify that the implementation they are thinking of buying actually conforms to the standard. This is becoming more important as the hype around Topic Maps grows, and vendors start claiming they support Topic Maps without even knowing what it is.

What a test suite could look like

So what would a test suite look like? Well, I imagine a zip file that contains a bunch of XTM files. One of them would be the master file, and it would contain something like the following (in LTM syntax, since I don't have the patience to type XTM):

#PREFIX tm @"http://psi.topicmaps.org/iso13250/model/"

/* Prototypical ontology */
[test-case = "Test case"]
[invalid-test = "Invalid test"]
[valid-test = "Valid test"]
[canonicalized = "Canonicalized"]
[comment = "Comment"]

tm:supertype-subtype(test-case : tm:supertype, valid-test : tm:subtype)  
tm:supertype-subtype(test-case : tm:supertype, invalid-test : tm:subtype)

/* Example instances */
[no-namespace : invalid-test = "No namespace"
  @"invalid/no-namespace.xtm"]
  /* this means that in the invalid/ directory there is a file called
     no-namespace.xtm which is not a valid XTM 2.0 file; the test
     suite driver should attempt to load it and verify that the
     implementation being tested complains */
  {no-namespace, comment, [[Not valid because there is no namespace
   declaration.]]}

[empty-tm : valid-test = "Empty topic map"
  @"simple/empty-tm.xtm"]
  {empty-tm, canonicalized, "baseline/empty-tm.cxtm"}
  /* the occurrence points to a CXTM file containing the same topic
     map in CXTM format; the test suite driver should load it (and
     not get any errors) then produce a CXTM file, and this must be
     identical to the CXTM file in the test suite */

Given this it should be trivial to make a test suite driver (that is, a program) that uses the test suite to verify that implementations detect all the invalid topic maps, accept all the valid ones, and always produces the correct canonical output (which means they've interpreted the file correctly).

So what are we waiting for?

Unfortunately the CXTM specification is not finalized yet. It's being picked up now by the new editors who have replaced Kal Ahmed, and we hope to have a stable draft ready very soon. In the meantime, however, one could start building the set of test cases, the ontology for the master file, and the general infrastructure.

In fact, there is a SourceForge project to produce such a test suite already. So it would be possible to get started right away. If you're interested you can join the mailing list on SourceForge.

Similar posts

Comments

Arnar Lundesgaard - 2006-08-03 09:34:35

Once such a test-suite reaches maturity, could it also be used for formal certification of topic-map processor conformance? A sort of minimum-interoperability badge.

Is this something that ISO (or the working group) could/would manage or would another organisation be more appropriate?

Lars Marius - 2006-08-03 09:48:20

Yes, I think using it for a kind of certification would be a very good idea. One could also imagine writing articles like this one: http://www.xml.com/pub/a/1999/09/conformance/index.html

ISO normally does not do this kind of thing, and the XML and XSLT test suites were originally hosted by OASIS. The W3C eventually took them over, but I think we could reasonably use OASIS for this if we wanted it to have an official "stamp of approval". OASIS is inexpensive to join, and they are flexible enough that one can conduct nearly all business online and via phone.

Add a comment