>It has been a long month of slowly converting the W3C XQuery/XPath 2.0 Test Suite into something that is usable. It’s actually a bit harder than it really needs to be as the test suite is really designed for XQuery, and it just so happens that most of the tests can be run against XPath 2.0 implementations as well. This has to do with the intentional overlap of XQuery and XPath 2.0. After all XQuery is just XPath 2.0 on steriods.
The entire test suite has been run against the eclipse PsychoPath XPath 2.0 processor and here are the current results:
- Test: 8193
- Failures: 2951
- Errors: 63
- Code Coverage: 74.3 % (coverage results reported by EclEmma.)
These results include both the untyped, and typed (i.e. xml schema) results. Overall, I’m fairly impressed considering the original 1.0 version of PsychoPath was based on the 2004/10 XPath 2.0 specification. So it was out of synch with the changes that occured in 2005.
Most of the problems we are experiencing seem to be around the Numeric calculations. The processor it self was implemented using the built in java primitive types. Unfortunately this doesn’t provide enough accurrecy in the Math related tests. Mukul Gandhi has already provided several patches to help in this area, but more work is still needed.
The W3C Test Suite is fairly comprehensive, but as I stated it really takes a lot of manual massaging of the meta-data to get a useable test suite. Particularly if you are only interested in the XPath 2.0 parts of the suite. Some more consistency or a standard API library would be helpful in extracting the relevant portions.
In the meant time, the next several months will be devoted to going through the failing tests. PsychoPath 1.1 is planned to be released with eclipse Web Tools Platform 3.2 in December 2009, and PsychoPath 1.2 will follow that in June 2009 with WTP 3.3.
Stay tuned for some “State of XML” at eclipse reports around the eclipse Galileo release date. I’ll take off my committers hat, and report purely as a user and adopter on the state of the XML tools at eclipse.