The documentation falls into 2 realms:
DOM and CSS
These are sort of API references to all the DOM and CSS properties. However, browsing through, they don’t seem to be terribly useful. At best, these references are programmatically generated (seeded with initial values). It’s not completely clear what its setting out to achieve but my guess is something like “if we put all the information in a wiki, maybe people will come and fill it out,” which seems a little awkward (especially since it’s already been done before).
The primary original content provided by this reference is related to the ‘browser compatibility’ section of each item. For example, examine the background-postition CSS property page. There’s a whole bunch of information being tested for – and a whole lot of ‘N’s in the columns – in fact there’s even a couple rows that contain nothing but ‘N’s. This says to me a couple things:
- When every browser fails a test then you’re probably testing the wrong thing.
- The fact that the actual return values aren’t shown makes this extra frustrating (every browser fails background-position: left – but what do they fail with? what are they returning, instead?).
- There doesn’t seem to be any attempt to test for compliance. Overwhelmingly the test values appear to be “the value that goes in should be the value that comes out” – which is rarely the case. Granted, it’s much easier to write tests of this nature (just generate them programmatically) but the end result doesn’t really help anyone.
Open Web Tests
Thankfully raw dumps of the results are provided for each browser so we can use this as an opportunity to see what tests are being performed (and how the results compare).
3. document/document-attachEvent-typeof-test.html:testTypeOf failed
"[DocumentAttacheventMethod] typeOf(document.attachEvent) != 'undefined'"
Expected not to be <undefined> (String)
Firefox 3 is failing this test because it doesn’t have Internet Explorer’s proprietary attachEvent method? If you continue through the results you’ll see similar indicators all throughout. If you were to open Internet Explorer’s results you would see a number of fails for Netscape-specific methods/properties.
Opening up the HTML Firefox 3 results we see a similar situation. In this case the HTML tests appear to focus on the availability of attribute expandos on DOM elements. For example, here’s one failing test from Firefox 3:
293. attributes/ilayer-above-reflection-test.html:testReflection failed
"[IlayerAboveAttribute] ilayer.above reflects <ilayer above="foo">"
Expected <foo> (String) but was <undefined>
The ilayer element was a Netscape-specific HTML extension. Testing for its DOM compliance (and, by extension, its existence) seems quite futile.
If I had to guess as to how these tests were generated I’d say “open up a browser, spider every DOM property (or HTML attribute), and turn those into existence tests.”
This raises an important point: This suite is not built for any sort of standards compliance – at all – it’s simply designed to check for compatibility between browsers. Looking at this suite as a compatibility suite we can start to see some use – but it’s still terribly limited (only checking for existence is hardly a good-enough indicator of a browser’s proper support of a property or method).
If there’s one over-arching theme to the Google Doctype release it’s been “whatever we can generate automatically, or release with the least amount of fuss, let’s do that, no matter how simple it may be.” I’m definitely looking forward to when a full release comes along, but this doesn’t appear to be it.