Last year I did some work on implementing a Selectors API Test Suite, which I’ve just updated to run in IE 8.
I’ve uploaded a copy of the suite here:
https://johnresig.com/apps/selectortest/
You can get the source here:
http://github.com/jeresig/selectortest/tree/master
For right now I’m getting the following result:
- WebKit Nightly 99.3% (16 failing – doesn’t support complex :not() expressions)
- Firefox Nightly 99.3% (16 failing – doesn’t handle ‘undefined’ being passed in, correctly)
- IE 8 RC 1 45.9% (1171 failing – Major problem areas are lack of whitespace trimming, incorrect exceptions being thrown, and lack of full CSS 3 selector support)
- Opera 10a1 99.0% (22 failing – Empty string checking in attributes fails and some disconnected checkbox checks fail)
I’ve also written about the improvements that querySelectorAll is bringing to web developers, along with some of the hardships associated with it.
GeekFG (February 12, 2009 at 12:31 pm)
Great stuff
Rodrigo Moyle (February 12, 2009 at 12:36 pm)
Hi, i open the testsuite and all tests fail, its ok?
Ryan Breen (February 12, 2009 at 12:45 pm)
As a follow up on the hardships post, now that you have more information on the comparative success rates of different implementations, I’m curious whether any performance improvement from the native Selectors API will remain once libraries are burdened with enough wrapping code to make the API work consistently.
In the IE8 case, would you even bother trying to use the native API in jQuery, or would the overhead outweigh the gains?
Charles Lawrence (February 12, 2009 at 12:57 pm)
I ran the test in FF3 and IE7 and all tests failed.
James L. (February 12, 2009 at 1:01 pm)
Same here, all tests failed in FF3.0.6.
(And that red background is just an eye-killer. heh)
John Resig (February 12, 2009 at 1:01 pm)
@Rodrigo, Charles, and James: Naturally, that’ll be the case. Firefox 3, Opera 9.6, and Internet Explorer 7 don’t have an implementation of the selectors API, so they’ll all fail. This is something new that’s coming in the upcoming versions of the browsers.
@Ryan: The overhead is significant – depending on the implementation it can be quite large (for example, Prototype’s overhead is about 100% over native – see the ‘improvements’ link for more details).
In the case of jQuery/Sizzle and the use of the Selectors API in IE 8 – we definitely use it. The performance gains that it provides are just too great to ignore, so we just kind of have to bite the bullet and hope for the best.
Mislav (February 12, 2009 at 1:02 pm)
To clarify: this is a test suite ONLY for new and unreleased versions of modern browsers.
Most current browsers will have 0% because they lack the Selectors API support.
John Resig (February 12, 2009 at 1:10 pm)
@Mislav: Well, except Safari and Chrome, which have been shipping the Selectors API code for a while now. But yeah, this is mostly for the upcoming releases.
Ryan Breen (February 12, 2009 at 1:13 pm)
Mislav, the best I’ve seen from a current browser version is Safari 3.2.1 which gives me 81.7%.
It’s exciting to see that big jump in the Nightlies. Ship! :-)
David Smith (February 12, 2009 at 1:23 pm)
@Ryan Breen: I believe Safari 3.1 did better actually. 3.2 shipped a regression in case sensitivity (fixed in trunk, but hasn’t shipped in Safari yet) :(
Charles Lawrence (February 12, 2009 at 4:27 pm)
Doh, would have helped if I read the “API” part. heh
David Smith (February 12, 2009 at 5:07 pm)
Turns out I was wrong about that. It’s just that the test has expanded since I tried it with 3.1 :)
Jeffrey Gilbert (February 12, 2009 at 5:28 pm)
It’s amazing to even see that many tests on one page. That’s no overnight project to come up with that many selector tests. I’m curious how many people and over what time frame it took to generate all those unique testing scenarios and if it was just you, what planet you come from :)
David Smith (February 12, 2009 at 8:22 pm)
ok, trunk webkit builds (which will be the upcoming nightly build) should get 100% now :) hooray test suites.
Wedge (February 13, 2009 at 2:45 am)
Chrome 2.0.162.0 (dev channel): 99.3%
User (February 13, 2009 at 8:28 am)
Sorry for off-top, but what do you think about jQuery.ru?
Pelle (February 14, 2009 at 3:09 pm)
That test case includes plenty of CSS 3 selectors – right? And IE 8 will only support CSS 2.1 selectors as I understand? If we would filter out all of the errors which can be tracked down to CSS 3 selectors – then how would IE 8 compare to the competition?
Does the Selectors API demand support for specific selectors? If not – would it be useful with two versiosn of the test case – one for CSS 2.1 compliance and one for CSS 3?
Kit Grose (February 15, 2009 at 11:09 pm)
As of today (16/02/09), WebKit Nightly fails only one test (#309: FAIL Inside Element: :target selector).
I love the progress in JS in both FF and WebKit at the moment, and can’t wait until the WebKit changes hit regular users.
This is a great (and voluminous) test suite though.
Bleeding Edge (February 16, 2009 at 7:02 pm)
Gecko/20090216 Minefield/3.2a1pre (latest trunk nightly build)
99.3%: 2148 passed, 16 failed
AppleWebKit/530.1 (KHTML, like Gecko) Chrome/2.0.163.0 (latest snapsnot/nightly build)
100.0%: 2163 passed, 1 failed
309. FAIL Inside Element: :target selector
David Smith (February 16, 2009 at 11:25 pm)
The “Inside Element: :target selector” test will fail if you go to http://ejohn.org/apps/selectortest/#target (including reloading the test page). Be sure you’re using the http://ejohn.org/apps/selectortest/ url instead.
Michael Mattiacci (February 17, 2009 at 12:41 pm)
I agree with Pelle. Since IE8 does not support CSS3, querySelectorAll should not work with CSS3 selectors, according to the W3C Selectors API. Perhaps a CSS2.1 test would be useful.
Jeffrey Gilbert (February 20, 2009 at 12:03 pm)
The world doesn’t revolve around what IE8 supports. Thankfully forward thinking browser vendors are making useful alternatives that continue to gain acceptance without that ball and chain weighing them down
Robert Biggs (March 12, 2009 at 11:47 am)
One thing to be aware about IE8’s implementation of querySelectorAll, it only works with CSS2.1 selectors. So, if you update your code to check for support for querySelectorAll, but the user is supplying an argument using CSS3 selectors, IE8 will not return the correct result. So, until the IE team finally supports the CSS3 selectors that Mozilla, Webkit and Opera are supporting, you will still need to test for IE to feed it a JavaScript Selector API to handle CSS3 selectors properly.
I suppose one way to make IE8’s support for querySelectorAll would be to parse the supplied selector to see if it is CSS 2.1 or 3. If 2.1, using IE8’s querySelectorAll method, otherwise use a JavaScript Selector API.
Joel Gascoigne (March 20, 2009 at 10:55 am)
That’s really useful and interesting, and unfortunately the IE8 result is not at all surprising! I’m certainly glad that IE is losing market share, for all our sake’s. This and all the amazing JavaScript demonstrations out there such as http://www.chromeexperiments.com/ is showing that we’re at a really great time for web development. There’s going to be some truly amazing things coming up. Thanks John!
Pelle (March 24, 2009 at 2:54 pm)
Just for reference – what Robert Biggs says is not entirely true – according to the spec, if I remember correctly, querySelectorAll should throw an exception when presented with unsupported selectors so it would be enough to supply a fallback mechanism for when querySelectorAll fails – no need to do any custom parsing.