A big push of Firefox 3 has been to improve its overall performance (memory, speed, ui responsiveness, JavaScript, etc.). One thing that I wanted to see get a little love was the performance of timers (setTimeout and setInterval). However, in order to make my case, I had to do some analysis.
I built a super-simple test case: Moving a div, one pixel at a time, 100 pixels. I did this using setTimeout, setInterval, and using Mozilla’s internal nsITimer interface. I wanted to try and get an accurate reading on a couple things:
- How long of a delay is there between timer calls?
- How much fluctuation is there on timer calls?
- How well do multiple simultaneous timers scale?
- What is the minimum delay achievable with a timer?
Test Case:
All charts were rendered using the jQuery chart plugin, Flot.
The Results (OSX):
Key: The horizontal axis is the recursion depth of the timer call, vertical axis is the actual delay between the timer calls – in milliseconds, and the line numbers correspond to the number of simultaneous timers being called.
The graphs are organized as such: (From top left, going clockwise) Firefox 2, Safari 3, Opera 9, Firefox 3.
setTimeout: 0ms delay, 10ms delay, 20ms delay
setInterval: 0ms delay, 10ms delay, 20ms delay
nsITimer: 0ms delay, 10ms delay, 20ms delay
Modified Firefox 3: Now, after having run all of those tests, I went through and commented out the timer filtering code in Firefox to see what the results would be. View full results (from left-to-right, nsITimer, setInterval, setTimeout, top-to-bottom 0ms, 10ms, 20ms delays).
Conclusions (OSX-only)
- WebKit’s timer code is the thing of dreams. Their results are as smooth as a baby’s bottom. Simply incredible.
- Opera is really messy – all over the place in its tests.
- Firefox 2 is generally pretty stabilized, but has nasty spikes in delay (from where the garbage collector kicks in).
- Firefox 3 has less nasty, long, delays – but has weird dips down to 0-2ms (this should probably be looked into).
- Firefox 2, Opera, and Safari all have a bottom window of 10ms for delays – Firefox 3 is now 15ms, for some reason (this should probably be looked into).
- Once you start moving into the range of 64-128 simultaneous timers, you’re pretty much out of luck in most browsers.
- nsITimer can’t be beat (for speed) with 1-2 simultaneous timers.
- There’s something really wrong with nsITimer in Firefox 3.
- Removing the timer filtering code (in Firefox 3) reveals that there’s a significant amount of timer logic located in the DOM code – as opposed to the pure timer code.
As noted above, these tests were run on OSX only. If anyone is interested in producing some results from Internet Explorer, Firefox 2, Firefox 3, and Opera, you’re welcome to it. In the meantime, Vlad produced some basic results from Firefox 3 on Windows XP (setTimeout, setInterval, and nsITimer). Note how different they look from the OSX results. It may be a safe bet that some of the above results, and conclusions, may be localized to just the OSX platform.
I’m hoping to spawn some discussion from this, to better figure out which of these issues are actually bugs and need to be resolved (in particular, in Firefox 3). Don’t worry, Webkit team, you guys are completely off-the-hook.
Scott Schiller (December 19, 2007 at 12:00 am)
Great digging, John. 20ms was the de-facto standard for JS animation until some time ago; looks like Safari can handle much lower timings, and quite reliably. Nicely done.
(I’ll wager that IE will turn out to be pretty good on Windows for timing reliability, and Firefox 2 much less so.)
Peter Kasting (December 19, 2007 at 1:14 am)
I know on Windows, system timers don’t have a resolution finer than 10ms (unless you set up your own special timer code to use high resolution clocks or something), but why on OS X would everyone have a 10ms floor? (This question may be naive; I don’t code on OS X, so perhaps it has similar limitations.) Is there some JS or DOM spec somewhere that says that timers must not fire sooner than 10ms, or sites that have been found to break if your timers have finer resolutions, or something?
Darren (December 19, 2007 at 1:31 am)
I believe Safari sets the minimum to 10ms so as to match IE’s behaviour. There is a lot of javascript code out there that sets setInterval to 0, which works fine in IE because 10ms is the lowest on Windows. If other browsers/platforms allowed timers to fire quicker than 10ms then stuff will break.
Dave Hyatt (December 19, 2007 at 1:54 am)
WebKit’s timer code actually uses a sliding scale. It will honor intervals less than 10ms until it detects that you are “looping”, i.e., setting a new timeout from within a timeout callback. At that point it will start using 10ms to avoid hogging the CPU. This allows one-shots to fire very very quickly while still allowing repeating timers to be throttled effectively.
John Resig (December 19, 2007 at 6:24 am)
@Scott: It definitely looks like 10ms is the new de-facto across browsers – which is really good to know (theoretically allowing you to have ~100fps animations).
@Peter: I’m not sure about the limit on OSX either – but I think (as Dave also mentioned) that it most likely has to do with making sure that the CPU isn’t hogged excessively. At least with a delay of 10ms a web page could still function. Plus, if Windows has a limit at 10ms, I assume it’s easiest to just keep it the same across platforms.
@Dave: Thanks for the clarification – I didn’t realize that you could go less than 10ms in WebKit, that’s good to know.
Jake Archibald (December 20, 2007 at 7:52 am)
Since nsITimer is quicker than setInterval, is there a way to get around the permission stuff and use it on a standard web page?
Jake.
Gregor Petrin (December 20, 2007 at 1:25 pm)
I ran the tests under XP. Internet explorer couldn’t run the tests, the red dot was basically stationary, if a little higher than it should be.
Then I compared Opera 9.23, Firefox 2.0.0.11 and Firefox 3 Beta 1 (didn’t install Beta 2 yet, hope it doesn’t matter too much). Opera was much smoother on XP, but I have no idea what was happening with Firefox 2! I ran the tests on my work machine as well just to see what it would look like there and it was the same. The results are at http://freeweb.siol.net/gregopet/timers/timerTestsXP.png
Gregor Petrin (December 20, 2007 at 3:03 pm)
And here are the Linux test results (same computer as the XP tests, the Linux distro is Kubuntu Gutsy).
Same browsers as before, Konqueror could run the tests (unlike IE under XP) but the results wouldn’t show up. Here are the results:
http://freeweb.siol.net/gregopet/timers/timerTestsLinux.png
Moran (December 20, 2007 at 10:20 pm)
I noticed you’ve used PlotKit in the past. Any significant reason you used Flot this go around?
John Resig (December 22, 2007 at 3:51 pm)
@Moran: A couple reasons. The reason why I tried it, in the first place, was that it’s built on top of jQuery (easy decision). However, I’ve been immensely pleased with its configuration setup and its ability to handle random datasets. I was able to construct all of these graphs without any sort of rejiggering of the internals (which hadn’t been the case with PlotKit in the past). I’m quite pleased.
Moran Ben-David (January 2, 2008 at 8:57 am)
Thanks John. Love your insight, blog and projects!