In my work with the Firebug team over the past couple months I’ve been working with Jan Odvarko on a way to provide some form of unit testing that we can build off of. The result of my work is a new Firefox/Firebug extension called FireUnit.
FireUnit provides a simple JavaScript API for doing simple test logging and viewing within a new tab of Firebug.
For example, here’s some of the API that you can use (we’re starting with the basics and looking to expand with more methods, later).
// Simple true-like/false-like testing fireunit.ok( true, "I'm going to pass!" ); fireunit.ok( false, "I'm going to fail!" ); // Compare two strings - shows a diff of the // results if they're different fireunit.compare( "The lazy fox jumped over the log.", "The lazy brown fox jumped the log.", "Are these two strings the same?" ); // Compare a string using a regular expression fireunit.reCompare( /The .* fox jumped the log./, "The lazy brown fox jumped the log.", "Compare a string using a RegExp." ); // Display the total results fireunit.testDone();
The results will appear in a ‘Test’ tab in Firebug (which must be installed in order for Fireunit to work). Each of the results can be expanded to show additional information including a full stack trace of where the test ran and a comparison with a diff.
FireUnit also provides a couple methods for simulating native browser events:
// You can also simulate browser events var input = document.getElementsByTagName("input")[0]; fireunit.mouseDown( input ); fireunit.click( input ); fireunit.focus( input ); fireunit.key( input, "a" );
And a way of running a batch of test files (each of which would contain a number of individual tests).
// Or run multiple pages of tests: fireunit.runTests("test2.html", "test3.html"); // Place at the end of every test file in order to continue fireunit.testDone();
We’ve been using this test runner to run a number of Firebug tests, especially ones that are network based.
Depending on the suite it’s pretty easy to adapt existing test suites to display their results in FireUnit.
Running the jQuery selector test suite that has the following snippet added:
if ( typeof fireunit === "object" ) { QUnit.log = fireunit.ok; QUnit.done = fireunit.testDone; }
Yields the following results (which are completely navigable):
If you want to get started using FireUnit you can head on over to the Fireunit.org site and download the latest extension.
You can also grab the source off of the repository on Github.
Jan has also written a blog post detailing a little bit more about what we’re using FireUnit for in the Firebug Working Group.
This is still a very early release of our work – there’s obviously a ton of room left to grow – so feedback is expected and appreciated.
Ricardo Vega (December 18, 2008 at 12:24 am)
Whoa! Again, thanks John this is just a must! I’m in the middle of placing js unit testing to a code and well, it was being a pretty hardtime (I have a very basic java knowledge) so jsunit was being rude with me :)
Again, this is awesome.
Aaron McCall (December 18, 2008 at 12:26 am)
John,
A big thank you to both you and Jan for what looks to be a very helpful addition to the toolset of Javascript developers. And having it integrated with FireBug–icing on the cake!
Lance Fisher (December 18, 2008 at 12:36 am)
This is really cool, John. I can think of more uses than just unit testing. This is nice. This can give you end-to-end automated integration testing.
Gregory (December 18, 2008 at 12:37 am)
Nice work! This actually makes JS Unit testing possible without being a HUGE pain in the ass.
Love it. Will it get put into Firebug do you think?
thecancerus (December 18, 2008 at 12:37 am)
thanks.. after jquery this is something i am going to use everyday :)
Rock.yin (December 18, 2008 at 12:42 am)
Great Work!
Tanks for sharing.
Paul Wilkins (December 18, 2008 at 12:42 am)
I would think that it’s important for the test tab to be able to separate out the successful tests in some manner. The reason being that as projects grow, the number of tests will too.
Because nobody wants to scroll through hundreds of successful tests to find the one or two failed ones, I think that the successful and failed tests should be either grouped together in a separate tree, or have the ability to show only failed tests by default.
Mladen Mihajlovic (December 18, 2008 at 12:44 am)
Looking good, one question though: Why move away from using the good old “assert” keywork to something like “ok”? Everybody knows how asserts work so why change a good thing?
Michael Manfre (December 18, 2008 at 12:54 am)
This will be really useful for a project I’m working on.
Tom (December 18, 2008 at 1:06 am)
Will this work for those of us writing extensions? is there a unit testing solution for extension developers?
Scott (December 18, 2008 at 1:43 am)
Will we be able to easily add BDD style asserts, similar to the Scriptaculous unit test framework? Or are there plans to make FireUnit extensible?
V1 (December 18, 2008 at 3:09 am)
Seems like another must have addon for us developers :)
Nÿco (December 18, 2008 at 4:11 am)
Small request: the source code lacks a LICENSE/COPYING file, saying what license the software is published under.
Patrick Donelan (December 18, 2008 at 4:33 am)
@Mladen: I’m guessing you’re not a Perl programmer? Check out Test::More etc.. on CPAN. Personally I’d love to see even more of those concise test methods being used, is() for euqality, like() for regex string comparison, etc.. but as John showed it’s easy to create your own test API object if you don’t like their choice of method names.
Anup (December 18, 2008 at 7:20 am)
Very interesting. Will be interested to see how this proceeds.
About QUnit: I tried the snippet you had above and nothing happened on some Unit tests I had. I found I needed to upgrade the version of QUnit testrunner.js I had. I got the latest from http://docs.jquery.com/QUnit which is the same build number as the build number your selector tests uses, but still no luck. I eventually copied your actual testrunner file and it worked. I assume you made some edits to QUnit testrunner? Or, the build version has not been updated?
Robert Kieffer (December 18, 2008 at 7:32 am)
John, one of the main reasons for having a unit test suite is to make it easy to do cross-browser testing. Thus, the utility of a browser-specific solution like FireUnit seems a little questionable. Are there any plans to enable FireUnit support on other platforms? … a “FireUnitLite” perhaps?
John Resig (December 18, 2008 at 8:00 am)
@Paul Wilkins: In the test tab click ‘Options’ then de-select ‘Passing Tests’. Now you’ll only see tests that fail.
@Mladen Mihajlovic: It’s totally up to you – but I’ve seen both assert and ok used, I don’t really think the name is that big of a deal, in this case.
@Tom: This will be very useful for those creating extensions – we’ve been using it to test Firebug. I hope to have more written up on that soon.
@Nÿco: Just added a LICENSE file containing the MPL.
@Anup: Sorry about that, I still have one commit to do for QUnit – I should have it up today.
@Robert Kieffer: Sure thing, I’ll see what we can do!
lrbabe (December 18, 2008 at 8:21 am)
Why did you added the “fireunit” object instead of extending console?
It would have enabled other browser which already implement console (webkit already, opera soon or later) to extend it as well. Thus, allowing us to build such unit test, but cross-browser.
Billy Reisinger (December 18, 2008 at 8:32 am)
What is the problem that FireUnit is trying to solve that hasn’t been solved already? I’m asking because I don’t know, not because I’m trying to be a d***. I get the feeling that there was a specific thing you guys needed it to do, or obviously you would have chosen something that met your needs already.
I know a few people on this thread have pooped on JSUnit, but it is a great testing framework. One of the nice things about JSUnit is that it is relatively easy to integrate it into a build system and a continuous integration server. It provides a return value (i.e. 0, 1), can assuredly be run on any machine (that can run Java and a web browser), and can even be used to create JUnit-style reports. Also, like Robert mentioned, it can be used with all the major browsers. Lastly, it follows XUnit assertion conventions, so it looks familiar to people who have already used JUnit or NUnit (for example).
I could see if you needed to test javascript that runs outside of the DOM or something…
John Resig (December 18, 2008 at 8:49 am)
@lrbabe: We could look at that in the future – but for now we don’t want to clutter up the console object with a ton of new methods – better to have it confined within its own object.
@Billy Reisinger: We’ve been using it to test Firebug – which isn’t something that you can do with regular JSUnit (since you need to be able to dip into the browser chrome).
Kevin McDonagh (December 18, 2008 at 10:02 am)
This is really cool. If I could just add though. Instead of FireUnit.ok(), FireUnit.false() have you considered the merits in just Assert.ok(), Assert.false() Assert.false? I ask because obviously the later is more immedietly self explanatory and abides by the convention. Would it pose problems in name spacing?
Joe McCann (December 18, 2008 at 12:50 pm)
Wicked John! Congrats…again.
Adam Peller (December 18, 2008 at 1:12 pm)
Nice. Any thoughts on developing some sort of common API with what was learned from this exercise and QUnit, DOH, and any others in this space? It would be wonderful to eventually have some neutral API spread the way ‘console’ has.
Bertrand Le Roy (December 18, 2008 at 4:57 pm)
+1 to Robert: cross-browser is an absolute must. Being able to automate and log test passes in each browser is next. Still, pretty cool to have that integrated in FireBug.
Michiel van der Blonk (December 18, 2008 at 8:00 pm)
Selenium already does it all (together with SeleniumRunner). Not that FireUnit is a bad thing to have, that is. It’s still something that is more easy to run directly from the browser.
Alan Hogan (December 19, 2008 at 4:12 am)
I would like to echo the call for the ok() method to be aliased (if not renamed) to assert(). But as you said, John, not a huge deal. I would imagine it is more common, though.
Empee (December 19, 2008 at 8:35 am)
Yep Selenium IDE. I just found Michiel’s comment.
Really, isnt fireUnit trying to grow up to the same thing?
DiegoG (December 19, 2008 at 11:12 am)
Does it support nightly builds? I mean, is there any chance to run unit tests unatended, extract reports, and so on? How do you envision such scenario?
Volker (December 19, 2008 at 11:18 am)
Great work, I was on my way to write an extension myself to get access to the browser chrome for firing events and now I can use fireunit.
However, if you intend to create an XUnit test framework, in my opinion it would be nice to stick to the common interface. But that’s just my opinion.
For cross browser development, it would be great if fireunit could be used as a UI for e.g. yui unit tests, which I’m using already. (http://developer.yahoo.com/yui/yuitest/) This way, I could write cross browser tests that could be executed in fireunit when running in firefox. Then the fireunitLight would just need to provide things like the click() function that other frameworks are lacking.
Nicholas C. Zakas (December 24, 2008 at 12:12 am)
Hey Volker (and everyone),
I’ve written a FireUnit extension for YUI Test. It’s just a proof of concept at the moment, but it basically uses FireUnit as a UI to output YUI Test results. Take a look at http://www.nczonline.net/blog/2008/12/19/fireunit-extension-for-yui-test/.
Brahmana (December 24, 2008 at 2:23 am)
Hi,
This is a good thing. However I would like to know the motivation/rationale for coming up with yet another testing framework when there are already tools like FireWatir, Selenium and probably several more which I do not know. From what I know, FireWatir can do operations that require chrome permissions and hence can be used to test FF extensions also (along with . So is there something that all these tools were missing and made you come up with FireUnit?
Regards,
Brahmana.
Cory (January 5, 2009 at 8:04 pm)
Hi John!
This looks pretty cool. There are a couple of things we’d need before this is usable to us. I’ll explain why, I think this is a pretty easy feature to add and could have a huge payoff for others, even those with different use cases than me.
1) A simple reporting scheme, i.e. an overall pass/fail or count of passes/fails available through an API.
2) A way to write the result to disk for IPC purposes.
Our scenario is this: developers writing code are able to use your extension, but we have an automated continuous-integration-and-testing server that reports overall pass/failed status after every commit. In order to run our tests, we use Xvfb, run selenium, dump the selenium results to a file and have the buildbot report a red or green based on what’s in the file.
So the key is, it has to work in such a way that a headless server can run it and then report the results without an eyeball looking at the screen.
Selenium is not a unit test framework though. In fact, no unit tests work this way in our environment yet, although we’re trying to figure out whether jsunit is an option. So far, jsunit’s heavy reliance on frames makes it very hard to work with. FireUnit could convert us easily with the above two things.
Something to think about. :-) Thanks!
Tero Piirainen (January 7, 2009 at 3:32 pm)
Hello,
Thank you for this utility. On our Flowplayer website we now have a public test that uses fireunit
http://flowplayer.org/demos/test/basics.html
What I missed was the ability to tell how many tests there are so at the end it would say that x/n tests passed.
Todd Alexander (January 8, 2009 at 2:46 am)
Another great addition to the Javascript community.
LX (January 13, 2009 at 7:35 am)
Great work! What about other browsers, like MSIE?
Greetings, LX
Yok (January 13, 2009 at 9:46 pm)
Very nice! But why not display passed tests in green?
Tim Molendijk (January 19, 2009 at 8:11 pm)
Very cool. What would be an awesome addition though is a way of profiling the test runs. F.e. if I could tell it to run every test case n times consecutively and then include the profiling report.
Andrew (February 2, 2009 at 11:38 am)
Is there any way to integrate it into continuous integration?
PierreR (February 9, 2009 at 8:55 am)
Is there a plan to make FireUnit work with the latest testrunner.js (from the QUnit site) ?
To have it work I have to replace the latest testrunner (2009) with the old one (2008) from your demo link.
Thanks.
Aubrey (February 23, 2009 at 4:14 pm)
Would it be possible to make some sort of standalone tester that worked with, say, prism? I could start it up and it would hit the tests and see what passed/failed?
Cameron Westland (March 27, 2009 at 5:53 pm)
I made a simple screencast showcasing FireUnit.
http://bigbangtechnology.com/blog/post/writing_tests_first_in_fireunit
Hope someone finds it useful!
Ionut David (April 15, 2009 at 2:45 am)
Congrats for the nice work you done on the FireUnit.
In the future is there going to be a fire unit tool in which you can write unit test and run them under Opera, Safari or cursed IE6+?
Because when writing js on IE browsers is likely that it won’t work.
Regards,
David.
Nicolas (May 11, 2009 at 6:45 pm)
Nice, but does it work outside the browser? JS is not only HTML DOM…
Nicolas (May 11, 2009 at 6:48 pm)
Bah, I didn’t notice it was so tied to Firebug (in which case *of course* it won’t work out of a browser).