Genetic A/B Testing with JavaScript

I’ve long been interested in the concept of A/B testing (Also called split testing). It’s a simple concept that should sit will with most mathematically-inclined types: You have a baseline interface in which you adjust a single variable, at random, for each user that visits your application. After a given amount of time you should be able to see if certain variables affect how your users behave (either negatively or positively).

A product was recently released called SnapAds which allows its users (advertisers) to permute different variations of an ad and display different versions to users, based upon how well they perform over time.

But that’s not what I was interested in, specifically (even though it is a cool idea). The team that created this also created another product a while back that never saw a full release: Genetify. Genetify provides developers with a JavaScript library for doing any number of A/B tests on a site (tweaking CSS, JavaScript, or HTML elements) all trained over time using a Genetic algorithm backend.

This means that no matter how many different A/B tests you have on a page the genetic algorithm will adapt to the input (users visiting the page and hopefully achieving some pre-defined goal) and slowly show a more-optimal page layout to the user.

Genetify provides a demo on their site showing the basics of how it works along with a simple text tutorial.

To get started with Genetify you will need to include the library in the head of your page along with a couple CSS rules.

<script src="genetify.js"></script>
<style> 
  .v { display: none; }
  .genetify_disabled { display: none !important; } 
  .genetify_enabled { display: block !important; } 
</style>

And then before the closing body tag on your site include the following:

<script>genetify.vary();</script>

Here are some examples of different ways in which a page layout can be changed using Genetify.

HTML Elements / CSS Classes

The easiest technique is one which allows you to simply toggle HTML using some inline CSS classes.

<div class="sentence">One way of saying something</div>            
<div class="sentence v anotherway">Another way of saying something</div>

The first class specified ends up becoming the name of one of the “genes” which is used to train the genetic algorithm. Thus if the user completes a specified goal while the “anotherway” element is toggled then the algorithm is trained to recognize that showing the “anotherway” element might be more desirable and will show it more over time.

A goal can be recorded by specifying a goal name and a weight for completing the goal. You’ll need to call a JavaScript method that records the completion of a goal wherever in your code you think the goal was completed (such as the user signing up for something).

genetify.record.goal('signup', 100);

The next-most-common technique will likely be that of toggling CSS rules. Multiple rules are defined using a similar name but with the addition of a simple alphabetical name on the end which is used for categorization.

#navbar { color: red; }
#navbar_vA { color: green; }
#navbar_vB { color: blue; }

Note that you can have any number of rules – you aren’t limited to the traditional “A/B” style of testing where there’s only two options – specifying any number of rules will continue to yield results.

Finally Genetify provides the developer with the ability to toggle JavaScript variables.

highlight = function(elem){
  elem.style.borderColor = 'green';
}
highlight_vRed = function(elem){
  elem.style.borderColor = 'red';
}

I’m less excited about this particular technique – it’s kind of clumsy to clutter the global namespace with variables and to expect a changed output. I think a better technique would be to toggle a property value within the genetify object and check how that’s changed, instead.

Perhaps the biggest question that surrounds Genetify, right now, is over its longevity and openness. It seems like the project has taken the backburner in favor of the team’s other project, SnapAds (and understandably so – since that has an obvious revenue stream). Although a recent comment by its creator, over at Hacker News has fueled speculation: He’s looking for interest in the possibility of open sourcing the codebase for anyone to use.

Right now Genetify is two components:

  1. The JavaScript frontend that does the A/B rotation of site components. Currently this file is only available on the Genetify demo site as a doubly-packed file (using Dean Edwards’ Packer). It wasn’t too hard to un-pack it and run it through a JavaScript beautifier in order to get some sane output. Of course that doesn’t make it any more “open source” – just developer-readable. You can now tweak the code to communicate with the server of your choosing, instead.
  2. The Genetify backend is unclear at this point. It appears to only exist on the Genetify servers and it’s not clear if the backend will accept input from non-Genetify domains. If the intent of the team holds true then should probably see this code become available soon.

I’m already quite excited about this utility. I think it shows a lot of promise for developers who want to roll their own A/B testing solutions. I hope the team comes through and releases an Open Source solution that developers can really start to hack on it.

Posted: November 25th, 2008

Subscribe for email updates

25 Comments (Show Comments)


Comments are closed.
Comments are automatically turned off two weeks after the original post. If you have a question concerning the content of this post, please feel free to contact me.

Secrets of the JavaScript Ninja

Secrets of the JS Ninja

Secret techniques of top JavaScript programmers. Published by Manning.

Subscribe for email updates

John Resig Twitter Updates

@jeresig / Mastodon

Infrequent, short, updates and links.