At Pivotal we are all about rapid iteration and continual feedback for our code, and try to get our clients to get excited about this same mode of thinking with respect to their products. To make that a bit easier, I’ve been working on a simple ruby wrapper for the Google Website Optimizer that allows you to perform split tests with just a few lines of code.
To get my feet wet I’ve set up a simple experiment on this blog — http://pivotallabs.com/blabs — and need a few more participants to get interesting data. If you get this is an RSS feed (you know who you are) and haven’t been to the site in a while c’mon by and check it out. In addition to a silly split test, we’ve been working on the site in general and there are some cool new updates.
I hope to follow up with the code and results of this test in another post. Meanwhile, I’m interested in experiences other folks have had with split testing. What kinds of tests have you done? What tools did you use, or did you roll your own? What kinds of patterns did you discover for managing multiple tests from a development and deployment perspective?
hmm interesting service, i gotta try this out, seems to be a good solutions to roll the dice when a team cannot decide, or for boring ROI research :)
February 12, 2009 at 6:32 pm
Parker,
Is there a supported Google API for the website optimizer? Is it rolled into another API? Analytics? AdWords? Or are you doing this “by hand”?
February 13, 2009 at 7:29 am
Can I make a site suggestion? How about individual talk pages for your pivotal lunch talks? I’d prefer to link to a page for “Making a Case for Cucumber” so you guys get credit where credit is due rather than linking directly to a .mov file. (and if your feed linked to that page as well that would be thuper).
While I am at it. Your comment page requires JavaScript – the name, email, URL and captcha fields don’t show up if javascript is blocked by noscript and there is no indication that they are missing until you go to submit and get errors.
Whine whine whine.
I do like the idea of experimental web site design though.
February 13, 2009 at 10:28 am
@Camron – it’s all JS, which has pros and cons. I hear nuconomy has a great API for tracking everything, so I’d look at that if you need more than Optimizer offers.
February 13, 2009 at 7:30 pm
Parker,
Yeah I was wondering how you actually configured the test. But from your second post it looks like you just configured it via the web, and copied out the test’s ID into your code. Is that right?
February 14, 2009 at 11:13 pm
Hey guys, just wondering if you ever followed up on this post? We’re trying to implement this on a template basis rather than individual pages.
I.e. change the header & measure the changes in conversions from 1 style to the next & so on. Then start measuring individual elements once we’re satisfied the actual layout is working for us.
Would love to see your code for this.
Cheers
Stu
June 1, 2009 at 6:07 am