Monday 7 May 2007

Split testing web pages

Why test your web pages? If you don't, they are unlikely to be as effective as they might be.

As Sumantra Roy says about the art of making landing pages: there's no one right or wrong way and you can't tell what works except by testing.

Split testing web pages is much more complex technically than testing email. It's easy to see how one can create different versions of an email article and send them to subsets of a mailing list, but how do you achieve the same effect on a web page with a single URL?

There are a number of different techniques available but each will generally provide the following:

  • A way to serve different content to different visitors.
  • A way to identify visitors who've come to the site before (so they see the same content each time)
  • A way to record which content was served and whether it resulted in the desired effect (conversion) or not
  • Statistical tools to analyse the results
The simplest form of test is what is called "univariate": one where only one thing changes. You might for example test only the headline on a page. This is very restricting, but the analysis is mathematically very simple, and you don't need many results to get a reliable conclusion.

The alternative is a "multivariate" test, in which several things can change at once. For example you might have three versions of a headline, two of a product image and four of a call to action. Because of the number of combinations to be tested the analysis is more complex and you may need many more results before a reliable conclusion can be reached.

Some tools will allow you to vary the proportion of tests in which a particular variant is shown: this feature is not particularly useful from an analytical point of view, but if you are testing something radically different from the norm you may wish to limit the risk involved by only showing it to a small percentage of visitors.

How to pick a split testing tool? Don't be seduced by a feature list or pricing: there are three key questions you should know the answer to before you make a choice:
  1. Is this software technically compatible with your site? (Some tools don't play well with dynamically-generated content, for example).
  2. Do the staff who you expect to operate it have the requisite skills? (If your marketing people don't know HTML, for example, is that going to be an issue?)
  3. Will it interfere with search engine rankings? (Constantly changing content can cause problems with search engines, for fairly obvious reasons.)
In the next few posts I'll look into some of the issues around these and other questions, and how some popular split testing tools stack up.

1 comment:

Unknown said...

These are really nice techniques for the split the web pages!

- J.
Web Marketing