E-commerce A/B Testing

Out of the entire conversion optimization process, A/B testing gets the most attention. It’s the star of the show, but like all stars, it needs a strong supporting cast to put on a great show. Properly conducted, testing takes out the ‘guessing’ part of improving a website and replaces it with ‘just try it out’ approach.

The hypotheses we test are based on a foundation of customer research, so that each iteration – every change we make – is based on real data from your target customers. That means each hypothesis is more likely to work, shortening the time between testing and measurably increasing sales.

How our testing process works

A simple A/B test compares one web page to a second version of that page, driving equal amounts of traffic to each. All tests run for at least one business cycle (and preferably 2-3, to eliminate all external influences like holidays, etc.). We use tools to estimate necessary sample size in advance so that each test complies with strict statistical requirements (and avoids the common pitfalls of shoddy testing).

Far from being the final step, A/B testing is often the starting point of a more ambitious optimization program.

Testing is an iterative process. The more you do, the better results you get. With each test, we learn something new about your customers that helps us get them where they want to go.

  • Each test we conduct feeds into the next test, leading to better results.
  • Conversion optimization is an ongoing process and testing is just its pinnacle. Testing serves to verify that the hypotheses are sound and to point you (and us) in the right direction – up.
  • We employ a research and test-driven approach to obtain the best results for you.

Tools we use

We use Optimizely, Visual Web Optimizer, Google Optimize and others (have a preference? Just tell us!).

Google Optimize A/B Testing
Google Optimize A/B Testing
Visual Website Optimizer A/B Testing
Visual Website Optimizer A/B Testing

Key deliverables:

  • List of treatment hypotheses, prioritized based on potential impact and ease of implementation
  • A complete testing program derived from hypotheses with a proposed testing timeline
  • A mock-up of design, specified variation, expected result, necessary sample size and proposed duration, for every test.
  • Weekly test reports and a final test program report, listing all the results and improvements achieved
  • Estimated ROI of the entire test program – so you’ll know exactly how much value we’ve delivered