The Objeqt Optimization Blog:

Actionable Posts to Help You Improve Your Conversion Rates

Whether you’re just getting started with conversion rate optimization or you’re an old pro, check out our library of how-to-posts, guides, and more. Click the topic you’re interested in now!

Innovative Testing: How to continue growing revenue – even after you tested everything?

Innovative Testing

Published by

October 24, 2017

This isn’t your first rodeo. You started the conversion optimization process a while ago. You’ve analyzed your website, identified issues, and came up with a bunch of things to fix and hypotheses to test.

And you tested them.

Again and again.

And many of those tests you ran resulted in gains – maybe large, maybe small, but improvements nonetheless.

Finally, after you ran tests on every aspect of the website, you reached a point where the law of diminishing returns kicked in. You began noticing it in inconclusive results, and then perhaps even in tested variations producing negative results. You create a few more hypotheses to test, but to no avail.

No matter what you tried, it just seemed that there was no way to nudge your numbers up.

Have you peaked? Should you rest on your laurels, put up your feet and pat yourself on the back?

Heck no.

There is always potential to improve.

While the process of iterative (A/B and multivariate) testing is the best method to follow for long term, repeatable results in conversion optimization, it has one limitation. After testing every part of the website, you will inevitably reach the point where there is no way to further optimize your site.

Which means it’s time to step out of iterative testing and try innovative testing.

Innovative Testing Example

CrazyEgg had a pretty good website. User testing didn’t reveal any outstanding issues, the web page was relatively optimized, and there weren’t any real problems to fix. No obvious wins to be had either.

And yet, after doing research, the consultants decided that major changes were required. And no, they weren’t just trying to get more money! It was the only way to create a major lift in conversions.

The team decided, based on heat maps of the homepage, that a major redesign was necessary to remove a few distractions that they detected, and improve the copy in general. They implemented these changes and tested them – resulting in a 26% increase in conversions.

That was innovative testing.

How does this apply to you?

When you test and optimize everything you possibly can, the phenomenon you may encounter is called a local maxima.

From this point on, unless you change your initial assumptions and testing scenarios, you cannot expect a measurable increase in conversions. Graphically it is represented like this:

Innovative Testing - Local Maxima
Innovative Testing – Local Maxima

So what does this mean? It means that you are limited by a number of factors, which we call a paradigm limitation. Your current design, target audience, services and products have, essentially, hemmed you in. You’ve done as much as you can do with the paradigm as it stands you’ve reached your limits.

And you’re stuck – stuck in a box.

Of course, the solution to finding yourself in a box is to break out of it – which requires thinking outside of it. When you are so close to the project, or the business, this can be incredibly hard to do.

How to get out of the box

Paradigm limitations happen as a result of making choices. Each decision narrows the playing field. Which is good – that’s how we create valuable niches and develop followings who know what to expect from us. It’s how businesses are built. But, those early decisions also anchor you to your spot.

I’m not creating a fancy metaphor here – anchoring is a cognitive bias, one which influences people to rely too heavily on the first piece of information offered (the anchor) when making decisions. Once an anchor is set, other decisions are made based on treating that anchor as unalterable fact.

Now we can get metaphorical – because we need to break the chains tying you to your anchor.

And that is going to require a substantial, possibly radical change.

“But won’t this ruin my business?”

Maybe. But here’s the deal: You can have what you have now, or you can risk it for something more.

Or, as Henry Ford put it:

“If you do what you’ve always done, you’ll get what you’ve always gotten.”

(Clearly, taking a few risks worked well for him!).

In practical terms, this means that you might need to redesign your site, or at least a substantial part of it.

On a more strategic level, you may need to adjust your value proposition to appeal to a wider audience, or different target audience. Or you may need to expand your products or services, like when Starbucks began offering food. Whatever the eventual choice is, it will likely mean making a transition from one concept to another.

The choice of target audience, product range or target market is outside of the scope of the conversion optimization. These kinds of decisions can only be a business decision of the owner. But a radical redesign of the website? That is something that conversion optimization methods can help with.

How conversion optimization can help

Conversion optimization generally can solve the local maxima problem using two main methods:

  1. Innovative testing
  2. Radical redesign

Both methods mean taking bold steps forward and abandoning iterative testing.

Up to this point, conversion optimization took the safe route of incremental improvements that, over time, lead to significant growth in a gradual step-by-step approach. From here on, we’ll have to rely upon quantum leaps forward, which are risky, but, if successful, highly profitable.

Quantum leaps can deliver increases in conversions that are comparable to, if not better than, the early phases of low hanging fruit solutions (those quick and easy wins that I’m sure you miss!).

Innovative testing with conviction

Innovative testing means testing for larger changes on the website. That can mean changing multiple and/or major elements of design of a single page (a landing page, for example) or even changing entire portions of the web site.

These tests should be approached with caution.

Before you embark on them, you should ensure that ample research data is available. Most importantly, this is where the qualitative research will be the most useful.

Whereas iterative testing can produce many ideas for tests and improvements from insights yielded by quantitative data and a limited amount of user feedback, innovative testing relies heavily on user feedback.

The difference here is primarily due to the effort that must be invested to make substantial design changes. User input will enable you to design the website or parts of it to match with what your users expect and want.

User testing, interviews, live chat transcripts, surveys and polls can all provide insights into exactly what your visitors want, why they want it and how they want to achieve it. If you follow the voice of customers/visitors data, you will be able to serve them get them to really engage with you.

But don’t expect your users to deliver a new website design to you – they’re not designers. They may not even know what they want, or what’s possible to want. Not to quote Ford every time, but he did say “If I’d asked people what they wanted, they would have said faster horses.”

The man said a lot of smart things.

To overcome the “faster horses” conundrum, your job will be to not only gather their answers in surveys and interviews, but to verify those answers based on what they actually do and how they actually use your site.

And, you’ll have to use a little intuition.

As UI expert Anders Toxboe says:

You innovate through intuition. When we use our intuition, we make best guesses and rely on our previous experience. We study what others are doing and use best practices. An integral part of innovation is thinking experience, aesthetics, and flow as a whole. When innovating, we rely on designers, best guesses and discussion as instruments for deciding upon what is the best design. The goal of innovation is to maximize the potential improvement through optimization. Optimization vs innovation, blog post.

So the answer in successful innovation and breaking out of the local maxima lies in maintaining a delicate balance of intuition and a data driven approach. Keep in mind that the stakes are high – innovation is the only way to grow, and often, the only way to remain competitive and survive in the long term.

Alex Birkett of ConversionXL put it best:

Whatever the case, when the time comes to make bigger changes – when you decide to jump from your local maximum to another possibility – make the decision with conviction.

Radical redesign

Radical redesign is innovative testing writ large.

Radical redesign requires that you depart from the 100%-data-driven approach and allow the designers to freely create the features and layout that they feel comfortable with.

It’s the riskiest approach of all.

The potential for a costly mistake is very real, as is evident from the debacle of Marks & Spencer’s website redesign in 2014. The trick is to submit the new design to some form of user testing. Select real users to see if the redesign works better and solves the problems that could not be addressed with the old design.

The best way to test a redesign is to release the improved version for limited user testing before going live to prove the concept.

One way to do this is to use a beta version of the site and offer a limited sample of users to opt in and analyze the performance of the variation. This can be a great way to engage your customers and make them feel part of the process.

Or, you can offer the redesigned version only to new users to see how they react.

If you have a large site with sizeable traffic, radical redesign can be evolutionary and phased in over an extended period of time, which allows you to test specific aspects of the new design separately.

Any way you choose to do it, make sure you are not alienating existing visitors.

But is it really absolutely necessary to redesign the entire site?

Nielsen Norman Group provides this list of reasons for radical redesigns:

  • The gains from making incremental changes are miniscule or nonexistent
  • The technology is severely outdated, making critical changes impossible
  • Architecturally the site is a tangled mess
  • Severely low conversion rates site-wide
  • Benchmarking research reveals your site is far inferior to the competition

Source: Radical Redesign or Incremental Change?

Before taking the leap, carefully weigh out the potential benefits of implementation of the new design and the benefits you expect to gain.

Innovative testing conclusion: Don’t try this unless you’ve tried everything else!

Innovative testing and radical redesign are the methods of last resort. They should be used only once you determine that the current website has reached its absolute limits and cannot be significantly improved.

Before you commit to the redesign, do a full spectrum of qualitative analysis to ascertain that redesign will not actually worsen the performance of your website. Every aspect should also be tested to ensure that the visitors will react favorably to changes. Your long-term customers may have negative reactions to change, especially if it is sudden and radical. The best way to deal with that is to retain some level of familiarity wherever possible. In order to be sure do customer research and testing. You know the importance of customer research?

But perhaps the biggest challenge you’ll face is inertia and that anchoring bias. The key here is to have a strong foundation in iterative testing already in place, so that the testing mindset is established. That way, the organization will have confidence in the outcome of the conversion optimization methodology and the expertise necessary to apply it in the process of redesign.

The innovative testing and redesign require not only boldness but also a certain amount of caution. Finding the right balance between these two traits will make the difference between the successful redesign and a costly debacle.

However, if your organization has adopted testing and conversion optimization and had success using it, you probably have what it takes to successfully conduct the redesign. Just keep in mind all the lessons of conversion optimization and always keep testing. The end result will transform the local maxima from the box at the beginning to an entirely new landscape.

Innovative Testing - Radical Redesign
Innovative Testing – Radical Redesign

Published by

Edin is a Senior CRO Consultant. Edin is into Google Analytics and testing (any A/B testing tool really) and likes to write about it. You can follow Edin on Twitter.