User Testing: Why You Should Be Testing Your Website And How to Start

User Testing

Published by

July 19, 2017

Usually when we talk about testing in conversion optimization, we’re talking about testing hypotheses: “If we change X, we think Y will happen.”

But there’s another type of testing involved in CRO that gets far less press: user testing.

What is user testing?

No matter the product or service you sell, the best way to see if your business is actually viable is to expose it to your target audience and see how they react. Often, this process can reveal unexpected insights or offer helpful guidance to the seller.

It’s why developers deploy “beta” editions of software before launching their product, so real users can catch the bugs before the product hits the marketplace.

Websites are no different. Designing a website always involves making assumptions about what visitors will expect to see and to do on the site. And no matter the amount of research and experience they’re drawing from, designers and web developers can’t predict 100% of site visitors’ actions and expectations.

Only a trial audience of real users can pinpoint potential issues, friction spots, and other problems with a website. They’ll come up with issues you didn’t even realized your site had!

User Testing - Customer Centered Design
User Testing – Customer Centered Design

By observing how visitors actually navigate your website, you can accomplish several important goals:

  • Validate or invalidate your initial assumptions
  • Uncover unforeseen benefits of your products or new ways to approach your content
  • Find flaws and issues that interfere with visitors’ use and enjoyment of the site
User Testing Illustrated
User Testing Illustrated

Remember the old engineering maxim: “You can make anything foolproof, but the fools are so ingenious.”

User testing can save you from rolling out costly mistakes

The best example of why user testing is crucial? Imagine your ecommerce site is poised to adopt new shopping cart software. Any hiccup in the shopping cart process will hurt conversions considerably, which makes any change well worth testing.

Before you roll out the new cart for your entire audience, you’ll want to make sure that it works on a small scale, and iron out any technical or other difficulties. This way, you can uncover bugs and glitches, remove or add features to reduce friction and improve usability, and generally improve the experience before it is deployed as complete – all without risking your entire income.

User Testing - Customer Experience
User Testing – Customer Experience

User testing also comes in especially handy when you’re designing a product category menu (a common feature of most ecommerce sites). Since website owners and designers are familiar with the products they’re selling, they often organize products into what *they* think are logical product categories.

Then visitors arrive, and get completely lost trying to find what they’re looking for. Conversion rates drop, and the owners are left wondering why!

To avoid poor outcomes in situations like these, submit your changes to tests by real users.

Pros and cons of different user testing methods

Websites can be user-tested in multiple ways. One way is to set up a few computers in an office and invite random people to navigate a given website or page, while you record their experiences. This is known as lab-based testing.

A researcher will directly observe users’ behavior, record it, and make their own notes. Often, participants will also be interviewed about their experiences, impressions, and opinions after the session ends. The results are then analyzed and used to craft hypotheses for improving the site’s UX (and thus its ability to convert).

However, there are a few problems with this technique. It’s expensive, complicated, and the simple fact is that people behave differently when they’re observed. When you’re looking over someone’s shoulder, they’ll act and react differently than they would if they were alone.

User Testing - Lab Based Testing
User Testing – Lab Based Testing

Another approach is remote user testing. This method records users’ actions as they navigate your website, ranging from simple mouse movement recordings to eye-tracking and heatmapping.

Remote user testing tends to be both cheaper and more effective than lab-based testing. It doesn’t involve the physical presence of another person, so testers can behave more freely.

There are numerous tools and web services that offer this type of user testing, including, UserBob, TryMyUI, and more. Most services allow you to select or specify for users who match your target audience so your test results realistically align with what you’ll see on your live site.

How many users do you need?

Now, you might think more users = better. But when user testing, you’ll only need between five to 15 test participants for accurate results. In fact, just 10 testers will likely uncover 90% of your site’s issues.

Interestingly, for every additional user above 5, the number of additional issues identified drops off sharply. Plus, the more testers you recruit, the more data-intensive the analysis process will be and the longer it will take.

The best practice is to employ several batches of user testers, and test sequentially. In the first step, you’ll test with 5 users, then solve the issues they identify. Next, you’ll repeat the process with another 5 users. Do this until you get to the point where users can’t identify any issues.

Select your testers from the audience most likely to understand and appreciate your content – AKA your target audience. If your site offers products or services meant for a specific niche, recruiting a random web surfer as a tester won’t serve you well. You’ll get meaningless observations and “find” issues that don’t even exist for your target audience.

Run effective user tests with the process below

Truly informative tests don’t happen by accident. They begin with a plan.

Once you’ve chosen your testing method and testers, it’s time to form a plan for the most effective way to run your test. This is your test protocol, and it consists of the tasks you’ll assign your testers to do.

Prioritize testing the most important areas of your site – those that make you money!

This means your tasks should typically involve interacting with product pages, buying products, or signing up for your demo or newsletter (the big conversions you’re asking for). The most useful tests will always be those that require the tester to go all the way through the conversion funnel.

To create your protocol:

Design tasks around a specific objective, such as “Find and buy the most popular product in [Category Name].” Or, “Find a product that you like.” Or even, “Buy something from the site.”

Avoid asking abstract questions, like “Do you feel secure enough to buy something here?” The point of this exercise is to observe people as they behave, as they do (or don’t do) something.

Ask your testers to comment out loud as they go through the site, and make sure not to put words in their mouths. If they mention that something is an issue, take them at their word. If they don’t mention an issue you think is there, don’t suggest it to them! (It’s probably not as big an issue as you think.)

To test every part of the shopping experience, your test protocol must include payment. Most payment services allow the use of dummy credit card numbers for test purposes. Inserting dummy numbers enables testers to complete the purchase so you can see if and how everything works.

Watch out for these user testing pitfalls

While it can provide invaluable feedback, user testing is far from foolproof. Here are the four most common pitfalls to be aware of, plus how to avoid them:

  1. Sample selection mismatch: Always select your testers to match your target audience.
  2. Forgetting your mobile audience: Don’t forget to include mobile audience testing! Mobile visitors are a quickly growing segment for ecommerce stores. Ensuring that mobile users enjoy an optimal experience future-proofs your website (and will increase conversions).
  3. Taking user results as gospel: The act of observation alters the behavior of the observed. It’s practically unavoidable. So take user testing with a grain of salt, and always confirm your qualitative results with quantitative data.
  4. Inaccurate representation of risk: Testers will not behave exactly like real users do. The best way to see this is by user-testing a website that offers high-value goods or services. Real visitors will often have issues paying $500 for a product, while testers won’t, since they’re not risking their own money.

While these limitations are very real, they in no way make user testing pointless. Far from it. User testing is the only way you can directly observe and hear visitors’ opinions as they navigate your site.

Sure, heat mapping and surveys are helpful, too. But they also remove you from the people giving feedback, so you can’t be sure that your assumptions about the data match what was actually going on in those users’ minds while they were scrolling or answering survey questions.

The user-testing experience can be eye-opening for precisely that reason. What you may think of as the “best” user interface design may turn out to be clunky and difficult to navigate in the hands of real users. Without the raw input of actual humans, it’s too easy to assume that your “perfect” design isn’t the problem – it’s your users! (Hint: it’s probably your design.)

User testing allows you to spot usability issues and fix them easily, in a way no other type of testing can duplicate. The end result is a user-centric design that even brand-new users can easily understand and navigate.

Straight talk is invaluable

When you’re optimizing your ecommerce website for conversion, user testing isn’t optional – it’s a must. There’s nothing like insights from actual users to reveal and remove obstacles standing in the way of conversion!

While surveys can uncover issues that users are aware of and can verbalize, user testing helps optimizers spot issues that users cannot accurately recollect or explain well. Simply by providing real-time feedback as they navigate your site happens and complete assigned tasks, your users will provide you with countless insights.

You can find more useful guidelines for user testing here, developed by the web usability authorities at Nielsen Norman Group and based on their years of experience with user testing.

Series Navigation<< How to Use Customer Research to Amplify Your CRO EffortsUser Survey Guide: If You’re Not Conducting Surveys, You’re Losing Conversions >>

Published by

Edin is a Senior CRO Consultant. Edin is into Google Analytics and testing (any A/B testing tool really) and likes to write about it. You can follow Edin on Twitter.