Conversion Research: 6 Treasure Troves of Qualitative Research You Can Access Right Now

This entry is part 4 of 5 in the series Customer Research As An Enhancer For Ecommerce CRO Efforts

Conversion research is an important part of the CRO process. Qualitative research isn’t just about visitor surveys and user testing. While both of these methods give us the opportunity to directly observe visitors’ behavior and get a glimpse of the thought process behind it, surveys and user testing suffer from a serious limitation: visitors and testers are aware that they are being observed.

And that changes things.

Read any quantum physics lately?

One of the fundamental premises of quantum theory (known as Heisenberg’s principle) is that by the very act of watching, the observer affects the behavior of the observed.

And this is where it gets weird: The same thing is true of electrons.

Conversion Research - Needless to say, finding your way around Heisenberg’s department can be confounding
Conversion Research – Needless to say, finding your way around Heisenberg’s department can be confounding

Electrons will move and behave differently when observed, literally changing their “flight paths”. The more they are observed, the more their behavior changes.

What is true of electrons is equally true of people. As soon as we become aware that our actions are observed, we can’t help but change them. There’s no practical way to eliminate this complication, but fortunately, observation doesn’t completely invalidate findings when it comes to conversion optimization.

That said, ideally, we would have access to visitors’ opinions expressed of their own free will, without knowing or caring if they are observed.

And thanks to the advent of social networks, live chats and recorded support calls, that’s an ideal we can actually attain.

Conversion research #1: “Spy” on your customers using chat logs

Support calls, live chat recordings, transcriptions, or logs are easy to get — they should already be among your resources (just ask your customer relations department).

The first advantage of these qualitative data sources is that they contain the opinions of customers who have questions about, or issues with, your service or products.

Their second advantage is that people tend to talk more freely in these channels. They specifically point out the problems they’ve observed, which may not be the case with surveys.

The cherry on top? You don’t have to offer any incentive to compel these users to provide their opinions.

If you don’t have these resources already available, put their implementation on your priority list.

First, establish a support call line and a live chat on your website. Consistent research shows that the presence of live chat on ecommerce sites increases conversions. The live chat must be manned and available at all times, or at least during periods of normal operating hours.

Conversion Research - chat logs
Conversion Research – chat logs

Live chat enables visitors to pose their issues in real-time to a live person and get answers immediately. The fact that such prompt service is available offers many benefits, in addition to resolving potential hurdles that stop visitors from converting:

  • It increases visitor trust
  • It creates a bond between the visitor and the website
  • And, in the context of qualitative research, it creates a permanent record of the issue so that it can be analyzed.
Conversion Research- support?
Conversion Research- support?

Analyzing chat logs is very similar to interpreting open-ended survey results. A good first step is to look for repetitions: Repeated words, phrases, question types and concerns. Finding common keywords allows you to do a keyword search to get an idea of how frequently each type of question is raised. From there, you can resolve the issues that come up most frequently (an indication that they are the most significant).

Support call transcripts or recordings also require the same approach. It’s all about determining categories and calculating frequency to find which fixes will have the most impact.

Apart from live chat’s conversion benefits, qualitative data is why most ecommerce sites nowadays offer live chat. Having consumer data on record saves a lot of effort (and money, too) when doing research for conversion optimization, marketing, and copywriting.

Conversion research #2: Mine useful messaging from product reviews

Most e-commerce websites allow visitors to review products they’ve bought. This feedback is highly valuable in several ways:

  1. It works as “social proof,” which builds trust with would-be purchasers and improves conversions
  2. Voice-of-customer data can be used to improve your marketing messaging and copy. Let your users tell you the most important benefits!
  3. You can use this feedback for your qualitative research

A common mistake businesses make with reviews is to limit the feedback to ratings only (usually using stars), without the option for written feedback. This type of rating is useless.

If your product or service receives many 1-star ratings, it will have a hugely negative impact on conversions — and to boot, you won’t have any understanding of why customers are unhappy. Even if you receive 5-star ratings, you still won’t know why (and neither will your prospective customers)!

Conversion Research - Sony, for example, has made user reviews extremely useful at first glance
Conversion Research – Sony, for example, has made user reviews extremely useful at first glance

Written feedback lets you know what to improve and what customers like about the product. This is why every review system should allow written feedback.

Bear in mind that not many visitors will trouble themselves with leaving elaborate feedback. This is why we recommend that some sort of incentive should be offered.

The most obvious incentive is a monetary one, like offering discounts, coupons, or other shopping benefits to customers who write a review. It’s certainly effective, and can be seen on many sites that reward reviewers with points or tokens that lower the price of a product. If it can be executed without significant impact on the bottom line and protected from manipulation, this type of system works well.

The second way to incentivize customers to leave feedback is to create some sense of achievement, but without offering any direct monetary benefits. Instead, the reviewers might be shown the number of views their reviews had and how helpful they were. They might even reach “Elite Reviewer” status.

This way, a person’s individual sense of accomplishment effectively drives them to leave detailed, helpful reviews. This is the system that Amazon.com uses with great success.

Finally, there is the option of offering access to premium features of the website or service. This is usually used by SaaS companies, which provide reviewers with features not otherwise found in the package they purchased — but Zappos also uses a similar system. Earn enough points by purchasing products and leaving reviews, and you gain access to their “elite”-level perks (like faster shipping).

Conversion research #3: Tackle research head-on with customer interviews

Customer interviews offer the significant benefit of being able to ask participants to elaborate on their answers. You can conduct these interviews either in person or over the phone.

Don’t be too quick to jump on the phone rather than do the footwork to interview in person. Remote interviews may be more cost-effective, but they remove an essential ingredient that is only available via direct contact with the interviewee.

If you’re in the same room with the person you interview, you can observe body language, facial expressions, and other subtle non-verbal communication that can reveal a great deal more about the subject than only verbal answers.

Conversion Research - Never lose sight of this fact, and take it into account when interpreting your interview responses
Conversion Research – Never lose sight of this fact, and take it into account when interpreting your interview responses

It must be noted that face-to-face interviews are time-consuming and difficult to arrange, especially if the business is online. Customers may be spread across the globe!

The best alternative in this case is the video interview. Skype, Zoom, and other videoconferencing platforms provide a satisfying middle ground for interviews, and also offer the option of recording the interview for later analysis.

To be useful, interviews must be structured and well-planned. Interview questions can be the same as in those you use in an online survey, but prepare additional questions that complement or expand upon the primary ones.

You should also plan selection criteria for your interviewees ahead of time. It’s very important to only choose respondents from your existing or potential customer base (people who have already bought something from your website).

The fact that they’ve already purchased something means that they already find value in your products or services, which makes them your target customers. You want their feedback. You don’t want feedback from people who are uninterested in what you offer, or who are chronically supportive friends or family members.

To get the best results from in-person or video interviews, they shouldn’t feel like interviews at all. There is an art to the “un-interview,” and establishing a rapport is vital.

A good interview flows like a regular conversation. It feels more natural than awkward. Interviewers need strong interpersonal communication skills to pull this off.

Conversion Research - customer service
Conversion Research – customer service

Use interviews to find out how customers use your product or service, what their anxieties and motivations are, and what their deeper emotional backgrounds might be. Interviews are all about creating a depth of understanding — going past the surface to find the emotional, psychological, and purely pragmatic underpinnings behind user behavior.

These findings will enable you to improve your copy, even to the point of using the responders’ answers verbatim as statements of your product’s benefits; in headlines or ads; and even as part of your value proposition. Successful interviews can also become testimonials, provided the respondent agrees to it.

Another use for the interview is to find out about your competitors. Questions like, “Did you consider any other product/service?” can reveal the ways your product is better than other available solutions, so you can update you value proposition with those insights.

After the fact, you can analyze interview findings the same way you interpret survey results: Comb through responses and identify the most frequently mentioned issues.

For further reading on conducting interviews, check out: Start Talking! How To Do Customer Interviews That Reveal Priceless Insights

Conversion research #4: “Eavesdrop” on social media posts

Social media channels, including Facebook and Twitter, and other places where customers spend time, like blogs and forums, yield an even greater opportunity to glean customer opinions.

Every day on social media, your customers are freely sharing their opinions of products they bought from you. This can be a source of free promotion, if those opinions are positive. And it can also be a valuable source of user feedback and voice-of-customer data (even if opinions are negative).

When people post about products voluntarily, they’re much less inhibited. They’ll tell you how they really feel in unvarnished language. It’s raw, authentic, and can provide deeper insights than even surveys or interviews (where, again, people are aware they’re being observed).

This is why we believe it’s vital to track relevant social media posts and blog mentions.

A few handy tools to monitor when your brand or product is mentioned:

  • Meltwater: Use it to instantly search blogs, Twitter, and Facebook for specific terms
  • Social Mention: Use it to monitor multiple websites in one place and find out which keywords people use when talking about your company
  • IFTTT (If This Then That): Use it to automate simple online tasks using “recipes”. For example, “If [website] mentions [company], then send me an email alert”

There are many other tools out there. When selecting those you prefer, choose tools that show you not only where you were mentioned, but exactly what was said. That will give you the voice-of-customer data you need.

Conversion Research - A lot of presents can come from the social media box… if you just open it
Conversion Research – A lot of presents can come from the social media box… if you just open it

On your website, you can also provide social media links for visitors to use to share their opinions directly on Facebook or Twitter.

Quantitative analysis can then help you identify which social media platforms are most useful to your website, since your analytics tool will track the number of visitors coming from each individual channel.

Conversion research #5: Got a niche product? Learn to love blogs and forum posts

Even more than social media, blog posts and forum posts can offer deep, comprehensive opinions and critiques of products and services. Tracking these mentions should be a matter of course — you need to know what people are saying about you, because these reviews directly affect conversions.

And, for the purpose of conversion optimization, these reviews are very useful as additional sources of insight of what people expect, what they find, and where you might be losing them.

For niche products especially, blog posts and forum posts are the best and most readily available sources of voice-of-customer data. Influencer opinions and reviews reach your target audience, giving the most valuable form of social proof you can have. They can also do serious damage to your conversions if their impressions are less than favorable.

Since conversion optimization is primarily interested in the performance of the website as the main source of conversions, issues with products themselves may be outside of your purview. But blogs and forums still contain useful feedback that you can use to improve the copy on your website and address usability issues.

Conversion research #6: How to stay focused & avoid data overwhelm

Too often, the most readily available sources of user feedback are ignored in favor of more complicated and expensive methods of acquiring qualitative feedback. Don’t let that happen to you.

We recommend beginning your qualitative data gathering process by making a comprehensive review of existing available sources. Just be careful not to get distracted by issues that are not directly related to conversion optimization (which is all too easy to do!).

Conversion Research - This maxim is just as applicable for conducting customer research as it is for leading Apple
Conversion Research – This maxim is just as applicable for conducting customer research as it is for leading Apple

Your task as an optimizer is to improve the website, not the products or services themselves. To reduce the risk of distraction, approach your data gathering with a plan. Use your survey questions as a guideline for what types of answers to look at more closely.

And if product issues continue to arise, pass on that information to the appropriate people — so you can stay focused on your job.

Series Navigation<< User Survey Guide: If You’re Not Conducting Surveys, You’re Losing ConversionsHow to Find All the Qualitative Data You Need for CRO Testing (In Unexpected Places) >>

Published by

Edin is a Senior CRO Consultant. Edin is into Google Analytics and testing (any A/B testing tool really) and likes to write about it. You can follow Edin on Twitter.