How to get the most from your website A/B testing

Guest Post: Ramel Levin

In today's conversion-driven digital markets, A/B testing is a fundamental tool that takes the guesswork out of website optimization. It is a great way to quickly try to improve the design and performance of your web pages.

This short article discusses three specific issues that can improve your A/B testing and lead to better website optimization:

  • Data-driven versus user experience-driven testing
  • Audience segmentation
  • Testing procedure

Before You Start

A/B testing is a valuable tool that enables the comparison of two versions of a web page to identify which of them elicits more responses, causes more visitors to subscribe to a service, or leads to more conversions. But how do you choose which pages to test? And on each page, what elements should you test?

You can start with best practices. Most web analysts will recommend testing calls to action, the content placed above or below the fold, and the structure of a registration form because all of these can lead to quick wins and have a strong impact on conversion.

You can also make a hypothesis about which areas of a page may be causing frustration or abandonment. This process can be aided by in-page web analytics, which monitor your visitors' behavior on the page (including where they are clicking, hovering or abandoning a funnel) and provide insights into which page elements to A/B test.

1.     Data-Driven Versus User Experience-Driven Tests

Data-driven A/B testing relies on hard numbers, including on the mathematical analysis of how people use your webpages, click different elements and interact with every section. It is based on statistics, calculations, responses to questionnaires and/or feedback forms that you ask your visitors to complete. Data-driven A/B testing is structured, stable and easy to measure.

User experience (UX)-driven tests, on the other hand, are based on hundreds or thousands of theories involving issues like font type, font size, color, position of the call to action, page design and more. In many ways, user experience is related to creativity and personal opinion, which are very hard to measure.

Data-driven A/B testing should be your goal. Gather as much significant data as possible, structure it to ensure high-quality results, test and analyze it, and draw conclusions. The result of the A/B test of your home page may lead to the conclusion that no changes are required. However, if the test was conducted scientifically using large amounts of data and relying on tools that provide you with insights into your visitors' behavior, then you will know the results of the test are valid.

2.     Segment Your A/B Test Audience

Today, more and more companies personalize their web pages to more accurately deliver their messages and campaigns to different target audiences. To accomplish this, it is important to be able to segment visitors. For example, you can segment the target population of an A/B test by new versus returning visitors. If you sell a service to which your customers are already subscribed, you do not want to offer them something they already have, but rather show them a page with other supplementary products, a training video, the page of your support organization or information about how they can maximize the benefits of your service.

This methodology is also known as targeted A/B testing; you do not test with everybody…only with specific segments of the target population. As author Avinash Kaushik states in Web Analytics 2.0, "Absolutely nothing is more important in analytics than segmentation. Why? Because monoliths don’t come to our websites. I know that seems like such a shock. Yet most of our reporting and analysis happens at an aggregate level…Different types of visitors come to your website. And they all come with different intentions, with different problems or solutions, and with different personas. This unique phenomenon mandates that you have a very effective and persistent segmentation strategy as part of your web-analytics process."

Other popular segmentation options for A/B testing include:

  • Geographical location—This is useful if you're considering localizing your website or if you wish to further outline the services you provide to specific locations (for example, shipping).
  • Source—This method enables you to segment your visitors according to what brought them to your website (e.g. PPC campaigns, social channels, email campaigns or organic web search). It is very useful when you're trying to measure the success of a campaign or when deciding on your company's marketing budget.
  • Previous behavior—You can target your message or campaign based on whether a visitor has previously shopped on your site, visited but not shopped, or has items in the shopping cart from a previous visit. It also allows you to assess the user experience and make improvements where necessary.

If you want to test other behavioral aspects like how visitors respond to a color or message, you can run a multivariate test or factorial experiment to test different combinations of different sections or elements of a page.

In summary, segmentation provides an infinite number of possibilities for tests that you can run, further perfecting your optimization cycle.

3.     Test, Test and Retest

Many businesses make the mistake of A/B testing a web page, analyzing the results, making the changes (if the results of the A/B test show that changes are indeed needed), and then moving on to another page.

This should not be the case, however. After running the A/B test, you may learn that your original hypothesis was incorrect, that the results are inconclusive, or that both versions of a tested page had the same conversion rate. This does not mean you should stop testing; it only means that you should go back to the drawing board, study your visitors' behavior, make another hypothesis and test again. This is the only way to ensure a continued process of optimization.

Testing is a cyclic and iterative process. You can run many tests on different sections of the same webpage or test a single element (for example, a call to action) across several pages. Moreover, user preferences change over time, the devices they use to interact with your website are constantly advancing, and you want to make sure that they always have an optimum experience and that they keep converting. This can only be achieved through continuous testing.

What's Next?

After collecting the data, segmenting it, conducting the test, analyzing the results and making the changes, you can move on to understanding why one version performed or behaved better than another. This can be achieved with in-page analytics tools like heat maps, visitors' recordings and form analytics. These provide actual insights into your visitors' behaviors when they interact with your pages, explaining why they reacted differently to each version.

The visualization of people's actual behavior online will help you draw conclusions that you can implement across all website pages, so that you can enhance your visitors' experience, increase conversion and improve your bottom line.

As VP of Marketing, Ramel is responsible for the Clicktale brand, marketing communication, marketing acquisition and retention.

Ramel has over 20 years in the Hi-Tech Development & Marketing industries. Prior to joining Clicktale, he was the Marketing Director at Panaya, where he led the company's entrance into new markets as well as grow existing markets. Earlier in his career, Ramel was Marketing Director at ARX for over 6 years and was also the founder of WorldMate, where he led a mobile bootstrap from inception to a multi-million dollar revenue company.
Ramel holds an MBA from Hebrew University and a B.A. in Psychology & Computer Science from Bar-Ilan University, both in Israel.




Talk to us to explore how customer experience analytics can improve your business