Tips and Trends in AB Testing

In my work as marketing acquisition manager at Clicktale, I frequently use a wide variety of A/B tests. A/B testing is a method of comparing two versions of a webpage or app against each other to determine which one performs better. The main goal for using A/B testing is to increase conversion rates and learn about website visitors’ preferences and behavior.

Today, most A/B test tools enable you to split your traffic and segment your audience, allowing for greater specificity in your tests. In other words, you can set up stricter goals, test multiple aspects of your web pages – buttons, colors, designs and imagery, not to mention text – and then receive detailed diagnostic reports.

When setting up a new A/B test, I recommend taking these things into consideration:

  1. Be specific when deciding what to test. What do you expect to achieve if changes are adopted?
  2. Choose the page on which you want to conduct the test, and then evaluate which A/B testing solution makes the most sense.
  3. Set up two designs to test. Remember: one is the original and the other is the new design you think will improve your KPIs for the page, be it conversions, engagement or another metric. Don’t be afraid to test big changes, as they might just change the value of your webpage, leading to a significant impact on your business.
  4. Track your test results. Most tools will give you a confidence score, and results with 95% confidence or more are considered accurate.
  5. Don’t stop! Keep testing again and again, changing one element at a time.

I’ve come across several innovations in A/B testing that allow for a much greater degree of specification and can be tailored to fit the specific KPI's of each company.. Here are four interesting trends:

  • Mobile everywhere. Every company today understands that having a mobile-friendly site is imperative. And obviously the way visitors interact with your site on a mobile device is quite different than how they interact with a site they’re viewing from their desktop. Use A/B testing to check the differences between mobile and desktop. Just as important, use A/B testing to check the habits of mobile users so that you can optimize your mobile site to promote the actions you’d like your users to take. Given the small screen size, which elements capture their attention? Where should you place your most important assets?
  • Website personalization: Today’s A/B testing tools enable you to run tests based on personalized information or offers you are showing your visitors. Some enterprises today are using Big Data to offer tailored content (based on assumptions about the way a visitor will behave) to engender the feeling of a personalize shopping or browsing experience. You can run A/B tests to examine if the tailored content leads to more conversions or engagement.
  • Dynamic testing: Today, we have the option of creating numerous variants at the outset of a test, and then narrowing down the best results as the test goes along. In other words, you can create five versions of a test and then as you see the winners and losers emerge, the losers get dropped until you have an ultimate winner. This automation in A/B testing tools is a recent addition to their feature set, and is a great way to speed testing cycles.
  • Testing visitor emotions: At Clicktale we often see session replays that make it very easy to discern the emotional mindset of the visitor. For example, if the visitor is moving the mouse in a restless manner, or scrolling up and down the webpage frantically, you can assume they haven’t found what they are looking for and are not satisfied the page. With A/B testing, you can change the location of items on your page and then measure whether the test version led to a more successful session.

A/B testing is all about trying to improve your business results. In order to do this, you need to be focused on creating visitor journeys that are satisfying to the visitor, and will keep him or her coming back to your site. The combination of A/B testing together with other visual insights like session replays and heatmaps is the most effective way to measure how visitors are using your website and how successful they are in achieving their goals.

Talk to us to explore how customer experience analytics can improve your business