Stop using split testing as a crutch for cowardice! Accelerate Business with Tim Ash

My interview partner today is Tim Ash, the founder and chair of Conversion Conference (with over two dozen shows in the US and Europe since 2010) and author of Landing Page Optimization, a marketing classic now in its second edition which has been translated into six languages. Tim has been a leader in the conversion rate optimization (CRO) industry since founding SiteTuners in 2002, and has worked with clients like Google, Facebook, Nestle, Sears, Intuit, Coach, Nespresso, Canon, Symantec, and Verizon Wireless. 

Tim, how far has CRO come in the past 15 years?

Awareness has gone up but the level of expertise is still very low.

CRO is often put it into the digital marketing “ghetto” as just one of the online marketing channels. It’s used to make small tweaks instead of making it a strategic function for competitive advantage which is how the best companies use it.

At SiteTuners we’ve created a detailed Conversion Maturity Model for our clients, and we rate each organization based on several dimensions:

  • Customer Experience (online and offline – across all touchpoints)
  • Organization & Skills (what in-house capabilities the company has in terms of staff)
  • Marketing Technology Stack (which important layers of technology assist in optimization)
  • Measurement & Accountability (what metrics are being measured and rewarded)
  • Process & Culture (formal processes around optimization and deep-seated attitudes at all levels)

In each dimension we have objective scales for placing people at either Unoptimized, basic, intermediate, and advanced. Based on that, we’re still seeing very low levels of sophistication at most companies.

That’s frustrating after 15 years in the industry. Why is that?

Proper CRO often lacks the support on the executive team that it requires. Many companies just put a box onto the org chart, whereas at the most effective organizations we have seen the CRO team reporting up to the CMO directly. That provides the executive political cover for bringing in the big sledgehammer, and fixing the parts of the business that need it.

In the widest sense you need to look at CRO in terms of business acceleration, and most don’t see it that way - so they don’t get the full benefit of it

Additionally, the discipline is very multi-faceted and experienced CRO people are hard to find and hard to keep. They are in demand everywhere which is why a lot of companies partner with SiteTuners for our skilled resources. We can be the expert surge capacity that our client need to get key projects done.

Where should CRO sit within the organization?

Certainly not under analytics, which is often a passive research activity, like driving down the highway and looking at your rear view mirror. Analytics can only tell you what happened in your current context (web experience, and internal workflow).

What often needs to happen is that you need to restructure your whole prospect and client facing approach, so the old metrics may become irrelevant or even cease to exist. The customer experience (CX) side is where the real action is, but to change that in any fundamental way requires political power within the company, that is where the hard work is.

For example, what if you have to change your business model to meet CX expectations instead of just optimizing “buy now” or “sign up for our demo” button text? That requires executive higher-level support. That is why we find ourselves in the roles of change-agents within the company.

CRO should permeate your entire online marketing organization, and it is not ultimately something that you should outsource to a tactical split testing consultancy. Our goal at SiteTuners is to make our clients self-sufficient within two years, so that they don’t need us anymore. Therefore, one of our biggest practice areas is knowledge-transfer around our Conversion Maturity Model. We can objectively tell you where you are, and what specific steps would get you to the next level.

Do you even call it CRO when it’s that strategic?

Yes, CRO is the current term-of-art. My book was entitled Landing Page Optimization because that was the original term at the time. Sean Ellis and others have called it growth hacking. At SiteTuners, business acceleration is the term we usually use.

Anyone who reduces CRO to “split testing” is really missing the boat. But that is currently often the state of the discussion at companies. They are talking about “velocity” and how many tests can you run. But that is often missing the point. There are much bigger opportunities if you are able to tackle more fundamental changes to the business.

Maybe you need to have different conversion actions altogether, or maybe you need to create a content marketing strategy for various customer journeys, or maybe you need a radically different marketing technology stack, or maybe you need to wrestle back some control from the brand-side people and give more power to the direct-response champions at the company.

If we focus on the marketing technology stack for the moment, which categories are key?

There are several key components that our experienced clients share:

  • You need to have solid analytics across all touchpoints, online and offline – so your numbers are solid and enable you to more confidently make decisions
  • Qualitative behavior tools like Clicktale are also critical for gaining insight into what’s broken or sub optimal
  • You need a flexible content management system where you can get things done without need for IT involvement, and without breaking things
  • You need to be able to segment the site experience based on what you know about the person
  • This, in turn, usually relies on a marketing automation system of some sort
  • After that there is 3rd-party data append, and integration with your CRM or data warehouse to personalize experiences in real time

How do you view using Clicktale vs. lab or user testing?

There are limitations to user testing or interviews that use the talk-out-loud protocol: “here, buy a pair of red shoes and describe what you are thinking while doing so…”, because the talking is going to mess up the actual experience (which is not very deliberate and motivated by unconscious parts of the brain that are sub-verbal). It’s much better to watch people as they do things naturally and see their real actions. That’s why in-page analytics (or customer experience analytics as I refer to it) is critical.

How do you view Clicktale and split testing?

You can do lots of testing, but where are you getting your ideas for what’s broken? The focus should be on actual CX by watching how customers behave in real time.

Clicktale gets you insights and ideas for what’s broken or sub-optimal. Split testing is a crutch, it’s for cowards. If something is broken, just fix it. You don’t need to test everything. You don’t need anyone’s permission.

Ton Wesseling from in the Netherlands recently reminded me on a podcast that we did that people often forget that there is a significant cost to testing. If something comes up that needs split testing validation you have got to remember that someone needs to design, implement, and QA the tests. Testing also means that sub optimal test variations will be running for the duration of the test: which has an opportunity cost. On top of that, many tests will fail to find a better version or may be inconclusive. You should subject every test to a stringent ROI analysis. There should be a pretty high financial bar before investing in a test.

But there are plenty of other things that you should probably be doing: i.e. fix the real issues, or redesign your whole site so that it really meets the expectations of your customers. Don’t just do the tactical testing stuff because it is easier and has a repeatable process. That is like the drunk looking for his lost car keys at night under the lamp-post because the light is better there…
The Clicktale team would like to thank Tim Ash for the thought provoking guidance and telling it like it is!

Talk to us to explore how customer experience analytics can improve your business