The Double-edged Sword of Rating Systems

When I travel to the US on business, I always turn to Yelp so I can find the best Dojo near me for a good workout. It never fails and I’m always satisfied. Crowd-sourced rating systems like Yelp have become part of our lives – we’ve become a society that rarely ventures out without checking a rating or a crowd-sourcing review website to see what others think about it. But the evolution of ratings has started to reveal a dark side. Let’s have a look at where we started and where we’re going.

THE ROOTS OF RATINGS

Sites like Yelp and TripAdvisor grew out of the rating systems of the last century. These include Michelin guides that first started rating restaurants in the 1920s; film grading that also began in the 1920s; and hotel star rating systems that got going in the 1950s. Historically, these ratings and others like them were based on an objective list of criteria and carried out by experts in their fields. Like their counterparts today, the focus of these ratings was to give customers information to help them choose the product or experience that was right for them.

Businesses saw in ratings the potential to learn more about their products and services from their customers, and leverage that information to improve. The Customer Satisfaction Score (CSAT) model entered the business world in the 1980s, offering a scientific and psychological model for measuring the gap between customer expectations before and after using a product or service. The idea was to retain customers, reduce churn and increase value. The Net Promoter Score (NPS) index, introduced in 2003, gauges customer loyalty. Major companies use NPS along with CSAT and other indexes to try to measure customer perceptions of product health.

But these indexes are problematic – customers need to opt in to be surveyed. They tend to primarily opt in when they’re experiencing extremes – love it or hate it – while those in the middle remain unsurveyed. What’s worse is that companies can manipulate results in a variety of ways, for instance, by knowing exactly when to survey people to achieve the best results. Overall, it’s difficult for businesses to act on results because no actual causes are associated with customers’ attitudes.

THE POWER OF THE CROWD

The evolution of rating systems was bumped up with the advent of the internet and social media. Almost overnight, any human with a data package could be a critic of, well, anything – businesses, books, hotels, restaurants, taxi drivers, and the list goes on.

Though rooted in new technology, these rating systems serve two purposes simultaneously – guidance for customers and information for businesses. They also give “the crowd” tremendous influence. The mere fact that any anonymous user can have power over a business is where it starts getting scary.

One low rating too many for an Uber driver and he can be deactivated. For every star a restaurant loses on Yelp, revenue falls by between 5-10%; reach one star, and you can pack your tables and pots because your restaurant will likely go out of business.

When it came out that the parents of Ahmad Khan Rahami, charged with setting bombs in New York and New Jersey, were the owners of a small New Jersey restaurant, the crowd took revenge…via Yelp. Reviewers rated the restaurant, not for the food but for the owner’s’ connection to their son, despite the fact that they had earlier alerted police. The result was 13 pages of one-star reviews that forced the family to close their restaurant. This is just one of many politically motivated acts of revenge on businesses via Yelp. (See article)

BIG BROTHER GETS IN ON THE ACT

Looking across to China, we see another example of how ratings can turn nefarious when the government gets involved. Alibaba and the Chinese government are working together to create a social credit system that will eventually give every Chinese citizen a trustworthiness rating. By mining Big Data, the government will create a score for each citizen based on online activities (purchases, payments), medical histories, education, social behavior and more. Good scores can increase people’s chances of finding a good job, meeting a partner on a dating site, getting VIP hotel reservations without a deposit, and so on. Bad scores could prevent them from receiving a bank loan, signing a lease or finding a life partner. Here, quite easily, a person’s entire life can be ruined by an unfavorable rating.

TRANSFORMING RATINGS INTO A FORCE FOR IMPROVEMENT

With the quantity of data and our ability to store and process it growing every minute at a phenomenal rate, and open source algorithms democratizing data processing, the possibilities are endless. The data exists. What we choose do with can ultimately be either Orwellian or Utopian.

When all you have is rating numbers, your take-away is by definition, reductive. While these rating may be useful to users, they are of limited value to the company. For complex human behavior, such as interactions on a web or mobile site, you need more complex summary data, on which you can base corrective action.

As head of Data Science at Contentsquare, I think long and hard about how customer data can be used as a force for good, while protecting individuals and helping the businesses they frequent improve. It’s challenging, but we’re working hard to get it right. In my next post, I’ll talk about just how we tread this fine line and come out at the other end with systems that are actionable, valuable and free from distortion.

Clicktale was acquired by Contentsquare in 2019. Since then, tools and features mentioned in this blog may have evolved. Learn more about our Digital Experience Analytics Platform.