By Tara Lasker

Like a lot of people, I rely on user reviews for virtually all of my purchase decisions. For example, in the last week I’ve read reviews on:

• Yelp for restaurants (and even which dishes to order from said restaurants)

• Overstock.com to give me a better idea on the quality/color of a mirror I was about to purchase

• Airbnb to decide whether the location and appearance of a vacation rental was all it was cracked up to be

While I’ve come to depend on these reviews—I’d be hesitant to buy something that didn’t have some kind of rating— this mountain of data can be paralyzing. My husband and I are notoriously slow decision makers, and the cartoon below (from the always spot on xkcd.com) pretty much sums up how relying on user reviews has lengthened our purchase process. At one point we found ourselves wondering: who are these people anyway? 

What kind of person has the time to deconstruct and rate every detail of a lamp? I mean, you can find user reviews on anything—it’s remarkable.  Can these people even be trusted? And whose businesses are they hurting, or helping in the process?

As a market researcher, I think a lot about these people and the information they’re providing.  Sampling is such a critical part of research design but it’s often overlooked by data users. Here are some questions we should be asking about the people we entrust our hard-earned money to:

• Representativeness: This is a pretty simple concept, we need to ask: does this data represent the population it’s intended to?  Are Yelpers different than the average person? Do they care about the same things as me?

• Authenticity: Are the responses real or are people gaming the system? If authenticity weren’t a real concern before, the recent government crackdown on consumer review fraud should make us wonder who is actually writing some of these reviews. Even if nothing illegal is going on, it makes sense to ask whether there are incentives or disincentives for a sincere evaluation.

• Disposition: Are we only hearing from those who need a platform to vent or conversely those who are thrilled? Will reviews skew negative because consumers are much more likely to share a negative experience than a positive one? It's an important question and for Yelp's part, they share the breakdown of reviews by number of stars. In the chart below we find more positive reviews on Yelp than negative. 

User reviews have changed the path to purchase for many industries, some are slower to adopt (e.g., health care) but even the stragglers will have no choice but to accept that these strangers are influencing their brand perceptions and purchase likelihood. It's worth our time to ask just who these influencers are.

Tara is Research Director at CMB, she's also an avid user review reader who doesn’t have the time to write her own reviews.