The posts here represent the opinions of CMB employees and guests—not necessarily the company as a whole. 

Subscribe to Email Updates

What We’ve Got Here Is a Respondent Experience Problem

Posted by Jared Huizenga on Thu, Apr 14, 2016

respondent experience problemA couple weeks ago, I was traveling to Austin for CASRO’s Digital Research Conference, and I had an interesting conversation while boarding the plane. [Insert Road Trip joke here.]

Stranger: First time traveling to Austin?

Me: Yeah, I’m going to a market research conference.

Stranger: [blank stare]

Me: It’s a really good conference. I go every year.

Stranger: So, what does your company do?

Me: We gather information from people—usually by having them take an online survey, and—

Stranger: I took one of those. Never again.

Me: Yeah? It was that bad?

Stranger: It was [expletive] horrible. They said it would take ten minutes, and I quit after spending twice that long on it. I got nothing for my time. They basically lied to me.

Me: I’m sorry you had that experience. Not all surveys are like that, but I totally understand why you wouldn’t want to take another one.

Thank goodness the plane started boarding before he could say anything else. Double thank goodness that I wasn’t sitting next to him during the flight.

I’ve been a proud member of the market research industry since 1998. I feel like it’s often the Rodney Dangerfield of professional services, but I’ve always preached about how important the industry is. Unfortunately, I’m finding it harder and harder to convince the general population. The experience my fellow traveler had with his survey points to a major theme of this year’s CASRO Digital Research Conference. Either directly or indirectly, many of the presentations this year were about the respondent experience. It’s become increasingly clear to me that the market research industry has no choice other than to address the respondent experience “problem.”

There were also two related sub-themes—generational differences and living in a digital world—that go hand-in-hand with the respondent experience theme. Fewer people are taking questionnaires on their desktop computers. Recent data suggests that, depending on the specific study, 20-30% of respondents are taking questionnaires on their smartphones. Not surprisingly, this skews towards younger respondents. Also not surprisingly, the percentage of smartphone survey takers is increasing at a rapid pace. Within the next two years, I predict the percent of smartphone respondents will be 35-40%. As researchers, we have to consider the mobile respondent when designing questionnaires.

From a practical standpoint, what does all this mean for researchers like me who are focused on data collection?

  1. I made a bold—and somewhat unpopular—prediction a few years ago that the method of using a single “panel” for market research sample is dying a slow death and that these panels would eventually become obsolete. We may not be quite at that point yet, but we’re getting closer. In my experience, being able to use a single sample source today is very rare except for the simplest of populations.

Action: Understand your sample source options. Have candid conversations with your data collection partners and only work with ones that are 100% transparent. Learn how to smell BS from a mile away, and stay away from those people.

  1. As researchers, part of our job should be to understand how the world around us is changing. So, why do we turn a blind eye to the poor experiences our respondents are having? According to CASRO’s Code of Standards and Ethics, “research participants are the lifeblood of the research industry.” The people taking our questionnaires aren’t just “completes.” They’re people. They have jobs, spouses, children, and a million other things going on in their lives at any given time, so they often don’t have time for your 30-minute questionnaire with ten scrolling grid questions.

Action: Take the questionnaires yourself so you can fully understand what you’re asking your respondents to do. Then take that same questionnaire on a smartphone. It might be an eye opener.

  1. It’s important to educate colleagues, peers, and clients regarding the pitfalls of poor data collection methods. Not only does a poorly designed 30-minute survey frustrate respondents, it also leads to speeding, straight lining, and just not caring. Most importantly, it leads to bad data. It’s not the respondent’s fault—it’s ours. One company stood up at the conference and stated that it won’t take a client project if the survey is too long. But for every company that does this, there are many others that will take that project.

Action: Educate your clients about the potential consequences of poorly designed, lengthy questionnaires. Market research industry leaders as a whole need to do this for it have a large impact.

Change is a good thing, and there’s no need to panic. Most of you are probably aware of the issues I’ve outlined above. There are no big shocks here. But, being cognizant of a problem and acting to fix the problem are two entirely different things. I challenge everyone in the market research industry to take some action. In fact, you don’t have much of a choice.

Jared is CMB’s Field Services Director, and has been in market research industry for eighteen years. When he isn’t enjoying the exciting world of data collection, he can be found competing at barbecue contests as the pitmaster of the team Insane Swine BBQ.

Topics: data collection, mobile, research design, conference recap