Voices: the CMB Blog

Back in 2005, Malcolm Gladwell told us that focus groups are dead. Just last November, Jim Bryson, CEO of 20/20 Research, questioned whether qualitative research was thriving or dying: “If we take a narrow, more traditional view that qualitative is defined by the methods of face-to-face focus groups or interviews, particularly those held in a qualitative facility, then the case can be easily made that qualitative is dying.”

To all of this, I say: wait, what?! Qualitative is dying? I refused to believe it, so I embarked on a journey to explore where qualitative has been, and more importantly, where it’s going. During my research, I found plenty of evidence to support the fact that qualitative is not, in fact, dying. Great news, right? (Especially for me, because if it were true, I just might be out of a job I love.)I took a look at the fall 2014 Greenbook Research Industry Trends (GRIT) Report and focused on the data from Q1-Q2 of 2013 and Q1-Q2 2014. In this data, I learned:

• The use of traditional in-person focus groups increased from 60% (Q1-Q2 2013) to 70% (Q1-Q2 2014).
• Within the same time period, the use of in-person, in-depth interviews increased from 45% to 53%.
• Interviews and groups using online communities increased from 21% to 24%.
• The use of mobile qual (e.g., diaries, image uploads) increased from 18% to 24%.

Yes, it’s important to note that not all qualitative methodologies saw an increase in usage within this timeframe. In fact, there was a decrease in the usage of telephone IDIs, in-store shopping/observations, bulletin board studies, both chat-based and webcam-based online focus groups, and telephone focus groups.  All this notwithstanding, I think it’s fair to say that qualitative is still very much alive and well.

So why do people keep talking about qualitative dying? We can’t deny that there are a number of factors that affect how and when we use qualitative methodologies today (technology, access to big data, and text analytics are a few). But, this doesn’t mean qualitative is disappearing as a discipline. Qualitative is evolving at a rapid pace and feels more relevant than ever. Sure, we need to keep up with client demands for faster and cheaper research, but there will always be a need for the human mind (i.e., a qualitative expert) to analyze and synthesize the data to provide meaning and context behind the way people think and behave—and that is where actionable insights are born.   

Now that we know qualitative really isn’t dying, what does 2015 (and beyond) hold for us? The future is about truly integrated research—in which qualitative and quantitative are consistently, thoughtfully, and purposefully used together to provide well-rounded, actionable insights. We’re poised to do exactly that with our dedicated analytics team and network of expert industry qualitative partners. By using two equally important disciplines that are both alive and well, we can provide our clients critical insights they can really use. Far from killing off qualitative insights, technology and an evolving marketplace are helping make qualitative insights even stronger.

Anne Hooper is the Qualitative Research Director at CMB. After recently finding out that her 13 year old daughter did a quantitative assessment of her Jazz Band’s upcoming Disney trip itinerary, she’s determined that an intervention may be in order.

I know what you are thinking...“What the heck is she TALKING about? How can Tinder possibly relate to mobile ethnography?”  You can call me crazy, but hear me out first.For those of you who may be unfamiliar, Tinder is a well-known “hook up” app that’s taken the smartphone wielding, hyper-social Millennial world by storm. With a simple swipe of the index finger, one can either approve or reject someone from a massive list of prospects. At the end of the day, it comes down to anonymously passing judgment on looks alone—yet if both users “like” each other, they are connected. Shallow? You bet. Effective? Clearly it must be because thousands of people are downloading the app daily.

So what’s the connection with mobile ethnography? While Tinder appears to be an effective tool for anonymously communicating attraction (anonymous in that the only thing you really know about the other person is what they look like), mobile ethnography is an effective tool for anonymously communicating daily experiences that we generally aren’t as privy to as researchers. Mobile ethnography gives us better insight into consumer behavior by bringing us places we’ve never gone before but are worthy of knowing nonetheless (Cialis, anyone?). Tapping into these experiences—from the benign to the very private—are the nuts and bolts behind any good product or brand.

So how might one tap into these experiences using mobile ethnography? It’s actually quite easy—we create and assign “activities” that are not only engaging for participants, but are also designed to dig deep and (hopefully) capture the "Aha!" moments we aim for as researchers. Imagine being able to see how consumers interact with your brand on a day-to-day basis—how they use your product, where their needs are being fulfilled, and where they experience frustrations. Imagine “being there” when your customer experiences your brand—offering insight into what delights and disappoints them right then and there (i.e., not several weeks later in a focus group facility). The possibilities for mobile ethnography are endless...let’s just hope the possibilities for Tinder come to a screeching halt sooner rather than later.

Anne Hooper is the Director of Qualitative Services at CMB. She has a 12 year old daughter who has no idea what Tinder is, and she hopes it stays that way for a very long time.

Those who know me understand that I am not afraid to admit I love reality TV.  Combine that love with an interest in pop culture (generally), and a passion for understanding what people do and WHY they do it, and you have a match made in heaven. So obviously Catfish—the MTV series —is right up my alley.

Talk of "Catfishing" seems to be everywhere these days, but for the uninitiated, I’ll give you the quick (Wikipedia) definition: “A Catfish is a person who creates fake profiles online and pretends to be someone they are not by using someone else’s pictures and information.”  Put simply:  Catfishing is a relationship built on deception.

So what does Catfishing have to do with online qual?

As a qualitative researcher, I have to build “relationships” with strangers all the time, both online and in-person.  I can guarantee you that these relationships are genuine, authentic and honest—at least from my end.  My ultimate goal is to better understand research participants as human beings—how they live, what they value, what makes them ‘tick’, etc.  Most of the time, I truly feel that those I’m spending time with (both online and offline) are also being authentic and honest with me. Notice I said most of the time. 

Though it doesn’t happen often, it IS possible to come across a phony (AKA “Catfish”) in an in-person setting.  There are some pretty savvy people out there who seem to know how to make their way into a focus group for some extra cash.  Thankfully it’s rare—and most of the time these folks get weeded out before they even enter the room.  Online qualitative research, on the other hand, is ripe for Catfish.  Unless we are conducting video web-based research, there aren’t any visual clues to help us validate identities.  Therefore, we can’t be 100% sure that the person we THINK we are talking to is really that person.

The good news is that as researchers, we can take measures to protect ourselves from these Catfish participants online—it just takes a little effort and creativity.  Here are a few methods I’ve used successfully in the past:  

• Demographics:  If you have a participant that has an annual income of $50K and claims to spend an average of $10K a year on vacation, you’ve got yourself a red flag.  Taking the time to cross reference demographics with online responses can be extremely helpful in getting to the truth.

• Common sense:  Individual responses don’t stand alone, but pulled together they create a story.  At the end of the day you either have a story that makes sense or you don’t, and a story that doesn’t make sense is another red flag.  Just as one would do when moderating an in-person group, there are times when you must revisit what someone said earlier, and if necessary, request clarification.  (In the immortal words of Judge Judy: “If it doesn’t make sense, it’s not true.”) 

• Consistency:  A lack of consistency can be another red flag.  If a participant says one thing, but contradicts themselves sometime later, there might be a problem.  Here’s an example:  in a recent “vacation” study we had a participant who changed her dates of a travel a few times (not unusual).  She later confirmed purchasing a package (air, hotel, car) for a family of 5 one week prior to departure (somewhat fishy … especially for someone who was very price sensitive).  Her “confirmed” travel dates were from the 25th-30th of the month—and when she hadn’t checked in, as requested during that time, we reached out to her to find out that she was “already home” on the 29th.  Suspicious?  Very.  This lack of consistency—along with several other red flags—confirmed our suspicions that she was not being truthful and she was pulled from the study.  Again, to quote Judge Judy, “If you tell the truth, you don’t have to have a good memory.”

• Engagement:  There are always going to be participants who choose to do the bare minimum in order to get their incentive.  However, a lack of engagement and openness—coupled with any additional red flags—requires some investigation.  Is the participant just taking the easy way out by answering questions in as few words as possible, or are they skipping key questions altogether?  Skipping key questions (e.g., “Tell us what you like best about product X”) could be a sign that they really don’t use product X after all.  Again, it’s important for the moderator to probe accordingly and if the probes go ignored … you guessed it … another red flag.

With online research (and plenty of Catfish) here to stay, we need to continue to be vigilant in crossing our T’s and dotting our i’s.  I, for one, am ready to catch them … hook, line and sinker.

Anne is CMB’s Qualitative Research Director.  She enjoys travel and thanks to DVR, never misses an episode of Judge Judy. Anne especially loves being able to truly “connect” with her research participants—it’s in her Midwestern blood.   

Learn more about Anne and her Qualitative Research team here.