Welcome!

Welcome to the CMB Blog!  The posts here represent the opinions of CMB employees and guests, not necessarily the company as a whole.

Follow us on Facebook, LinkedIn, and Twitter!

Subscribe by Email

Your email:

CMB Consumer Pulse



ConsumerPulseLogo new resized 244


Download the latest market research reports from our Consumer Pulse series.

Named a Top NGMR Blog

An NGMR Top Blog

All posts by category

Voices of CMB: The Chadwick Martin Bailey Research Blog

Current Articles | RSS Feed RSS Feed

How to Catch a Catfish: Secrets of a Qualitative Researcher

 

catch a catfishBy Anne Hooper

Those who know me understand that I am not afraid to admit I love reality TV.  Combine that love with an interest in pop culture (generally), and a passion for understanding what people do and WHY they do it, and you have a match made in heaven. So obviously Catfish—the MTV series —is right up my alley.

Talk of "Catfishing" seems to be everywhere these days, but for the uninitiated, I’ll give you the quick (Wikipedia) definition: “A Catfish is a person who creates fake profiles online and pretends to be someone they are not by using someone else’s pictures and information.”  Put simply:  Catfishing is a relationship built on deception.

So what does Catfishing have to do with online qual?

As a qualitative researcher, I have to build “relationships” with strangers all the time, both online and in-person.  I can guarantee you that these relationships are genuine, authentic and honest—at least from my end.  My ultimate goal is to better understand research participants as human beings—how they live, what they value, what makes them ‘tick’, etc.  Most of the time, I truly feel that those I’m spending time with (both online and offline) are also being authentic and honest with me. Notice I said most of the time

Though it doesn’t happen often, it IS possible to come across a phony (AKA “Catfish”) in an in-person setting.  There are some pretty savvy people out there who seem to know how to make their way into a focus group for some extra cash.  Thankfully it’s rare—and most of the time these folks get weeded out before they even enter the room.  Online qualitative research, on the other hand, is ripe for Catfish.  Unless we are conducting video web-based research, there aren’t any visual clues to help us validate identities.  Therefore, we can’t be 100% sure that the person we THINK we are talking to is really that person.

The good news is that as researchers, we can take measures to protect ourselves from these Catfish participants online—it just takes a little effort and creativity.  Here are a few methods I’ve used successfully in the past:  

  • Demographics:  If you have a participant that has an annual income of $50K and claims to spend an average of $10K a year on vacation, you’ve got yourself a red flag.  Taking the time to cross reference demographics with online responses can be extremely helpful in getting to the truth.

  • Common sense:  Individual responses don’t stand alone, but pulled together they create a story.  At the end of the day you either have a story that makes sense or you don’t, and a story that doesn’t make sense is another red flag.  Just as one would do when moderating an in-person group, there are times when you must revisit what someone said earlier, and if necessary, request clarification.  (In the immortal words of Judge Judy: “If it doesn’t make sense, it’s not true.”) 

  • Consistency:  A lack of consistency can be another red flag.  If a participant says one thing, but contradicts themselves sometime later, there might be a problem.  Here’s an example:  in a recent “vacation” detective magnifying glassstudy we had a participant who changed her dates of a travel a few times (not unusual).  She later confirmed purchasing a package (air, hotel, car) for a family of 5 one week prior to departure (somewhat fishy … especially for someone who was very price sensitive).  Her “confirmed” travel dates were from the 25th-30th of the month—and when she hadn’t checked in, as requested during that time, we reached out to her to find out that she was “already home” on the 29th.  Suspicious?  Very.  This lack of consistency—along with several other red flags—confirmed our suspicions that she was not being truthful and she was pulled from the study.  Again, to quote Judge Judy, “If you tell the truth, you don’t have to have a good memory.”

  • Engagement:  There are always going to be participants who choose to do the bare minimum in order to get their incentive.  However, a lack of engagement and openness—coupled with any additional red flags—requires some investigation.  Is the participant just taking the easy way out by answering questions in as few words as possible, or are they skipping key questions altogether?  Skipping key questions (e.g., “Tell us what you like best about product X”) could be a sign that they really don’t use product X after all.  Again, it’s important for the moderator to probe accordingly and if the probes go ignored … you guessed it … another red flag.

With online research (and plenty of Catfish) here to stay, we need to continue to be vigilant in crossing our T’s and dotting our i’s.  I, for one, am ready to catch them … hook, line and sinker.

Anne is CMB’s Qualitative Research Director.  She enjoys travel and thanks to DVR, never misses an episode of Judge Judy. Anne especially loves being able to truly “connect” with her research participants—it’s in her Midwestern blood.   

Learn more about Anne and her Qualitative Research team here.

Comments

Great post Anne -- It's not everday you get to read about #mrx and MTV in the same blog!  
 
Glad you try to find and catch the catfish.
Posted @ Tuesday, March 12, 2013 8:21 AM by Judy
Thanks, Anne, for drawing attention to this problem. Yes, a good researcher can often tell if something is fishy. 
One point about your demographic example. A lot of seniors travel and have more wealth than income. Sometimes the outliers may not be liars. 
Thanks again for the post!
Posted @ Tuesday, March 12, 2013 7:30 PM by Frankie Johnson
I was not aware of this label! But I've certainly learned these tips for catching the catfish. 
You did not add, although I'm sure you do this as well, that a strong recruiting process can do wonders in this area.  
 
Personally, I continue to rely on telephone screening even for online qual studies. Just one more filter. 
 
The other challenge, at times, is not people deliberately lying, but them not actually realizing what brand (for example) they are using. Even requests to "describe the bottle" don't always weed out these kinds of errors.  
 
Thanks for a great piece.
Posted @ Wednesday, March 20, 2013 9:04 AM by Susan Abbott
Appreciate the comments Frankie and Susan! I hope I communicated that this is certainly not an exact science--there are ways we can (and should) "investigate" if/when we become suspicious. Taking into consideration our target (like Seniors) would be critical, as would Susan's example of properly doing our "due diligence" of great screening and re-screening. Thanks again!
Posted @ Thursday, March 21, 2013 9:45 AM by Anne Hooper
Dear Anne, 
Your blog-piece was wonderful for me, I just catch this..:-) 
I did not know about CatFish-on line "form" anything before, but I had an experiance on one, and during my investigation I cathed the CatFish's another fb-wall with same pictures! It was funny, but really tragicomical, he said he is not the fake-guy, but...so. Thanks a lot, I just recognized I am not alone....:-)
Posted @ Friday, February 21, 2014 5:41 AM by Angela
Post Comment
Name
 *
Email
 *
Website (optional)
Comment
 *

Allowed tags: <a> link, <b> bold, <i> italics