WELCOME TO OUR BLOG!

The posts here represent the opinions of CMB employees and guests—not necessarily the company as a whole. 

Subscribe to Email Updates

Debating the Usefulness of Self-Reported Market Research Data

Posted by Cathy Harrison on Thu, Jun 03, 2010

Jeffrey Henning (@jhenning), Kathryn Korostoff (@ResearchRocks) and I  (@VirtualMR) are participating in the complimentary AMA MRC Virtual Event: Unveiling Marketing Research's Future Online on June 23 (hope you can join us!). Our session, "Tweet Off! Three MR Tweeps Bicker, Badger & Bust out of 140 Characters", will involve debates on the prickliest topics in market research.

As a lead-in to the discussion, I am serving as the verdict judge for an early debate between Kathryn and Jeffrey on the subject of self-reported respondent data. 

Kathryn's point:  "Self-reported information is not perfect. But it is less perfect in some cases than in others."

Jeffrey's counterpoint: "Respondents, as a group, have sufficient ability to self-report to provide valuable data for market researchers."

My Take: 

No one would argue that self-reported data is perfect and any knowledgeable researcher would agree that some self-reported measures are better than others. 

In her argument, Kathryn makes several good points about the well-known biases of self-reported data, such as over-reporting, impacts to post-behaviors, and social desirability (supported by research including  Social Desirability Bias and the Validity of Indirect Questioning  by Robert J. Fisher © 1993 Journal of Consumer Research Inc.)   

Still, there are biases in every methodology.  And as researchers, one of our roles is to identify, control, and consider these biases in designing and evaluating market research studies.   

For example, behavioral/observational research beyond descriptive observational variables (just reporting the behavior that is observed) requires researchers to make inferences and evaluations, which introduces biases.  Or in the case of using sales data, customer databases or scanner data are robust but often do not include all distribution channels.

So, instead of discarding some ‘less-than-perfect' self-reported data, there are several ways researchers can address the shortcomings of self-reported data to yield useful conclusions:

1) Review the self-reported results with a critical eye as part of a comprehensive, multi-mode research plan

2) Calibrate purchase intentions based on historical data vs. actual behavior (for example, 75% of top 3 box purchase intent actually purchase)

3) Assign relative value to Inflated self-reported expenditures (high, medium, low) for use in a segmentation or product development/launch study

My Verdict:  Respondents, as a group, have a sufficient ability to self-report for market researchers to draw useful conclusions  - as long as actual magnitudes and values are interpreted relatively rather than literally.

 

Posted by Cathy Harrison.  Cathy is a client services executive at CMB, loves social media, music, and kick-butt research.  You can follow Cathy on Twitter at @virtualMR

Topics: methodology, data collection