This coming Wednesday is the American Marketing Association's Virtual Event: Unveiling Marketing Research's Future Online. As part of the event CMB's own Cathy Harrison will be debating hot topics with two of the other most widely read Tweeters in the market research community, Jeffrey Henning and Kathryn Korostoff. See below for all of the session details (AMA members can register for free here.)
Tweet Off: Three MR Tweeps Bicker, Badger & Bust Out of 140 Characters
Cathy Harrison, Sr. Project Manager, Chadwick Martin Bailey
Kathryn Korostoff, Research Rockstar
Jeffrey Henning, Vovici
Join three of the most prominent market researchers on Twitter as they debate contentious research issues of the day. Cathy Harrison (@VirtualMR), Kathryn Korostoff (@ResearchRocks) and Jeffrey Henning (@jhenning) will debate:
- The role of the market research department in a DIY (Do It Yourself) survey world. Point: MR is better off decentralized and embedded in other jobs. Counterpoint: Organizations need centralized MR expertise more than ever.
- Privacy in customer satisfaction research. Point: Follow up with every dissatisfied customer who takes a survey. Counterpoint: Customer satisfaction surveys are for measuring, not intervening.
- Market Research Online Communities. Point: The best MROCs are small, closed, branded and permanent. Counterpoint: Research communities can take a wide array of firms, each suited for different tasks.
- Social media market research. Point: Social media research will largely replace and supplant traditional consumer survey research. Counterpoint: Social media transforms and empowers surveys rather than replacing them.
Want to learn more about social media research? Watch our webinar featuring Georgeanna Liu and Chris James from AMD as they present a case study of how CMB conducted market research to support key social media strategy decisions and how AMD is using it today. Watch here.
Jeffrey Henning (@jhenning), Kathryn Korostoff (@ResearchRocks) and I (@VirtualMR) are participating in the complimentary AMA MRC Virtual Event: Unveiling Marketing Research's Future Online on June 23 (hope you can join us!). Our session, "Tweet Off! Three MR Tweeps Bicker, Badger & Bust out of 140 Characters", will involve debates on the prickliest topics in market research.
As a lead-in to the discussion, I am serving as the verdict judge for an early debate between Kathryn and Jeffrey on the subject of self-reported respondent data.
Kathryn's point: "Self-reported information is not perfect. But it is less perfect in some cases than in others."
Jeffrey's counterpoint: "Respondents, as a group, have sufficient ability to self-report to provide valuable data for market researchers."
My Take:
No one would argue that self-reported data is perfect and any knowledgeable researcher would agree that some self-reported measures are better than others.
In her argument, Kathryn makes several good points about the well-known biases of self-reported data, such as over-reporting, impacts to post-behaviors, and social desirability (supported by research including Social Desirability Bias and the Validity of Indirect Questioning by Robert J. Fisher © 1993 Journal of Consumer Research Inc.)
Still, there are biases in every methodology. And as researchers, one of our roles is to identify, control, and consider these biases in designing and evaluating market research studies.
For example, behavioral/observational research beyond descriptive observational variables (just reporting the behavior that is observed) requires researchers to make inferences and evaluations, which introduces biases. Or in the case of using sales data, customer databases or scanner data are robust but often do not include all distribution channels.
So, instead of discarding some ‘less-than-perfect' self-reported data, there are several ways researchers can address the shortcomings of self-reported data to yield useful conclusions:
1) Review the self-reported results with a critical eye as part of a comprehensive, multi-mode research plan
2) Calibrate purchase intentions based on historical data vs. actual behavior (for example, 75% of top 3 box purchase intent actually purchase)
3) Assign relative value to Inflated self-reported expenditures (high, medium, low) for use in a segmentation or product development/launch study
My Verdict: Respondents, as a group, have a sufficient ability to self-report for market researchers to draw useful conclusions - as long as actual magnitudes and values are interpreted relatively rather than literally.
Posted by Cathy Harrison. Cathy is a senior project manager at CMB, loves social media, music, and kick-butt research. You can follow Cathy on Twitter at @virtualMR