Voices: the CMB Blog

During a recent social media webinar, the question was raised “How do we convince clients that social media is statistically significant?”  After an involuntary groan, this question brought two things to mind:

• There are a lot of people working in social media research who do not understand the fundamentals of market research; and

• Why would anyone want to apply significance testing to social media data?

Apparently, there’s much debate in online research forums about whether significance testing should be applied to social media data.  Proponents argue that online panels are convenience samples and significance testing is routinely applied to those research results – so why not social media?  Admittedly that is true, but the ability to define the sample population and a structured data set should provide some test/retest reliability of the results.  It’s not a fair comparison.

I’m all for creative analysis and see potential value in sig testing applied to any data set as a way to wade through a lot of numbers to find meaningful patterns.  The analyst should understand that more things appear to be significant with big data sets so it might not be a useful exercise for social media.  Even if it can be applied, I would use it as a behind-the-scenes tool and not something to report on.

Anyone who has worked with social media data understands the challenging, ongoing process of disambiguation (removing irrelevant chatter). There are numerous uncontrollable external factors including the ever-changing set of sites the chatter is being pulled from.  Some are new sites where chatter is occurring but others are new sites being added to the listening tool’s database.   Given the nature of social media data, how can statistical comparisons over time be valid?  Social media analysis is a messy business.  Think of it as a valuable source of qualitative information.

There is value in tracking social media chatter over time to monitor for potential red flags.  Keep in mind that there is lot of noise in social media data and more often than not, an increase in chatter may not require any action. 

Applying sig testing to social media data is a slippery slope.  It implies a precision that is not there and puts focus on “significant” changes instead of meaning.  Social media analysis is already challenging – why needlessly complicate things?

Cathy is CMB’s social media research maven dedicated to an “eyes wide open” approach to social media research and its practical application and integration with other data sources. Follow her on Twitter at @VirtualMR 

On Sunday, The New York Times Magazine published a piece: How Companies Learn Your Secrets, by Charles Duhigg, author of the forthcoming The Power of Habit: Why We Do What We Do in Life and Business.  It’s an interesting article, especially for market researchers, and I recommend everyone take the time to read it.

Consumer "habits” are a big focus of the work we (market researchers) do as we seek to understand consumer behavior. From the perspective of the article, a large part of what we do is identify behavioral habits to help marketers find ways to insert their product or service into people's habit processes. 

In this blog, I want to focus on the insights the story shared about predictive analytics. Much of Duhigg's article looks at how Target conducts advanced analytics to identify data within their CRM system to predict whether a shopper is expecting a baby.  From a business process POV, and how we think about using predictive analytics, it’s important to point out a few relevant facts for market researchers:

1. It wasn’t a “fishing expedition”: The analysis started with a clear marketing benefit as the outcome – Target wanted to begin promoting itself to expectant mothers before the baby is born. As the article points out, by marketing to these families before the baby becomes public knowledge, Target can get beat the flood of marketers that begin pitching a range of products and services once the birth is entered into public record.  It was the marketing team that came to the analyst with a high-value opportunity.  The analyst did not create the winning marketing idea (“Hey! Let’s market to expectant mothers before the baby is born!”).  Instead, the analyst looked under every stone and in every corner of the data to find the key to unlock the opportunity.

2. The research didn’t stop with finding the key: The application of these insights required a lot more research to determine the best method of implementing the campaign.  For instance, Target ran several test campaigns to identify the best offers to send to the expectant mothers, and cycled through several messages to find just the right one in order to avoid revealing that Target was prying into the data.  Although the predictive analytics found the key, Target still relied on a comprehensive plan to make sure the findings were used in the best possible manner.

3. Don’t let this story increase your expectations: The Target approach has had a big impact on how the company markets to a highly valuable segment of shoppers.  It's a great success story, but it's also something that happened ten years ago.  While I’m sure the Guest Market Analytics team achieves many victories along the way, they also spent a lot of time reaching “dead-ends,” unable to find that magic key.  And most of the time, the predictive solution yields valuable but incremental gains, these high-profile stories are few and far between.

The article shares many interesting ideas and insights; the story about the re-positioning of Febreze highlights another great research success. I'm looking forward to reading Duhigg's book, and if it covers more of these thought provoking business cases, I expect we will be seeing Charles Duhigg’s name popping up in other discussions on market research.

Did you read the article? What do you think?

Did you miss our latest Webinar? Learn how Aflac Unleashed the Power of Discrete Choice, Positioning their Brand for the Future 

Posted by Jeff McKenna, Jeff is a Senior Consultant at CMB, and the creator and host of our Tools and Techniques Webinar Series.

When we set out, in our latest CMB Consumer Pulse, to discover how consumers are navigating the unprecedented range of options for accessing TV and movies, we knew some of the things we’d find out—e.g. the number of people who have completely cut the cord is still small, Netflix streaming is very popular, and many people are watching on smartphones and tablets.  However, some things surprised us:

Completely cutting the cord may be rare, but that’s not the only, or even the most important thing to worry about

People who have completely “cut the cord” entirely are a very small group (3% of the study’s population). The more compelling finding is the number of people who may be “on deck”— those who have tried Over-the-Top TV (OTT) and/or those who are considering reducing their pay TV service.

• More than half (54%) of respondents used the Internet to watch TV at least once.  (tweet this)

• Nearly half (43%) of pay TV subscribers said that they were at least somewhat likely to cut back on premium services (like Pay-Per-View or premium channels) in the next year. (tweet this)

This suggests that a big chunk of consumers’ TV-watching dollars could shift, even if their pay TV cords remain intact.  And, that awareness and perceived value of OTT have risen to a point where consumers of all kinds are investigating OTT as an option.

The biggest screen does NOT always win, and the definition of “watching TV” is changing

Everyone knows that new devices are impacting consumer behavior.  But our research suggests technology is driving fundamental changes in how people define the “TV watching experience.” For example:

• Among people who watch TV on tablets, 58% say that they watch TV on their tablet when they’re at home. (tweet this)

• Nearly two-thirds (63%) of those people say they’ve used a tablet even when they could have watched the same show on their TV. (tweet this)

There’s no indication that people will stop using their televisions entirely. But this does suggest that mobile devices have a role in the TV experience that transcends their mobility; and there are more scenarios where people will choose something other than the biggest available screen.

Not all elements of the TV watching experience are created equal

In our survey, consumers completed an exercise that forced them to trade off different attributes of a TV service provider based on what was most important to them.  Some of the findings were obvious (e.g. people want to pay less for their TV).  But other priorities offered more of a surprise:

• Streaming Content beats Owning Content:  Consumers prefer alternatives that rent/stream video over those where you download and own the content. (tweet this)  

• More Content beats Newest Content:  More consumers say having the largest catalog of content was more important than the fastest access to new content. (tweet this)

In short, technology and features matter but complexity is a barrier to switching over completely. Today, people who want an OTT solution have to manipulate a Rubik’s cube of devices, platforms/providers, and payment models.  However, that will change as more streamlined solutions come online. In the future, the most successful offerings will be those that provide a simple solution that's built around the things that matter most to consumers.

Check out this short video:

We surveyed nearly 1,500 consumers in the U.S. to get a pulse on their TV and video viewing habits and what it means for the future of OTT. Learn more, download our Consumer Pulse: The New Age of Television

Posted by Jon Giegengack, Jon is a Director at CMB and leads research for digital entertainment companies in television, music, videogames, and social media. As a veteran cord cutter, he knows the good, the bad and the ugly of today’s TV options.

A topic near and dear to my heart just became nearer and dearer. Take a look at one of the classroom assignments my kindergarten daughter recently brought home from school:

Isn’t this amazing? Kindergartners are learning the basics of market research! Ever since I got into this business of market research, whenever someone asked me about my job, the answer nearly always comes down to: “I write the survey questions that people ask when calling you during dinner time.”

But, not anymore! My answer can now be: “You know how you might want to learn what is the most popular animal at the carnival? I’m the one who writes the questions, counts the tally marks, and creates the bar charts to show you what is most popular.” How cool is that?

Wish your research had more pictures of ponies, or just more visual interest? Check out our webinar: Appearance Counts: How to Tell a More Visually Compelling Story with your Data

Posted by Jeff McKenna, Jeff is a Senior Consultant at CMB, and the creator and host of our Tools and Techniques Webinar Series.

Last week I blogged about the positive intentions of consumers who take the time to participate in customer feedback surveys.  One of the more disappointing findings is that most companies fail to respond when customers give poor scores or negative feedback.  On this point, we asked respondents to recall the last time they provided negative feedback or a complaint in a customer satisfaction interview, and whether a company provided a response.  A majority say they did not receive any response—just 35% recall any type of response from the company.

This was quite a surprise.  Companies are investing tens of thousands of dollars in their customer feedback programs, but many are missing the point.  When customers complain it’s because they feel disappointed or disrespected, treating them like an anonymous statistic is another kind of service failure. How do you teach a company or brand to mind their P’s and Q’s? Maybe we need an Emily Post for corporations!

When you think about it, customers who take the time to respond to a customer satisfaction interview have given time and effort to provide information that should be valuable to the company.  Those who’ve had a bad experience could easily turn and walk away.  Instead they respond to a request for feedback.  Whether it’s a complaint about a service failure, a suggestion, or even a review of a positive experience, companies need to let customers know they’re being heard.

And yet the majority of companies are turning their back on these people!

The thing is, technology makes it easy to manage and respond to feedback; and most people don’t expect much.  In fact, confirmation and a commitment to improve will go a long way to show appreciation.  And of course, a simple “thank you” can’t hurt.

Download the full report: Customer Satisfaction Surveys: Open the Door to Customer Engagement

Posted by Jeff McKenna, Jeff is a Senior Consultant at CMB, and the creator and host of our Tools and Techniques Webinar Series.