Voices: the CMB Blog

Originally posted on Research Access

Earlier this month, Chris Neal of Chadwick Martin Bailey shared with members of the New England chapter of the Marketing Research Association tips for running mobile diary studies, based on lessons learned from a recent project.For the Council for Research Excellence (CRE), CMB studied mobile video usage to understand:

• How much time is spent on mobile devices watching TV (professionally produced TV shows)?

• Does this cannibalize TV set viewing?

• What motivates consumers to watch on mobile?

• How can mobile TV viewing be accurately tracked?

The research included a quantitative phase with two online surveys and mobile journaling, followed by a series of home ethnographies. The quant work included a screening survey, the mobile diary, and a final online survey.

• The screening survey was Census balanced to estimate market size, with three groups recruited for comparison: those without mobile devices (smartphones or tablets), those with mobile devices who don’t watch TV on them, and those with mobile devices that they watch TV on. The total number of respondents was 5,886.

• The mobile diary activity asked respondents to complete their journal 4 times a day for 7 days.

• A final attitudinal survey was used to better understand motivations and behaviors associated with decisions about TV watching.

Along the way, CMB learned some valuable best practices for mobile diary studies, including tips for recruiting, incentives, design and analysis. The 10 key lessons learned:

1. Mobile panels don’t work for low incidence – Take care when using mobile panels – given the small size of many mobile panels, you may have better luck recruiting through traditional online panels, as CMB did. For this study, it was because of the comparatively low incidence of actual mobile TV watching.

2. Overrecruit – You will lose many recruits to the journaling exercise when it comes time to downloading the mobile diary application. As a general rule, over-recruit by 100% – get twice the promises of participation that you need. Most dropout occurs after the screening and before the participant has recorded a single mobile diary entry. For many members of online survey panels, journaling is a new experience. The second biggest point of dropout was after recording 1 or 2 diary entries.

3. Keep it short – To minimize this dropout, you have to keep the diary experience as short as possible: no more than 3 to 5 minutes long. The more times you ask participants to complete a diary each day, the greater the dropout rate.

4. Think small screen – Make sure the survey is designed to provide a good experience on small screens – avoid grids and sum-allocation questions and limit open-ended prompts and use of images. Use vertical scales instead of horizontal scales. “Be wary of shiny new survey objects for smartphone survey-takers,” said Chris. Smartphone users had 5 times the dropout rate of tablet or laptop users in this study. Enable people to log on to their journal from whatever device they were using at the time, including their computer.

5. Beware battery hogs – When evaluating smartphone apps, be wary of those that drain battery life by constantly logging GPS location. Check the app store reviews of the application.

6. Keep consistent – Keep the diary questionnaire the same for every time block, to get respondents into the habit of answering it.

7. Experiment with incentives to maximize participation – Tier incentives to motivate people to stick with the study and complete all time blocks. To earn the incentive for the CMB study, Chris said that respondents had to participate at least once a day for all 7 days, with additional incentives for every journal log entered (participants were reminded this didn’t have to involve actual TV watching, just filling out the log). In the end, 90% of journaling occasions were filled out.

8. Remind via SMS and email – In-app notifications are not enough to prompt participation. Use email and text messages for each time block as well. Most respondents logged on within 2 hours of receiving a reminder.

9. Use online surveys for detailed questions – Use the post-journaling survey to capture greater detail and to work around the limits of mobile surveys. You can then use these results to “slice and dice” the journal responses.

10. Weight by occasions – Remember to weight the data file to total occasions not total respondents. For missing data, leave it missing. Develop a plan detailing which occasion-based data you’re going to analyze and what respondent-level analysis you are going to do. You may need to create a separate occasion-level data file and a separate respondent-level data file.

Properly done, mobile diary studies provide an amazing depth of data. For this project, CMB captured almost 400,000 viewing occasions (mobile and non-mobile TV watching), for over 5 million occasion-based records!

Interested in the actual survey results? CRE has published the results presentation, “TV Untethered: Following the Mobile Path of TV Content” [PDF].

Jeffrey Henning, PRC is president of Researchscape International, a market research firm providing custom surveys to small businesses. He is a Director at Large on the MRA Board of Directors; in 2012, he was the inaugural winner of the MRA’s Impact award. You can follow him on Twitter @jhenning.

I recently came across an article focused on defunct exhibits at Disney parks. I’m a native Floridan so I flipped through the accompanying slide show with fond memories. And there it was...my all-time childhood favorite—Horizons at Epcot Center. From the robot butler to the holographic telephone, Horizons revealed a future full of promise, excitement, and funky monotone jumpsuits. 

It’s been 30 years, the future is now the present, and I don’t have a robot butler. Disappointing yes, but on the other hand, we do have the Roomba and I will argue Apple’s FaceTime is likely better than a hologram. So I think we can agree many companies have made serious innovations in the last few decades—they’ve understood that incremental change means incremental growth, and they’ve pushed the limits. Although product development is critical for companies to compete and grow, it also carries high risks, because it represents a big investment into new and unfamiliar territory—it’s crucial to get it right.

While we aren’t all Imagineers, there are strategies for new product and service development that have proven successful in a rapidly changing market—these strategies form the basis of our Best Practices in New Product Development. Two of these Best Practices are below:

1. Use advanced techniques that emulate real world trade-offs: In real life, people don’t evaluate the importance of individual features or attributes. They make choices between/among products. The more closely research emulates this process, the more accurate the findings will be. What people say they prefer, and what they actually choose, are often not the same thing. That’s why we use trade-off techniques (e.g., discrete choice) that let us derive the most important and relevant preferences as well as sophisticated data mining techniques that help us to create more accurate predictive models.

2. Build flexibility into the research: If you’re using trade-off techniques, channel Walt Disney himself (“if we can dream it, we can do it”) by including features that fall outside of current capabilities. This lets you mimic the current market and simulate a future market where these feature become available. So while you might not be ready to “do it,” if you’ve dreamed it, you can test it! That’s why, when appropriate, we build a user-friendly simulator. These simulators allow design decision-makers to run “what if” scenarios, providing additional insight when changes occur (e.g., a competitor responds with a new product, prices change, or when the technology to realize your stretch features catches up with your dreams).

We can’t promise your product development research will live as long as Horizons (16 magic filled years) but we can help ensure it’s useful for both the short and longer-term (at least until we all get our robot butlers). Check out the video below to learn how we help our make sure their new product development efforts are a success:

CMB New Product and Service Development from CMBinfo on Vimeo.

Athena is a Project Director at CMB, she looks awesome in a jumpsuit and is patiently waiting for her favorite Disney character, Donald Duck, to make a comeback.

At South Street Strategy Group, we love working on innovation projects as well as presenting  and exchanging ideas with fellow practitioners at innovation-related events.Earlier this year, South Street’s Managing Partner, Mark Carr, presented at the Front End of Innovation Conference in Boston along with Judy Melanson from CMB (South Street’s sister market research firm), and Jeremy Palmer, Vice President of Tauck (our client). The topic was Focused Innovation: Creating New Value for a Legacy Brand and we shared the success story of how we’ve helped Tauck ignite growth worldwide through its baby boomer focused Culturious brand. You can check out a recording of that presentation here.

Next week, we will be heading to Santa Clara, California for the Back End of Innovation Conference (Nov. 18-20) and can’t wait to get immersed in all of the activities – keynote sessions, breakouts, networking breaks, and meals with friends old and new. If you’re in the area or will be attending the conference, let us know. We’d love to meet up! 

South Street Strategy Group, an independent sister company of Chadwick Martin Bailey, integrates the best of strategy consulting and marketing science to develop better growth and value delivery strategies. 

For more on how CMB and South Street Strategy help executives strengthen their brands and positions, identify opportunities, segment their markets, overcome challenges and drive brand zeal and customer loyalty, check out a few of our case studies here.

Last week my Facebook feed was filled with tragic news. The Alchemist Brewery, in Waterbury, Vermont (where I happen to live), is closing its retail store—no more dropping in to taste their beloved Heady Topper. Luckily, my fellow craft beer lovers can find some solace in the fact that while the retail space will close, the brewery will turn their focus to canning the delicious brew. But not too long ago the only canned beers you could get were decidedly not craft—they were mostly watery domestic lagers without a hop in sight. However, if you’ve been to a liquor store in the past few years you’ve probably noticed nearly every style of beer can be found in cans. In fact, a number of domestic standards have also changed the look and feel of their cans—there are bow-tie shaped cans, wide mouth cans, and cans that change color as the beer gets cold. It’s a canned beer revolution!A decade spent in the Market Research industry changes the way you look at everyday things, like these changes to the beer market, so I decided to do a bit of secondary research. First, I wanted to understand what motivates people to purchase canned beer. The results of my highly unscientific online search are here:

• Beer cans work better with outdoor lifestyles (hiking, boating, etc.)

• Cans keep beer fresher

• Canned beer weighs less/is easier to carry

• It saves money on packaging and shipping

• Cans block light which is destructive to all the things that make beer so tasty

• Cans are sturdier (won’t break in your cooler during your road trips)

• Cans chill faster and keep beer colder

• Cans are easier to recycle

• Their favorite retail beer store closes (n=1)

So canning is great idea, right? In 2002 just one craft brewery canned their beer, now there are roughly 330 different craft breweries canning a little over 1,000 different brews. But, when I asked many of my fellow Vermonters whether they’d drink their craft beer from a can, I was told “no, that ruins the taste.”  

Ah, the taste…which is the reason aficionados buy craft beer in the first place. Luckily, the brewers of Sam Adams have been relentlessly pursuing beer can innovation, breaking down the key barrier that is that metallic taste. Just this summer, they debuted the SamCan:

“the result of two years of ergonomic and sensory research and testing…the new can design aims to provide a drinking experience that is a little closer to the taste and comfort of drinking beer from a glass. What you’ll notice: The larger, wider lid helps open your mouth allowing for more air flow during the drinking experience. The can opening is located slightly farther away from the edge of the lid, placing it closer to the drinker’s nose to help accentuate the hop aromas. The hourglass ridge creates turbulence (like our patented Perfect Pint glass) which “pushes flavor out of the beer” and the extended lip places the beer at the front of your palate to maximize enjoyment of the sweetness from the malt.”

The amazing thing is, that despite their research investment, Jim Koch, the co-founder and chairman of the Boston Beer Company, the producers of Samuel Adams, announced that they plan to "make the patent-pending design available, without any royalty or license fee, to all craft brewers who would like to use this can."

That’s great news for this Vermonter, who will continue to buy my craft brews in a can because let’s face it, it’s much easier to hike with backpack filled with canned beer than it is to carry a bottle.

Sam is a Data Manager at CMB, she loves hiking the green mountains of Vermont with or without a tasty brew.

CMB is observing #Movember with some awesome mustachery. Check out the hairy competition here.

It is human nature to love a good story, and it’s no wonder that for centuries storytelling has been a powerful force for human learning, change and advancement. As business strategists, we use stories in a variety of ways in both strategy development and implementation.In strategy development, we often learn from case studies—stories of relevant successes and failures as well as analogs from other industries—to help inform our thinking on problems we are helping our clients solve.

In strategy implementation, we use stories as a catalyst for organizational buy-in and change. The most effective business leaders we work with are expert at communicating a vision with clarity and passion and guiding organizations to implement strategies using stories, ongoing dialogues and narratives. They don’t simply make edicts or repeat the same message over and over. They use their influence and credibility to communicate intentions, future direction and strategies, and invite everyone to participate, interact, and become part of the continuing story. 

Good stories are memorable and engaging and completely believable. They connect us and help us understand and relate to others, whether those others are involved in telling the story or simply listening to it. When leaders want to rally teams and employees to implement new initiatives, they need to be authentic and communicate with conviction and energy in order to gain trust and commitment. By carefully crafting and honing messages and stories that they share and adapt over time, leaders become more effective at connecting and teaching,  guiding and motivating others through implementation successes, challenges and setbacks.

If you are interested in this topic and related research, below are some of my own favorites, from people I’ve worked with or learned from recently:

• Guide Innovation Through Storytelling  Rob Salafia and David Sollars aptly call themselves story archeologists. Their process really is about digging underneath the outer layer of the what’s and why’s behind change, to help business leaders uncover their own narratives that will motivate and engage larger audiences. South Street recently worked with Rob and Dave in facilitating a large innovation workshop at a top 5 U.S. health care insurance client.

• Strategy Made Simple - The 3 Core Strategy Questions  John Hagel insightfully points out in this blog post that “the ultimate form of differentiation is a compelling narrative—a unique and unfolding opportunity for the audience that invites their participation to help shape the outcome.” This and some of his other entries discuss modern strategy and the role of an ongoing narrative that’s focused on external audiences, not the executive suite.

• Conversational Intelligence, by Judith E. Glaser. This book is a great place to explore the cultural transformations that companies must go through in order to embrace change. Big hint here: it all leads back to how company leadership approaches change and the narrative around it.

What role do stories play in your willingness to get on board with change? Can you identify one strategic issue where storytelling can support your goal?

Jennifer is a Director at  South Street Strategy Group. She recently received the 2013 “Member of the Year” award by the Association for Strategic Planning (ASP), the preeminent professional association for those engaged in strategic thinking, planning and action.