WELCOME TO OUR BLOG!

The posts here represent the opinions of CMB employees and guests—not necessarily the company as a whole. 

Subscribe to Email Updates

BROWSE BY TAG

see all

CMB Welcomes Kathy Ofsthun as Head of Expanding Qualitative Strategy + Insights Practice

Posted by Megan McManaman

Fri, Sep 16, 2016

We're excited to announce that Kathy Ofsthun has recently returned to CMB as VP of Qualitative Kathy_final_casual_1_of_1-2.pngStrategy + Insights after spending almost 5 years at C Space as VP of Client Services. Kathy is back to head up CMB’s expanding Qualitative practice—growing clients’ businesses by bringing them closer to their customers.

Kathy  brings a wealth of experience and knowledge in qualitative methods, qual/quant synthesis, and creating connections and strategic partnerships. Her deep research expertise was developed through two decades of work with multinational companies, including a year in Shanghai managing the C Space APAC office. Her work has focused on topics as varied as New Product Development, Shopper Insights, Packaging, Brand Positioning, and Segmentation.

 "I’m thrilled to rejoin CMB at this exciting time," Kathy says. "As consumers move into the driver's seat, marketers and innovators need and want to be closer to their customers, understanding who they are, hearing their needs and incorporating their ideas. By including customers in co-creation of new products, communications development and more, brands can either fail faster or adapt and succeed. I look forward to helping clients leverage the voice of the consumer in order to achieve growth.”

Kathy will be headed to MRA's CRC in San Francisco next week, give us a ring, email us, or stop by booth 407 to say hello!

Are you going to CRC and want to get the most out of it? Check out Julie Kurd's blog:  How To Not Flunk the MRA Corporate Researchers Conference

Topics: strategy consulting, qualitative research, Kathy Ofsthun

Can Facial Recognition Revolutionize Qualitative?

Posted by Will Buxton

Wed, Aug 03, 2016

Full disclosure: I’m an Android and Google loyalist, but please don’t hold that against me or the rest of my fellow Android users, who, by the way, comprise 58% of the smartphone market share in the United States. As a result of my loyalty, I’m always intrigued by Google’s new hardware and software advancements, which are always positioned in a way that leads me to believe they will make my life easier. Some of the innovations over the years have in fact lived up to the hype, such as Google Now, Google Drive, and even Google Fusion, while others such as Google Buzz and Google Wave have not.

As a researcher, last year’s launch of Google Photos caught my eye. Essentially, Google
Photos now utilizes facial recognition software to group or bunch your photos based on people in them, scenery (i.e., beaches and Google_Photos_icon.svg-1.pngmountains) and even events (i.e., weddings and holidays). To activate the facial recognition feature, all you have to do is tag one photo with an individual’s name and all other photos with that person will be compiled into a searchable collection. Google uses visual cues within the photos and geotagging to create other searchable collections. While these features might not seem extraordinary—I can see who was the most frequent star of my photos (my enormous cat) or where I most commonly take photos (honeymoon sans enormous cat)—I began to imagine the possible impact these features could have on the market research industry.

Visual ethnographies are one of many qualitative research options we offer at CMB. This is a rich form of observation, and, for some companies, it can be cost prohibitive in nature, especially ones focused on a “cost-per-complete.” But, what if there was a way to remove some of the heavy lifting of a customer journey ethnography by quantifying some of the shopping experience using technology that could track date/time, location, shopping layout, products viewed, order in which products are viewed, and so on, all through recognition software? Would the reduction in hours, travel, and analysis be able to offset the technological costs of these improvements?

Market research, and, in particular, qualitative research have always been a combination of art and science, and to expect any technological advancement to adequately perform any cogent analyses is a bit premature and perhaps too reminiscent of The Minority Report. (I don’t think it worked out well). But the promise of these powerful tools makes it an exciting time to  be a qualitative researcher!

Will Buxton is a Project Manager on the Financial Services team. He enjoys finding humor in everyday tasks, being taken seriously, and his enormous cat.

Learn more about how our dedicated Qualitative practice helps brands Explore, Listen, & Engage.

 

 

 

Topics: methodology, qualitative research, mobile, storytelling, customer journey

Move Over Cupid: A Qualitative Researcher’s Guide to Valentine’s Day

Posted by Eliza Novick

Tue, Feb 09, 2016

eliza_blog_image.pngAs Valentine’s Day ticks closer, I’m reminded of my best and worst dates over the years. At best, I’ve enjoyed rosé, cheese, and interesting conversations; at worst, I had a beer spilled on me and endured lots of awkward pauses. Through all the ups and downs, I’ve perfected a few tricks that can help make a date a great success and avoid your typical first date pitfalls. Best of all, these are tricks that I can apply to my work as a qualitative researcher!

Moderating a focus group is kind of like going on a blind date with eight people at once while your boss watches. Yes, it can be awkward, but it’s critical that respondents really connect with the moderator to ensure that our clients get reliable findings. With that in mind, here are my top three tips for making it through a first date and for wow-ing clients by getting the most out of your qualitative research:

  1. Ask open-ended questions: Nobody likes stilted conversation, but sometimes it can feel hard to avoid. Rather than asking close-ended questions that end in one-word answers, try asking people to describe an experience. “What kind of things have you been cooking recently?” tends to get a lot more traction than, “Do you like to cook?” Likewise, “Tell me about a time you paid for an unanticipated medical expense” can take you (and your clients) much further than “Have you ever had an unanticipated medical expense?” Putting the emphasis on sharing a story encourages people to give detailed responses and speak genuinely about their interests and experiences.
  2. Don’t try to cover too much ground: Meeting new people can be overwhelming—there’s a lot to digest. So, I’ve found that it’s best to keep the conversation simple. Unlike the unfortunate fellow who asked me rapid-fire questions for two hours over drinks, try asking follow-up questions on one topic. This lets you get to know someone better and discover interesting details that you wouldn’t uncover if you were speeding through topics. It also works in qualitative since your respondents are coming into the conversation with virtually no context. They weren’t privy to the hours of client calls, discussion guide revisions, and marketing materials like the research team was. While it’s tempting to cram as much content as possible into the discussion guide, nine times out of ten, clients find more value in clear, detailed findings than high-level, scattered anecdotes. Besides, speeding through different topics makes it difficult to identify patterns. So, do everyone a favor—slow down, and see where the conversation takes you.
  3. Trust your gut: If something doesn’t seem right, trust yourself. If you’re on a date and things aren’t going well, it’s ok to leave early. Likewise, if your carefully laid research plans are not panning out as you had planned, it’s ok to take a different route. Try phrasing a question a different way. Or, if you have a sense that someone in the group disagrees with a point but is too shy to say so, ask them if they’ve got anything they’d like to share. Not only will this show your respondents that you’re listening and care about what they have to say, it will also elicit more honest responses that lead to better findings (and happy clients).

Qualitative research, like dating, is really about connecting with people—we get the best results when respondents feel they can relate to us researchers on a personal level. So, don’t be afraid to put yourself out there! Take your time, listen to the data you’re getting, and trust yourself. Easy!

Eliza is a qualitative researcher at CMB. In addition to applying her dating life to her work, she likes to be outside, read books, and cook. 

For the latest Consumer Pulse reports, case studies, and conference news, subscribe to our monthly eZine.

Subscribe Here!

Topics: qualitative research, research design

Embracing Mobile Market Research

Posted by Brian Jones

Thu, Jul 23, 2015

Who are the mobile consumers?

mobile research, cmbLet’s get this straight: I am not addicted to my smartphone. Unlike so many of my fellow train commuters who stare zombie-eyed into their small screens, I am not immersed in a personal relationship with pixels. I have an e-Reader for that. But, my smartphone IS my lifeline.I’ve come to depend exclusively on my phone to keep me on-time and on-schedule, to entertain me (when not using my e-Reader), to stay in touch with family and friends, and to keep up-to-date with my work email. It’s my primary source for directions, weather, news, photography, messaging, banking, and a regular source for payment, shopping, and ticketing/reservations. I haven’t purchased a PC in nearly a decade, and I don’t have a landline. I also use my smartphone to take market research questionnaires, and I am far from alone. 

Data around smartphone usage aligns with my personal experience. In a recent CMB online study of U.S. consumers, optimized for mobile devices, 1 in 6 Millennials completed the questionnaire on a smartphone. Other studies report similar results. This example illustrates the issue with representativeness. Major panel vendors are seeing over half of Millennials joining their panels via a mobile device. 

mobile research, cmb

How do we adapt?

Much has been hypothesized about the future of market research under the new paradigm of mobile commerce, big data, and cloud services. New technologies and industry convergence (not just mobile) have brought sweeping changes in consumer behaviors, and market researchers must adapt.

A key component of successful adaptation will be greater integration of primary market research with other data streams. The promise of passive or observational data is captivating, but it is largely still in the formative stages. (For more on passive data, check out our recent webinar.) We still need and will likely always need active “please tell me” research. The shift from phone to online data collection has quickly been replaced with the urgency of a shift to mobile data collection (or at least device agnostic interviewing). Our industry has lagged behind because the consumer experience has become so personalized and the trust/value equation for tapping into their experiences is challenging. Tackling mobile market research with tactical solutions is a necessary step in this transition.

What should we do about it?  

  1. Understand your current audience. Researchers need to determine how important mobile data collection is to the business decision and decide how to treat mobile respondents. You can have all respondents use a mobile device, have some use a mobile device, or have mobile device respondents excluded. There are criteria and considerations for each of these, and there are also considerations for the expected mix of feature phones, smartphones, tablets, and PCs. The audience will determine the source of sample and representation that must be factored into the study design. Ultimately, this has a huge impact on the validity and reliability of the data. Respondent invitations need to include any limitations for devices not suitable for a particular survey.
  2. Design for mobile. If mobile participation is important, researchers should use a mobile first questionnaire design. Mobile optimized or mobile friendly surveys typically need to be shorter in length, use concise language, avoid complex grids and answering mechanisms, and have fewer answer options, so they can be supported on a small screen and keep respondents focused on the activity. In some cases,questionnaire modularization or data stitching can be used to help adhere to mobile design standards.
  3. Test for mobile. All questions, images, etc. need to display on a variety of screen sizes and within the bandwidth capacity of the devices that are being used. Android and iOS device accommodation covers most users. If app based surveys are being used, researchers need to ensure that the latest versions can be downloaded and are bug-free. 
  4. Apply data protection and privacy standards. Mobile market research comes with a unique set of conditions and challenges that impact how information is collected, protected, and secured. Research quality and ethical guidelines specific to mobile market research have been published by CASRO, ESOMAR, the MMRA (Mobile Marketing Research Association), and others.
  5. Implement Mobile Qualitative. The barriers are lower, and researchers can leverage the unique capabilities of mobile devices quite effectively with qualitative research. Most importantly, willing participants are mobile, which makes in-the-moment research possible. Mobile qualitative is also a great gateway to explore what’s possible for mobile quantitative studies. See my colleague Anne Hooper’s blog for more on the future of qualitative methodologies.
  6. Promote Research-on-Research. Experts need to conduct and publish additional research-on-research studies that advance understanding of how to treat mobile respondents and utilize passive data, location tracking, and other capabilities that mobile devices provide. We also need stronger evidence of what works and what doesn’t work in execution of multi-mode and mobile-only studies across different demographics, in B2B studies, and within different countries.

But perhaps the most important thing to remember is that this is just a start. Market researchers and other insight professionals must evolve from data providers to become integrated strategic partners—harnessing technology (not just mobile) to industry expertise to focus on decision-making, risk reduction, and growth.

Brian is a Senior Project Manager for Chadwick Martin Bailey, the photographer of the image in this post, and an 82 percenter—he is one of the 82% of mobile phone owners whose phone is with them always or most of the time. 

Watch our recent webinar that discusses the results of our self-funded Consumer Pulse study on the future of the mobile wallet. 

Watch Here!

Topics: methodology, qualitative research, mobile, research design

Qualitative Research Isn't Dying—It's Evolving

Posted by Anne Hooper

Wed, May 06, 2015

qualitative research, anne hooperBack in 2005, Malcolm Gladwell told us that focus groups are dead. Just last November, Jim Bryson, CEO of 20/20 Research, questioned whether qualitative research was thriving or dying: If we take a narrow, more traditional view that qualitative is defined by the methods of face-to-face focus groups or interviews, particularly those held in a qualitative facility, then the case can be easily made that qualitative is dying.”

To all of this, I say: wait, what?! Qualitative is dying? I refused to believe it, so I embarked on a journey to explore where qualitative has been, and more importantly, where it’s going. During my research, I found plenty of evidence to support the fact that qualitative is not, in fact, dying. Great news, right? (Especially for me, because if it were true, I just might be out of a job I love.)I took a look at the fall 2014 Greenbook Research Industry Trends (GRIT) Report and focused on the data from Q1-Q2 of 2013 and Q1-Q2 2014. In this data, I learned:

  • The use of traditional in-person focus groups increased from 60% (Q1-Q2 2013) to 70% (Q1-Q2 2014).
  • Within the same time period, the use of in-person, in-depth interviews increased from 45% to 53%.
  • Interviews and groups using online communities increased from 21% to 24%.
  • The use of mobile qual (e.g., diaries, image uploads) increased from 18% to 24%.

Yes, it’s important to note that not all qualitative methodologies saw an increase in usage within this timeframe. In fact, there was a decrease in the usage of telephone IDIs, in-store shopping/observations, bulletin board studies, both chat-based and webcam-based online focus groups, and telephone focus groups.  All this notwithstanding, I think it’s fair to say that qualitative is still very much alive and well.

So why do people keep talking about qualitative dying? We can’t deny that there are a number of factors that affect how and when we use qualitative methodologies today (technology, access to big data, and text analytics are a few). But, this doesn’t mean qualitative is disappearing as a discipline. Qualitative is evolving at a rapid pace and feels more relevant than ever. Sure, we need to keep up with client demands for faster and cheaper research, but there will always be a need for the human mind (i.e., a qualitative expert) to analyze and synthesize the data to provide meaning and context behind the way people think and behave—and that is where actionable insights are born.   

Now that we know qualitative really isn’t dying, what does 2015 (and beyond) hold for us? The future is about truly integrated research—in which qualitative and quantitative are consistently, thoughtfully, and purposefully used together to provide well-rounded, actionable insights. We’re poised to do exactly that with our dedicated analytics team and network of expert industry qualitative partners. By using two equally important disciplines that are both alive and well, we can provide our clients critical insights they can really use. Far from killing off qualitative insights, technology and an evolving marketplace are helping make qualitative insights even stronger.

Anne Hooper is the Qualitative Research Director at CMB. After recently finding out that her 13 year old daughter did a quantitative assessment of her Jazz Band’s upcoming Disney trip itinerary, she’s determined that an intervention may be in order.

Topics: methodology, qualitative research

Qualitative, Quantitative, or Both? Tips for Choosing the Right Tool

Posted by Ashley Harrington

Wed, Aug 06, 2014

quantitative, qualitative, methodologyIn market research, it can occasionally feel like the rivalry between qualitative and quantitative research is like the Red Sox vs. the Yankees.  You can’t root for both, and you can’t just “like” one.  You’re very passionate about your preference.  But in many cases, this can be problematic. For example, using a quantitative mindset or tactics in a qualitative study (or vice versa) can lead to inaccurate conclusions. Below are some examples of this challenge—one that can happen throughout all phases of the research process: 

Planning

Clients will occasionally request that market researchers use a particular methodology for an engagement. We always explore these requests further with our clients to ensure there isn’t a disconnect between the requested methodology and the problem the client is trying to solve.

For example, a bank* might say, “The latest results from our brand tracking study indicate that customers are extremely frustrated by our call center and we have no idea why. Let’s do a survey to find out.”

Because the bank has no hypotheses about the cause of the issue, moving forward with their survey request could lead to designing a tool with (a) too many open-ended questions and (b) questions/answer options that are no more than wild guesses at the root of the problem, which may or may not jibe with how consumers actually think and feel.

Instead, qualitative research could be used to provide a foundation of preliminary knowledge about a particular problem, population, and so forth. Ultimately, that knowledge can be used to help inform the design of a tool that would be useful.

Questionnaire Design

For a product development study, a software company* asks to add an open-ended question to a survey: “What would make you more likely to use this software?” or “What do you wish the software could do that it can’t do now?”

Since most of us are not engineers or product designers, this question might be difficult for most respondents to answer. Open-ended questions like these are likely to yield a lot of not-so-helpful “I don’t know”-type responses, rather than specific enhancement suggestions.

Instead of squandering valuable real estate on a question not likely to yield helpful data, a qualitative approach could allow respondents to react to ideas at a more conceptual level, bounce ideas off of each other or a moderator, or take some time to reflect on their responses. Even if the customer is not a R&D expert, they may have a great idea that just needs a bit of coaxing via input and engagement with others.

Analysis and Reporting

In reviewing the findings from an online discussion board, a client at a restaurant chain* reviews the transcripts and states, “85% of participants responded negatively to our new item, so we need to remove it from our menu.”

Since findings from qualitative studies are not necessarily statistically significant, using the same techniques (e.g., descriptive statistics and frequencies) is not ideal as it implies a level of precision in the findings that is not necessarily accurate. Further, it would not be cost-effective to recruit and conduct qualitative research with a group large enough to be projectable onto the general population.

Rather than attempting to quantify the findings in strictly numerical terms, qualitative data should be thought of as more directional in terms of overall themes and observable patterns.

At CMB, we root for both teams. We believe both produce impactful insights, and that often means using a hybrid approach. We believe the most meaningful insights come from choosing the approach or approaches best suited to the problem our client is trying to solve. However, being a Boston-based company, we can’t say that we’re nearly this unbiased when it comes to the Red Sox versus the Stankees Yankees.

*Example (not actual)

Ashley is a Project Manager at CMB. She loves both qualitative and quantitative equally and is not knowledgeable enough about sports to make any sports-related analogies more sophisticated than the Red Sox vs. the Yankees.

Click the button below to subscribe to our monthly eZine and get the scoop on the latest webinars, conferences, and insights. 

Subscribe Here

Topics: methodology, qualitative research, research design, quantitative research

A Perfect Match? Tinder and Mobile Ethnographies

Posted by Anne Hooper

Wed, Apr 23, 2014

Tinder JoeI know what you are thinking...“What the heck is she TALKING about? How can Tinder possibly relate to mobile ethnography?”  You can call me crazy, but hear me out first.For those of you who may be unfamiliar, Tinder is a well-known “hook up” app that’s taken the smartphone wielding, hyper-social Millennial world by storm. With a simple swipe of the index finger, one can either approve or reject someone from a massive list of prospects. At the end of the day, it comes down to anonymously passing judgment on looks alone—yet if both users “like” each other, they are connected. Shallow? You bet. Effective? Clearly it must be because thousands of people are downloading the app daily.

So what’s the connection with mobile ethnography? While Tinder appears to be an effective tool for anonymously communicating attraction (anonymous in that the only thing you really know about the other person is what they look like), mobile ethnography is an effective tool for anonymously communicating daily experiences that we generally aren’t as privy to as researchers. Mobile ethnography gives us better insight into consumer behavior by bringing us places we’ve never gone before but are worthy of knowing nonetheless (Cialis, anyone?). Tapping into these experiences—from the benign to the very private—are the nuts and bolts behind any good product or brand.

So how might one tap into these experiences using mobile ethnography? It’s actually quite easy—we create and assign “activities” that are not only engaging for participants, but are also designed to dig deep and (hopefully) capture the "Aha!" moments we aim for as researchers. Imagine being able to see how consumers interact with your brand on a day-to-day basis—how they use your product, where their needs are being fulfilled, and where they experience frustrations. Imagine “being there” when your customer experiences your brand—offering insight into what delights and disappoints them right then and there (i.e., not several weeks later in a focus group facility). The possibilities for mobile ethnography are endless...let’s just hope the possibilities for Tinder come to a screeching halt sooner rather than later.

Anne Hooper is the Director of Qualitative Services at CMB. She has a 12 year old daughter who has no idea what Tinder is, and she hopes it stays that way for a very long time.

Topics: methodology, qualitative research, social media

What's the Story? 5 Insights from CASRO's Digital Research Conference

Posted by Jared Huizenga

Wed, Mar 19, 2014

CMB and CASROWho says market research isn’t exciting? I’ve been a market researcher for the past sixteen years, and I’ve seen the industry change dramatically since the days when telephone questionnaires were the norm. I still remember my excitement when disk-by-mail became popular! But I don’t think I’ve ever felt as excited about market research as I do right now. The CASRO Digital Research Conference was last week, and the presentations confirmed what I already knew—big changes are happening in the market research world. Here are five key takeaways from the conference:

  1. “Market research” is an antiquated term. It was even suggested that we change the name of our industry from market research to “insights.” In fact, the word “insights” came up multiple times throughout the conference by different presenters. This makes a lot of sense to me. Many people view market research as a process whereas insights are the end result we deliver to our clients. Speaking for CMB, partnering with our clients to provide critical insights is a much more accurate description of our mission and focus. We and our clients know percentages by themselves fail to tell the whole story, and can in fact lead to more confusion about which direction to take.

  2. “Big data” means different things to different people. If you ask ten people to define big data you’ll probably get ten different answers. Some define it as omnipresent data that follows us wherever we go. Others define it as vast amounts of unstructured data, some of which might be useful and some not. Still others call it an outdated buzzword.  No matter what your own definition of big data is, the market research industry seems to be in somewhat of a quandary about what to do with it. Clients want it and researchers want to oblige, but do adequate tools currently exist to deliver meaningful big data? Where does the big data come from, who owns it, and how do you integrate it with traditional forms of data? These are all questions that have not been fully answered by the market research (or insights) industry. Regardless, tons of investment dollars are currently being pumped into big data infrastructure and tools. Big data is going to be, well, BIG.  However, there’s a long way to go before most will be able to use it to its potential.

  3. Empathy is the hottest new research “tool.” Understanding others’ feelings, thoughts, and experiences allows us to understand the “why behind the what.”  Before you dismiss this as just a qualitative research thing, don’t be so sure.  While qualitative research is an effective tool for understanding the “why,” the lines are blurring between qualitative and quantitative research. Picking one over the other simply doesn’t seem wise in today’s world. Unlike with big data, tools do currently exist that allow us to empathize with people and tell a more complete story. When you look at a respondent, you shouldn’t only see a number, spreadsheet, or fancy graphic that shows cost is the most important factor when purchasing fabric softener. You should see the man who recently lost his wife to cancer and who is buying fabric softener solely based on cost because he has five years of medical bills. There is value in knowing the whole story. When you look at a person, you should see a person.

  4. Synthesizers are increasingly important. I’m not talking about the synthesizers from Soft Cell’s version of “Tainted Love” or Van Halen’s “Jump.” The goal here is to once again tell a complete story and, in order to do this, multiple skillsets are required. Analytics have traditionally been the backbone of market research and will continue to play a major role in the future. However, with more and more information coming from multiple sources, synthesizers are also needed to pull all of it together in a meaningful way. In many cases, those who are good at analytics are not as good at synthesizing information, and vice versa. This may require a shift in the way market research companies staff for success in the future. 

  5. Mobile devices are changing the way questionnaires are designed. A time will come when very few respondents are willing to take a questionnaire over twenty minutes long, and some are saying that day is coming within two years. The fact is, no matter how much mobile “optimization” you apply to your questionnaire, the time to take it on a smartphone is still going to be longer than on PCs and tablets. Forcing respondents to complete on a PC isn’t a good solution, especially since the already elusive sub 25 year old population spends more time on mobile devices than PCs. So what’s a researcher to do? The option of “chunking” long questionnaires into several modules is showing potential, but requires careful questionnaire design and a trusted sampling plan. This method isn’t a good fit for all studies where analysis dictates each respondent complete the entire questionnaire, and the number of overall respondents needed is likely to increase using this methodology. It also requires client buy-in. But it’s something that we at CMB believe is worth pursuing as we leverage mobile technologies.

Change is happening faster than ever. If you thought the transition from telephone to online research was fast—if you were even around back in the good old days when that happened—you’d better hold onto your seat! Information surrounds every consumer. The challenge for insights companies is not only to capture that information but to empathize, analyze, and synthesize it in order to tell a complete story. This requires multiple skillsets as well as the appropriate tools, and honestly the industry as a whole simply isn’t there yet. However, I strongly believe that those of us who are working feverishly to not just “deal” with change but to leverage it, and who are making progress with these rapidly changing technological advances, will be well equipped for success.

Jared is CMB’s Director of Field Services, and has been in market research industry for sixteen years. When he isn’t enjoying the exciting world of data collection, he can be found competing at barbecue contests as the pitmaster of the team Insane Swine BBQ

 

CMB Insight eZine


Sign-up here
for out monthly eZine for the latest Consumer Pulse
reports, case studies conference updates, webinars and more

Topics: qualitative research, big data, mobile, research design, quantitative research, conference recap

Jeffrey Henning:10 Tips for Mobile Diary Studies

Posted by Jeffrey Henning

Mon, Nov 25, 2013

Originally posted on Research Access

Earlier this month, Chris Neal of Chadwick Martin Bailey shared with members of the New England chapter of the Marketing Research Association tips for running mobile diary studies, based on lessons learned from a recent project.For the Council for Research Excellence (CRE), CMB studied mobile video usage to understand:

  • How much time is spent mobile diary researchon mobile devices watching TV (professionally produced TV shows)?

  • Does this cannibalize TV set viewing?

  • What motivates consumers to watch on mobile?

  • How can mobile TV viewing be accurately tracked?

The research included a quantitative phase with two online surveys and mobile journaling, followed by a series of home ethnographies. The quant work included a screening survey, the mobile diary, and a final online survey.

  • The screening survey was Census balanced to estimate market size, with three groups recruited for comparison: those without mobile devices (smartphones or tablets), those with mobile devices who don’t watch TV on them, and those with mobile devices that they watch TV on. The total number of respondents was 5,886.

  • The mobile diary activity asked respondents to complete their journal 4 times a day for 7 days.

  • A final attitudinal survey was used to better understand motivations and behaviors associated with decisions about TV watching.

Along the way, CMB learned some valuable best practices for mobile diary studies, including tips for recruiting, incentives, design and analysis. The 10 key lessons learned:

  1. Mobile panels don’t work for low incidence – Take care when using mobile panels – given the small size of many mobile panels, you may have better luck recruiting through traditional online panels, as CMB did. For this study, it was because of the comparatively low incidence of actual mobile TV watching.

  2. Overrecruit – You will lose many recruits to the journaling exercise when it comes time to downloading the mobile diary application. As a general rule, over-recruit by 100% – get twice the promises of participation that you need. Most dropout occurs after the screening and before the participant has recorded a single mobile diary entry. For many members of online survey panels, journaling is a new experience. The second biggest point of dropout was after recording 1 or 2 diary entries.

  3. Keep it short – To minimize this dropout, you have to keep the diary experience as short as possible: no more than 3 to 5 minutes long. The more times you ask participants to complete a diary each day, the greater the dropout rate.

  4. Think small screen – Make sure the survey is designed to provide a good experience on small screens – avoid grids and sum-allocation questions and limit open-ended prompts and use of images. Use vertical scales instead of horizontal scales. “Be wary of shiny new survey objects for smartphone survey-takers,” said Chris. Smartphone users had 5 times the dropout rate of tablet or laptop users in this study. Enable people to log on to their journal from whatever device they were using at the time, including their computer.

  5. Beware battery hogs – When evaluating smartphone apps, be wary of those that drain battery life by constantly logging GPS location. Check the app store reviews of the application.

  6. Keep consistent – Keep the diary questionnaire the same for every time block, to get respondents into the habit of answering it.

  7. Experiment with incentives to maximize participation – Tier incentives to motivate people to stick with the study and complete all time blocks. To earn the incentive for the CMB study, Chris said that respondents had to participate at least once a day for all 7 days, with additional incentives for every journal log entered (participants were reminded this didn’t have to involve actual TV watching, just filling out the log). In the end, 90% of journaling occasions were filled out.

  8. Remind via SMS and email – In-app notifications are not enough to prompt participation. Use email and text messages for each time block as well. Most respondents logged on within 2 hours of receiving a reminder.

  9. Use online surveys for detailed questions – Use the post-journaling survey to capture greater detail and to work around the limits of mobile surveys. You can then use these results to “slice and dice” the journal responses.

  10. Weight by occasions – Remember to weight the data file to total occasions not total respondents. For missing data, leave it missing. Develop a plan detailing which occasion-based data you’re going to analyze and what respondent-level analysis you are going to do. You may need to create a separate occasion-level data file and a separate respondent-level data file.

Properly done, mobile diary studies provide an amazing depth of data. For this project, CMB captured almost 400,000 viewing occasions (mobile and non-mobile TV watching), for over 5 million occasion-based records!

Interested in the actual survey results? CRE has published the results presentation, “TV Untethered: Following the Mobile Path of TV Content” [PDF].

Jeffrey Henning, PRC is president of Researchscape International, a market research firm providing custom surveys to small businesses. He is a Director at Large on the MRA Board of Directors; in 2012, he was the inaugural winner of the MRA’s Impact award. You can follow him on Twitter @jhenning.

Topics: methodology, qualitative research, mobile, research design

Deconstructing the Customer Experience: What's in Your Toolkit?

Posted by Jennifer von Briesen

Wed, Sep 25, 2013

Disassembled rubix 1More and more companies are focusing on trying to better understand and improve their customers’ experiences. Some want to become more customer-centric. Some see this as an effective path to competitive differentiation. While others, challenging traditional assumptions (e.g., Experience Co-creation, originated by my former boss, Francis Gouillart, and his colleagues Prof. Venkat Ramaswamy and the late C.K. Prahalad), are applying new strategic thinking about value creation. Decision-makers in these firms are starting to recognize that every single interaction and experience a customer has with the company (and its ecosystem partners) may either build or destroy customer value and loyalty over time.

While companies traditionally measure customer value based on revenues, share of wallet, cost to serve, retention, NPS, profitability, lifetime value etc., we now have more and better tools for deconstructing the customer experience and understanding the components driving customer and company interaction value at the activity/experience level. To really understand the value drivers in the customer experience, firms need to simultaneously look holistically, go deep in a few key focus areas, and use a multi-method approach.

Here’s an arsenal of tools and methods that are great to have in your toolkit for building customer experience insight:

Qualitative tools

  • Journey mapping methods and tools

  • In-the-moment, customer activity-based tools

    • Voice capture exercises (either using mobile phones or landlines) where customers can call in and answer a set of questions related to whatever they are doing in the moment.

    • Use mobile devices and online platforms to upload visuals, audio and/or video to answer questions, (e.g., as you are filling out your enrollment paperwork, take a moment to take a quick—less than 10 second video, to share your thoughts on what you are experiencing).

  • Customer diaries

    • E.g., use mobile devices as a visual diary or to complete a number of activities

  • Observation tools

    • Live or virtual tools (e.g., watch/videotape in-person or online experiences, either live or after the fact)

    • On-site customer visits: companies I’ve worked with often like to join customers doing activities in their own environments and situational contexts. Beyond basic observation, company employees can dialogue with customers during the activities/experiences to gain immediate feedback and richer understanding.

  • Interviews and qualitative surveys

  • Online discussion boards

  • Online or in-person focus groups

Quantitative tools

  • Quantitative surveys/research tools (too many to list in a blog post)

  • Internal tracking tools

    • Online tools for tracking behavior metrics (e.g., landing pages/clicks/page views/time on pages, etc.) for key interactions/experience stages. This enables ongoing data-mining, research and analysis.

    • Service/support data analysis (e.g., analyze call center data on inbound calls and online support queries for interaction types, stages, periods, etc. to look for FAQs, problems, etc.).

What tools are you using to better understand and improve the customer experience? What tools are in your toolkit?  Are you taking advantage of all the new tools available?

Jennifer is a Director at  South Street Strategy Group. She recently received the 2013 “Member of the Year” award by the Association for Strategic Planning (ASP), the preeminent professional association for those engaged in strategic thinking, planning and action.

Topics: South Street Strategy Group, strategy consulting, methodology, qualitative research, quantitative research, customer experience and loyalty