Voices: the CMB Blog

Who are the mobile consumers?

Let’s get this straight: I am not addicted to my smartphone. Unlike so many of my fellow train commuters who stare zombie-eyed into their small screens, I am not immersed in a personal relationship with pixels. I have an e-Reader for that. But, my smartphone IS my lifeline.I’ve come to depend exclusively on my phone to keep me on-time and on-schedule, to entertain me (when not using my e-Reader), to stay in touch with family and friends, and to keep up-to-date with my work email. It’s my primary source for directions, weather, news, photography, messaging, banking, and a regular source for payment, shopping, and ticketing/reservations. I haven’t purchased a PC in nearly a decade, and I don’t have a landline. I also use my smartphone to take market research questionnaires, and I am far from alone. 

Data around smartphone usage aligns with my personal experience. In a recent CMB online study of U.S. consumers, optimized for mobile devices, 1 in 6 Millennials completed the questionnaire on a smartphone. Other studies report similar results. This example illustrates the issue with representativeness. Major panel vendors are seeing over half of Millennials joining their panels via a mobile device. 

How do we adapt?

Much has been hypothesized about the future of market research under the new paradigm of mobile commerce, big data, and cloud services. New technologies and industry convergence (not just mobile) have brought sweeping changes in consumer behaviors, and market researchers must adapt.

A key component of successful adaptation will be greater integration of primary market research with other data streams. The promise of passive or observational data is captivating, but it is largely still in the formative stages. (For more on passive data, check out our recent webinar.) We still need and will likely always need active “please tell me” research. The shift from phone to online data collection has quickly been replaced with the urgency of a shift to mobile data collection (or at least device agnostic interviewing). Our industry has lagged behind because the consumer experience has become so personalized and the trust/value equation for tapping into their experiences is challenging. Tackling mobile market research with tactical solutions is a necessary step in this transition.

What should we do about it?  

1. Understand your current audience. Researchers need to determine how important mobile data collection is to the business decision and decide how to treat mobile respondents. You can have all respondents use a mobile device, have some use a mobile device, or have mobile device respondents excluded. There are criteria and considerations for each of these, and there are also considerations for the expected mix of feature phones, smartphones, tablets, and PCs. The audience will determine the source of sample and representation that must be factored into the study design. Ultimately, this has a huge impact on the validity and reliability of the data. Respondent invitations need to include any limitations for devices not suitable for a particular survey.
2. Design for mobile. If mobile participation is important, researchers should use a mobile first questionnaire design. Mobile optimized or mobile friendly surveys typically need to be shorter in length, use concise language, avoid complex grids and answering mechanisms, and have fewer answer options, so they can be supported on a small screen and keep respondents focused on the activity. In some cases,questionnaire modularization or data stitching can be used to help adhere to mobile design standards.
3. Test for mobile. All questions, images, etc. need to display on a variety of screen sizes and within the bandwidth capacity of the devices that are being used. Android and iOS device accommodation covers most users. If app based surveys are being used, researchers need to ensure that the latest versions can be downloaded and are bug-free. 
4. Apply data protection and privacy standards. Mobile market research comes with a unique set of conditions and challenges that impact how information is collected, protected, and secured. Research quality and ethical guidelines specific to mobile market research have been published by CASRO, ESOMAR, the MMRA (Mobile Marketing Research Association), and others.
5. Implement Mobile Qualitative. The barriers are lower, and researchers can leverage the unique capabilities of mobile devices quite effectively with qualitative research. Most importantly, willing participants are mobile, which makes in-the-moment research possible. Mobile qualitative is also a great gateway to explore what’s possible for mobile quantitative studies. See my colleague Anne Hooper’s blog for more on the future of qualitative methodologies.
6. Promote Research-on-Research. Experts need to conduct and publish additional research-on-research studies that advance understanding of how to treat mobile respondents and utilize passive data, location tracking, and other capabilities that mobile devices provide. We also need stronger evidence of what works and what doesn’t work in execution of multi-mode and mobile-only studies across different demographics, in B2B studies, and within different countries.

But perhaps the most important thing to remember is that this is just a start. Market researchers and other insight professionals must evolve from data providers to become integrated strategic partners—harnessing technology (not just mobile) to industry expertise to focus on decision-making, risk reduction, and growth.

Brian is a Senior Project Manager for Chadwick Martin Bailey, the photographer of the image in this post, and an 82 percenter—he is one of the 82% of mobile phone owners whose phone is with them always or most of the time. 

Watch our recent webinar that discusses the results of our self-funded Consumer Pulse study on the future of the mobile wallet. 

CMB’s portfolio is pretty expansive: we do research in a variety of industries and work with many different types of consumers—from insurance consumers to techies to medical experts to small business owners. We know how to get answers, and, more importantly, we are experts in getting to know our audiences. We learn more every day from our work, and we’re always willing to go above and beyond for our clients.I recently worked on a project with a kids’ media company that struck me as new, exciting, and something that I thought I would never come across unless I worked at a boutique kids-only research firm—I couldn’t wait to dive in! Our client wanted to understand kids today and segment the market beyond age and gender. The design, methodology, and project planning came naturally. However, when we reviewed the wording of the questionnaire, we knew we would have to step into our “kid” shoes and adjust accordingly.

Our approach to capture the appropriate wording and diction was twofold. First, we did a series of in-depth interviews, which uncovered ways to speak to various types of kids—leaving no child behind. (For example, we learned that there’s a group of kids out there who wish that they had a superpower of being able to shoot flames out of their backsides—please note this was selected over being invisible, being able to fly, and other “more appropriate” powers). Upon completing the IDIs, we felt that we could make lists exhaustive and kid-friendly, so we moved forward and programmed the survey.

Our next step was to do a round of pre-testing the program we planned to field, so we programmed the survey and observed while kids and parents went through the survey together. This allowed us to see which questions worked, which ones didn’t, which ones we had to further “gamify,” which ones had responses that were too similar, and which ones were confusing to kids. A max difference exercise is just one example of how the pre-testing helped us. We ran a force choice task asking kids to select between 2 descriptors, such as smart v. pretty, popular v. famous, and pretty v. popular. As we watched our pre-testers go through this process, we overhead a common sentiment: “why can’t I be both?”—which was an indicator that the force choice exercises were not a great method to use when doing research with kids. Thus, this pre-testing further refined our design into a clear, fun, and strong program that both kids and parents had fun doing together.

You may be asking yourself: why go the extra mile? Well, because we knew it would pay off, and it did! These extra steps allowed us to create an extensive segmentation that moved far beyond age and gender (the average age across the 7 segments varied between 8.2 years old to 8.6 years old) and had an 85% classification rate on the typing tool. This project was unique and adorable, but it was also a wonderful learning opportunity. It emphasized the importance of getting to know your audience and also proved that upfront research with experts can go a long way when you go into the field.

Hannah is a Project Manager on the TEM team and considers herself closer to being a kid than to having one. Her superpower of choice is omnilingualism.

Watch our recent webinar with Research Now to hear the results of our recent self-funded Consumer Pulse study that leveraged passive mobile behavioral data and survey data simultaneously to reveal insights into the current Mobile Wallet industry in the US.

There is a strong belief that gender identity can be used to predict behavior in the marketplace, and we see evidence of this belief in advertising every day (and we also regularly poke fun at this idea - see the video below). Despite this, the standard approach to collecting information about gender and behavior often lacks the depth and complexity necessary to reach the meaningful insights around gender identity. How can we fix this? One way forward is to incorporate social science into our questionnaire design. 

There’s a large body of evidence from social science research that indicates social identities, like gender, can have concrete economic implications for people belonging to certain groups. Gender is not only an expression of individual identity, but is also negotiated on a group level as we practice and enforce patterns of hierarchical social, political, and economic relationships (including work and family life). So, while one woman’s social, political, and/or economic profiles may deviate from the profiles of women as a group, she’s still subject to the systematic opportunities and barriers that these group profiles represent.

At CMB, we often leverage social science in questionnaire design to elicit responses that most closely reflect the market. As an industry, we could (and should) go further in the way we collect demographic information. For example, respondents are typically allowed to select only one employment status from a list of several options: employed part time, employed full time, full time homemaker, full time student, retired, or unemployed. From the social science perspective, this question is problematic because it ignores the fact that respondents may fall into more than one category and that women are more likely than men to experience overlap in these categories in their lifetime. A question like this might produce compromised data, particularly for respondents who are young, female, and/or low-income. Another example is marital status: is the marketplace behavior of a same-sex unmarried couple categorically different than that of a couple in a traditional marriage? Depending on the industry, the answer may vary, but with a few easy questionnaire tweaks, we can capture that information.

From segmentation to optimization, demographic information is often a critical part of the analyses that solve our clients’ business challenges. But our answers to their problems are only as good as the questions our surveys are asking. Revisiting demographic collection is an easy update that goes a long way towards generating higher quality data, making better evidence-based recommendations, and pushing businesses forward.  

Eliza Novick is an Associate Researcher at CMB. Her favorite Boston attraction is the New England Aquarium, particularly the Edge of the Sea exhibit where you can pick up clams and starfish. 

Want to know the latest on barriers and opportunities for the next generation of mobile payment providers?

After nearly a decade working on highly complex and strategic research projects, I’ve learned the one thing you can count on when dealing with massive amounts of data is Murphy’s Law—anything that can go wrong, will go wrong. No matter how much planning we do (and we take planning very seriously), the nature of market research means there’s bound to be a hiccup or two along the way.One of the best ways to deal with Murphy's Law is to accept that issues will arise but to make sure they don’t get in the way of the end goal—actionable insights. At CMB, our ability to seamlessly execute projects hinges on our capacity to adjust and course correct (when needed) to keep things on track. We put a lot of preparation and time in putting together solid project plans, focusing on business decisions, and conducting stakeholder interviews, but we also place a lot of emphasis on hiring and training strong problem solvers. We do this because we know that even the best laid out plans can still go awry, which is why it's important to manage problems proactively. For example, CMB firmly believes in conducting stakeholder interviews at the beginning of nearly all research engagements. This allows us to proactively re-shape/re-think the questionnaire design based on the information we’re hearing from the stakeholders. This helps prevent getting to the final presentation and delivering insights that are not relevant or useable for the key stakeholders.

Even the Patriots, as successful as they were this season and in the Super Bowl, run into problems and issues in each game that they play, regardless if they are playing the worst or best team in the league. If you read Peter King's Monday Morning Quarterback column the day after the Super Bowl, you'll remember that he highlighted Bill Belichick's pre-Super Bowl game meeting with his staff. Josh McDaniels, the Patriots’ offensive coordinator, summarized the meeting and said that Bill's main message was this: "This game is no different than any other one. It’s a 60-minute football game, and whatever issues we have, let’s make sure we correct them, coach them, and fix them. That’s our job." During that meeting, McDaniels, wrote two notes on his game play clipboard, "adjust" and "correct problems and get them fixed." Going into the game with those mantras was a reminder for him that the game is dynamic, and even the best laid plans need to be adjusted throughout the course of play.

While we can’t rely on Tom Brady, our approach to research engagements is no different. We encounter complex challenges day in and day out, and as our clients' needs change, we continue to think creatively and provide new and better solutions. When working with CMB, you can feel confident that we're putting together a solid project approach while simultaneously planning for the problems that may lie ahead. We might not be the Patriots, but we’re champions at execution just the same.

Cara is a Senior Research Manager. She enjoys spending time with her husband and dog, and she is STILL reveling in the "high" from the Patriots Super Bowl win.

Are YOU a strong problem solver? Come join our team!

I once again had the pleasure of attending the CASRO Digital Research Conference this year. It’s the one of the best conferences available to data collection geeks like me, and this year’s presentations did not disappoint. Here are a few key takeaways from this year’s conference.

1. The South shuts down when it snows. After having a great weekend in Nashville after the conference, my flight was cancelled on Monday due to about an inch of snow and a little ice. Needless to say, I was happy to return to Boston and its nine feet of snow.

2. “Big data” is an antiquated term. Over the past few years, big data has been the big buzz in the industry. Much like we said goodbye to traditional “market research,” we can now say adios to “big data.” Good riddance. The term was vague at best. However, that doesn’t mean that the concept is going away. It’s simply being replaced by new, more meaningful terminology like “integrated data” and “multi-sourced data.” But one thing isn’t changing. . .

3. Researchers still don’t know what to do with all that data. What can I say about multi-sourced data that I haven’t already said many times over the past couple years? Clients still want it, and researchers still want to oblige. But this fact remains: adequate tools still do not exist to deliver meaningful integrated data in most cases. We have a long way to go before most researchers will be able to leverage all of this data to its full potential in a meaningful way for our clients.

4. There’s a lot more to mobile research than how a questionnaire looks on a screen. For the past three or four years, it seems like every year is going to be “the year of mobile” at these types of conferences. Because of this, I always attend the mobile-related sessions skeptically. When we talk about mobile, more often than not, the main concern is how the questionnaire will look on a mobile device. But mobile research is much more than that. One of the best things I heard at the conference this year was that researchers should “follow the humans.” This is true on so many levels. Of course, a person can respond to a questionnaire invitation on his/her mobile device, but so much of a person’s daily life, including behaviors and attitudes, is shaped by mobile. Welcome to the world of the ultra-informed consumer. I can confidently say that 2015 is most definitely the year of mobile! (I do, however, reserve the right to say the same thing again next year.)

5. Researchers need to think like humans. It’s easy to get caught up in percentages in our world, and researchers sometimes lose sight of the human aspect of our industry. We like to think that millionaire CEOs are constantly checking their emails on their desktop computers, waiting for their next “opportunity” to take a 45-minute online questionnaire for a twenty-five cent reward. I attended sessions at the conference about gamification, how to make questionnaires more user-friendly, and also how to make questionnaires more kid-friendly by adding voice-to-text and text-to-voice options. All of these things have the potential to ease the burden on research participants, and as an industry, this must happen. We have a long way to go, but. . .

6. Now is the time to play catch-up with the rest of the world. Last year, I ended my recap by saying that change is happening faster than ever. I still think that’s true about the world we live in. With all of the technological advances and new opportunities provided to us, it’s an exciting time to be alive. However, I’m not sure I can honestly say that change is happening faster than ever when it comes to the world of research. I’ve been a part of this industry for a very fulfilling seventeen years, and sometimes my pride in the industry clouds my thinking. Let’s face the facts. The truth is that, as an industry, we are lagging far behind as the world speeds by. Research techniques and tools are evolving at a very slow pace, and I don’t see this changing in the near future. (In our defense, this is true for many industries and not only market research.) I still believe that those of us who are working to leverage the changing world we live in will be much better equipped for success than those who sit idly and watch the world fly.

I’m still confident that my industry is primed and ready for significant and meaningful change—even if we sometimes take the path of a tortoise. As a weekend pitmaster, I know that low and slow is sometimes the best approach. The end result is what really counts.

Jared is CMB’s Field Services Director, and has been in market research industry for seventeen years. When he isn’t enjoying the exciting world of data collection, he can be found competing at barbecue contests as the pitmaster of the cooking team Insane Swine BBQ.