WELCOME TO OUR BLOG!

The posts here represent the opinions of CMB employees and guests—not necessarily the company as a whole. 

Subscribe to Email Updates

BROWSE BY TAG

see all

Swipe Right for Insights

Posted by Jared Huizenga

Wed, Aug 17, 2016

Data collection geeks like me can learn a ton at the CASRO Digital Research Conference. While the name of the event has changed many times over the years, the quality of the presentations and the opportunity to learn from experts in the industry are consistently good.

One topic that came up many years ago was conducting surveys via cellphones with SMS texts. This was at a time when most people had cellphones, but it was still a couple of years before the smartphone explosion. I remember listening to one presentation and looking down at my Samsung flip-phone thinking, “There’s no way respondents will take a CMB questionnaire this way.” For a few simple yes/no questions, this seemed like a fine methodology but it certainly wouldn’t fly for any of CMB’s studies.

For the next two or three years, less than half of the U.S. population owned smartphones (including yours truly). Even so, SMS texting was getting increasing coverage at the CASRO conference, and I was having a really hard time understanding why. Every year was billed as “the year of mobile!” I could see the potential of taking a survey while mobile, but the technology and user experience weren’t there yet. Then something happened that changed not only the market research industry but the way in which we live as human beings—smartphone adoption skyrocketed.
Girl_and_phone.jpg

Today in the U.S., smartphone ownership among adults is 72% according to the Pew Research Center. People are spending more time on their phones and less time sitting in front of a computer. Depending on the study and the population, anywhere from 20%-40% of survey takers are using their smartphones. And if it’s a study with people under 25 years old, that number would likely be even higher. We can approach mobile respondents in three ways:

  • Do nothing. This means surveys will be extremely cumbersome to take on smartphones, to the point where many will abandon during the painful process. This really isn’t an option at all. By doing nothing, you’re turning your back on the respondent experience and basically giving mobile users the middle finger.
  • Optimize questionnaires for mobile. All of CMB’s questionnaires are optimized for mobile. That is, our programming platforms identify the device type a respondent is using and renders the questionnaire to the appropriate screen size.  Even with this capability, long vertical grids and wide horizontal scales will still be painful for smartphone users since they will require some degree of scrolling. This option is better than nothing, but long questions are still going to be long questions.
  • Design questionnaires for mobile. This is the best option, and one that isn’t used often enough. This requires questions and answer options to be written with the idea that they will be viewed on smartphones. In other words, no lengthy grids, no sprawling scales, no drag and drop, minimal scrolling, or anything else that would cause the mobile user angst.  While this option sounds great, one of the criticisms has been that it’s difficult to do advanced exercises like max-diff or discrete choice on smartphones.

One cautionary note if you are thinking that a good option would be to simply disallow respondents from taking a survey on their smartphones.  Did your parents ever tell you not to do something when you were a child?  Did you listen to them or did you try it anyway? What’s going to happen when you tell someone not to take a survey on their mobile device?  Either by mistake or out of sheer defiance, some people will attempt to take it on their smartphone. This happened on a recent study for one of our clients.  These people tried to “stick it to the man,” but alas they were denied entry into the survey. If you want “representative” sample, the other argument against blocking mobile users is that you are blocking specific populations which could skew the results.

The respondent pool is getting shallow, and market research companies are facing increased challenges when it comes to getting enough “completes” for their studies.  It’s important for all of us to remember that behind every “complete” is a human being—one who’s trying to drag and drop a little image into the right bucket or one who’s scrolling and squinting to make sure they are choosing the right option on an 11-point scale in a twenty row grid.  Unless everyone is comfortable basing their quantitative findings off of N=50 in the future, we all need to take steps to embrace the mobile respondent. 

Jared is CMB’s Field Services Director, and has been in market research industry for eighteen years. When he isn’t enjoying the exciting world of data collection, he can be found competing at barbecue contests as the pitmaster of the team Insane Swine BBQ.

Sign up to receive our monthly eZine and never miss a webinar, conference recap, or insights from our self-funded research on hot topics from data integration to Social Currency.

Subscribe Here!

Topics: mobile, research design, Market research

Passive Mobile Behavioral Data – Part Deux

Posted by Chris Neal

Wed, Aug 10, 2016

Over the past two years, we've  embarked on a quest to help the insights industry get better at harnessing passive mobile behavioral data. In 2015, we partnered with Research Now for an analysis37824990_thumbnail.jpg of mobile wallet usage, using unlinked passive and survey-based data. This year, we teamed up with Research Now once again for research-on-research directly linking actual mobile traffic and app data to consumers’ self-reported online shopper journey behavior.

We asked over 1,000 shoppers, across a variety of Black Friday/Cyber Monday categories, a standard set of purchase journey survey questions immediately after the event, then again after 30 days, 60 days, and 90 days. We then compared their self-reported online and mobile behavior to the actual mobile app and website usage data from their smartphones. 

The results deepened our understanding of how best to use (and not use) each respective data source, and how combining both can help our clients get closer to the truth than they could using any single source of information.

Here are a few things to consider if you find yourself tasked with a purchase journey project that uses one or both of these data sources as fuel for insights and recommendations:

  1. Most people use multiple devices for a major purchase journey, and here’s why you should care:
    • Any device tracking platform (even one claiming a 3600 view) is likely missing some relevant online behavior to a given shopper journey. In our study, we were getting behavior from their primary smartphone, but many of these consumers reported visiting websites we had no record of from our tracking data. Although they reported visiting these websites on their smartphones, it is likely that some of these visits happened on their personal computer, a tablet, a computer at their work, etc.
  2. Not all mobile usage is related to the purchase journey you care about:
    • We saw cases of consumers whose behavioral data showed they’d visited big retail websites and mobile apps during the purchase journey but who did not report using these sites/apps as part of the journey we asked them about. This is a bigger problem with larger, more generalist mobile websites and apps (like Amazon, for this particular project, or like PayPal when we did the earlier Mobile Wallet study with a similar methodological exercise).
  3. Human recall ain’t perfect. We all know this, but it’s important to understand when and where it’s less perfect, and where it’s actually sufficient for our purposes. Using survey sampling to analyze behaviors can be enormously valuable in a lot of different situations, but understand the limitations and when you are expecting too much detail from somebody to give you accurate data to work with.  Here are a few situations to consider:
    • Asking whether a given retailer, brand, or major web property figured into the purchase journey at all will give you pretty good survey data to work with. Smaller retailers, websites, and apps will get more misses/lack of recall, but accurate recall is a proxy for influence, and if you’re ultimately trying to figure out how best to influence a consumer’s purchase journey, self-reported recall of visits is a good proxy, whereas relying on behavioral data alone may inflate the apparent impact of smaller properties on the final purchase journey.
    • Asking people to remember whether they used the mobile app vs. the mobile website introduces more error in your data. Most websites are now mobile optimized and look/ feel like mobile apps, or will switch users to the native mobile app on their phone automatically if possible.
      • In this particular project, we saw evidence of a 35-50% improvement in survey-behavior match rates if we did not require respondents to differentiate the mobile website from the mobile app for the same retailer.
  4. Does time-lapse matter? It depends.
    • For certain activities (e.g., making minor purchases in grocery store, a TV viewing occasion), capturing in-the-moment feedback from consumers is critical for accuracy.
    • In other situations where the process is bigger, involves more research, or is more memorable in general (e.g., buying a car, having a wedding, or making a planned-for purchase based on a Black Friday or Cyber Monday deal): you can get away with asking people about it further out from the actual event.
      • In this particular project, we actually found no systematic evidence of recall deterioration when we ran the survey immediately after Black Friday/Cyber Monday vs. running it 30 days, 60 days, and 90 days after.

Working with passive mobile behavioral data (or any digital passive data) is challenging, no doubt.  Trying to make hay by combining these data with primary research survey sampling, customer databases, transactional data, etc., can be even more challenging.  But, like it or not, that’s where Insights is headed. We’ll continue to push the envelope in terms of best practices for navigating these types of engagements as Analytics teams, Insights departments, Financial Planning and Strategy groups work together more seamlessly to provide senior executives with a “single version of the truth”— one which is more accurate than any previously siloed version.

Chris Neal leads CMB’s Tech Practice. He knows full well that data scientists and programmatic ad buying bots are analyzing his every click on every computing device and is perfectly OK with that as long as they serve up relevant ads. Nothing to hide!

Don't miss out on the latest research, insights and conference recaps. Subscribe to our monthly eZine.

Subscribe Here!

Topics: advanced analytics, mobile, passive data, integrated data

Can Facial Recognition Revolutionize Qualitative?

Posted by Will Buxton

Wed, Aug 03, 2016

Full disclosure: I’m an Android and Google loyalist, but please don’t hold that against me or the rest of my fellow Android users, who, by the way, comprise 58% of the smartphone market share in the United States. As a result of my loyalty, I’m always intrigued by Google’s new hardware and software advancements, which are always positioned in a way that leads me to believe they will make my life easier. Some of the innovations over the years have in fact lived up to the hype, such as Google Now, Google Drive, and even Google Fusion, while others such as Google Buzz and Google Wave have not.

As a researcher, last year’s launch of Google Photos caught my eye. Essentially, Google
Photos now utilizes facial recognition software to group or bunch your photos based on people in them, scenery (i.e., beaches and Google_Photos_icon.svg-1.pngmountains) and even events (i.e., weddings and holidays). To activate the facial recognition feature, all you have to do is tag one photo with an individual’s name and all other photos with that person will be compiled into a searchable collection. Google uses visual cues within the photos and geotagging to create other searchable collections. While these features might not seem extraordinary—I can see who was the most frequent star of my photos (my enormous cat) or where I most commonly take photos (honeymoon sans enormous cat)—I began to imagine the possible impact these features could have on the market research industry.

Visual ethnographies are one of many qualitative research options we offer at CMB. This is a rich form of observation, and, for some companies, it can be cost prohibitive in nature, especially ones focused on a “cost-per-complete.” But, what if there was a way to remove some of the heavy lifting of a customer journey ethnography by quantifying some of the shopping experience using technology that could track date/time, location, shopping layout, products viewed, order in which products are viewed, and so on, all through recognition software? Would the reduction in hours, travel, and analysis be able to offset the technological costs of these improvements?

Market research, and, in particular, qualitative research have always been a combination of art and science, and to expect any technological advancement to adequately perform any cogent analyses is a bit premature and perhaps too reminiscent of The Minority Report. (I don’t think it worked out well). But the promise of these powerful tools makes it an exciting time to  be a qualitative researcher!

Will Buxton is a Project Manager on the Financial Services team. He enjoys finding humor in everyday tasks, being taken seriously, and his enormous cat.

Learn more about how our dedicated Qualitative practice helps brands Explore, Listen, & Engage.

 

 

 

Topics: methodology, qualitative research, mobile, storytelling, customer journey

What We’ve Got Here Is a Respondent Experience Problem

Posted by Jared Huizenga

Thu, Apr 14, 2016

respondent experience problemA couple weeks ago, I was traveling to Austin for CASRO’s Digital Research Conference, and I had an interesting conversation while boarding the plane. [Insert Road Trip joke here.]

Stranger: First time traveling to Austin?

Me: Yeah, I’m going to a market research conference.

Stranger: [blank stare]

Me: It’s a really good conference. I go every year.

Stranger: So, what does your company do?

Me: We gather information from people—usually by having them take an online survey, and—

Stranger: I took one of those. Never again.

Me: Yeah? It was that bad?

Stranger: It was [expletive] horrible. They said it would take ten minutes, and I quit after spending twice that long on it. I got nothing for my time. They basically lied to me.

Me: I’m sorry you had that experience. Not all surveys are like that, but I totally understand why you wouldn’t want to take another one.

Thank goodness the plane started boarding before he could say anything else. Double thank goodness that I wasn’t sitting next to him during the flight.

I’ve been a proud member of the market research industry since 1998. I feel like it’s often the Rodney Dangerfield of professional services, but I’ve always preached about how important the industry is. Unfortunately, I’m finding it harder and harder to convince the general population. The experience my fellow traveler had with his survey points to a major theme of this year’s CASRO Digital Research Conference. Either directly or indirectly, many of the presentations this year were about the respondent experience. It’s become increasingly clear to me that the market research industry has no choice other than to address the respondent experience “problem.”

There were also two related sub-themes—generational differences and living in a digital world—that go hand-in-hand with the respondent experience theme. Fewer people are taking questionnaires on their desktop computers. Recent data suggests that, depending on the specific study, 20-30% of respondents are taking questionnaires on their smartphones. Not surprisingly, this skews towards younger respondents. Also not surprisingly, the percentage of smartphone survey takers is increasing at a rapid pace. Within the next two years, I predict the percent of smartphone respondents will be 35-40%. As researchers, we have to consider the mobile respondent when designing questionnaires.

From a practical standpoint, what does all this mean for researchers like me who are focused on data collection?

  1. I made a bold—and somewhat unpopular—prediction a few years ago that the method of using a single “panel” for market research sample is dying a slow death and that these panels would eventually become obsolete. We may not be quite at that point yet, but we’re getting closer. In my experience, being able to use a single sample source today is very rare except for the simplest of populations.

Action: Understand your sample source options. Have candid conversations with your data collection partners and only work with ones that are 100% transparent. Learn how to smell BS from a mile away, and stay away from those people.

  1. As researchers, part of our job should be to understand how the world around us is changing. So, why do we turn a blind eye to the poor experiences our respondents are having? According to CASRO’s Code of Standards and Ethics, “research participants are the lifeblood of the research industry.” The people taking our questionnaires aren’t just “completes.” They’re people. They have jobs, spouses, children, and a million other things going on in their lives at any given time, so they often don’t have time for your 30-minute questionnaire with ten scrolling grid questions.

Action: Take the questionnaires yourself so you can fully understand what you’re asking your respondents to do. Then take that same questionnaire on a smartphone. It might be an eye opener.

  1. It’s important to educate colleagues, peers, and clients regarding the pitfalls of poor data collection methods. Not only does a poorly designed 30-minute survey frustrate respondents, it also leads to speeding, straight lining, and just not caring. Most importantly, it leads to bad data. It’s not the respondent’s fault—it’s ours. One company stood up at the conference and stated that it won’t take a client project if the survey is too long. But for every company that does this, there are many others that will take that project.

Action: Educate your clients about the potential consequences of poorly designed, lengthy questionnaires. Market research industry leaders as a whole need to do this for it have a large impact.

Change is a good thing, and there’s no need to panic. Most of you are probably aware of the issues I’ve outlined above. There are no big shocks here. But, being cognizant of a problem and acting to fix the problem are two entirely different things. I challenge everyone in the market research industry to take some action. In fact, you don’t have much of a choice.

Jared is CMB’s Field Services Director, and has been in market research industry for eighteen years. When he isn’t enjoying the exciting world of data collection, he can be found competing at barbecue contests as the pitmaster of the team Insane Swine BBQ.

Topics: data collection, mobile, research design, conference recap

3 “Magical” Steps to Curbing Information Overload

Posted by Jen Golden

Wed, Feb 24, 2016

iStock_000024159442_Illustration.jpgRecently the WNYC podcast “Note to Self” (@NoteToSelf) released a week-long challenge to its listeners aimed at curbing information overload in our daily lives. In today’s internet-driven society, we’re hit from all angles with information, and it can be difficult to decide what information or content to consume in a day without being totally overwhelmed. I decided to participate in this challenge, and as the week progressed, I realized that many of the lessons from this exercise could be applied to our clients—who often struggle with information overload in their businesses.

The “InfoMagical” challenge worked like this: 

Challenge 1: “A Magical Day” – No multi-tasking, only single-tasking.

  • This challenge centered on focusing on one task at a time throughout the day. I knew this was going to be a struggle right from the start since my morning commute on the train typically involves listening to a podcast, scanning the news, checking social media, and catching up on emails at the same time. For this challenge, I stuck to one podcast (on the Architecture of Dumplings). By the end of the day, I felt more knowledgeable about the topics I focused on (ask me anything about jiaozi), as opposed to taking in little bits of information from various sources. 
  • Research Implications: Our clients often come to us with a laundry list of research objectives they want to capture in a single study. To maintain the quality of the data, we need to make trade-offs regarding what we can (or can’t) include in our design. We focus on designing projects around business decisions, asking our clients to prioritize the information they need in order to make the decisions they are facing. Some pieces may be “nice to have,” but they ultimately may not help answer a business decision. By following this focused approach, we can provide actionable insights on the topics that matter most.

 Challenge 2: “A Magical Phone” – Tidy up your smartphone apps.

  • This challenge asked me to clean up and organize my smartphone apps to keep only the ones that were truly useful to me. While I wasn’t quite ready to make a full commitment and delete Instagram or Facebook (how could I live without them?), I did bury them in a folder so I would be less likely to absentmindedly click through them every time I picked up my phone. Organizing and keeping only the apps you really need makes the device more task-oriented and less likely to be a distraction. 
  • Research Implications: When we design a questionnaire, answer option lists can often become long and unwieldy. With more and more respondents taking surveys on smartphones, it is important to make answer option lists manageable for respondents to answer. Often, a list can be cleaned up to include only the answer options that will produce useful results. Here are two ways to do this: (1) look at results from past studies with similar answer options lists to determine what was useful vs. not (i.e., what options had very high responses vs. very low) or (2) if the project is a tracker, run a factor analysis on the list to see if it can be paired down into a smaller sub-set of options for the next wave. This results in more meaningful (and higher quality) results going forward.  

Challenge 3: "A Magical Brain" – Avoid a meme, trending topic, or “must-read” today.

  • I did this challenge the day of the Iowa Caucuses, and it was hard to avoid all the associated coverage. But, when I looked at the results the next day, I realized I was happy enough just knowing the final results. I didn’t need to follow the minute-by-minute details of the night, including every Donald Trump remark and every Twitter comment. In this case, endless information did not make me feel better informed. 
  • Research Implications: Our clients often say they want to see the results of a study shown every which way, reporting out on every question by every possible sub-segment. There is likely some “FOMO” (fear of missing out) going on here, as clients might worry we are missing a key storyline by not showing everything. We often take the approach of not showing every single data point; instead, we only highlight differences in the data where it adds to the story in a significant and meaningful way. There comes a point when too much data overwhelms decisions. 

The other two pieces of this challenge focused on verbally communicating the information I learned on a single topic and setting a personal information mantra to say every time I consumed information (mine was “take time to digest after you consume it”). By the end of the challenge, even though I didn’t consume as much information as I typically do in a week, I didn’t feel like I was missing out on anything (except maybe some essential Bachelor episode recaps), and I felt more knowledgeable about the information I did consume. 

Jen Golden is a Project Manager on the Tech/E-commerce practice at CMB. She wishes there was more hours in the day to listen to podcasts without having to multi-task.  

For the latest Consumer Pulse reports, case studies, and conference news, subscribe to our monthly eZine.

Subscribe Here!

Topics: mobile, business decisions, research design

Black Friday Is Dead…Long Live Black Friday

Posted by Megan McManaman

Tue, Dec 22, 2015

retail2.pngIf you noticed the annual coverage of Black Friday shoppers seemed somewhat muted this year, you weren’t imagining things. While Cyber Monday sales were the highest since its debut in 2005, Black Friday sales were at their lowest since 2011. We all know how many elves flew (or didn’t) off the shelves, but to learn more about consumer holiday shopping behaviors, we partnered with Research Now for a quick survey of smartphone owners, ages 18 and up. 

Does 2015 mark the end of Black Friday—retail’s highest and holiest holiday? One retailer, REI, even opted out of this year’s Black Friday altogether, though their website did allow shoppers to make purchases online. The 87% of respondents who reported shopping on Black Friday might suggest that its imminent death is exaggerated. But the 81% of those Black Friday shoppers who did at least some of their shopping online suggest the explosion of ecommerce may have circumscribed the usual Black Friday frenzy.   

And then we have mobile—2015 marked the introduction of app-only deals from retail giants Amazon, Walmart, and Target. Of respondents who did shop from their smartphone or tablet, on either Black Friday or Cyber Monday, a full 27% purchased through an app. Still, a Cyber Monday dominated by in-app sales may be a few years away—61% of the Black Friday and Cyber Monday online shoppers used a PC to make their purchases. 

Need further evidence that online shopping and mobile technology are disrupting the traditional holiday shopper customer journey? “Just” 67% of Black Friday deal-seekers said they actually braved a brick and mortar store—this on a day once defined by the in-store experience. Is nothing sacred? 

Megan is CMB’s Senior Product Marketing Manager. She can’t stand Christmas music and was once visited by 3 ghosts. 

Topics: technology research, mobile, retail research, customer journey

Say Goodbye to Your Mother’s Market Research

Posted by Matt Skobe

Wed, Dec 02, 2015

evolving market researchIs it time for the “traditional” market researcher to join the ranks of the milkman and switchboard operator? The pressure to provide more actionable insights, more quickly, has never been so high. Add new competitors into the mix, and you have an industry feeling the pinch. At the same time, primary data collection has become substantially more difficult:

  • Response rates are decreasing as people become more and more inundated with email requests
  • Many among the younger crowd don’t check their email frequently, favoring social media and texting
  • Spam filters have become more effective, so potential respondents may not receive email invitations
  • The cell-phone-only population is becoming the norm—calls are easily avoided using voicemail, caller ID, call-blocking, and privacy managers
  • Traditional questionnaire methodologies don’t translate well to the mobile platform—it’s time to ditch large batteries of questions

It’s just harder to contact people and collect their opinions. The good news? There’s no shortage of researchable data. Quite the contrary, there’s more than ever. It’s just that market researchers are no longer the exclusive collectors—there’s a wealth of data collected internally by companies as well as an increase in new secondary passive data generated by mobile use and social media. We’ll also soon be awash in the Internet of Things, which means that everything with an on/off switch will increasingly be connected to one another (e.g., a wearable device can unlock your door and turn on the lights as you enter). The possibilities are endless, and all this activity will generate enormous amounts of behavioral data.

Yet, as tantalizing as these new forms of data are, they’re not without their own challenges. One such challenge? Barriers to access. Businesses may share data they collect with researchers, and social media is generally public domain, but what about data generated by mobile use and the Internet of Things? How can researchers get their hands on this aggregated information? And once acquired, how do you align dissimilar data for analysis? You can read about some of our cutting-edge research on mobile passive behavioral data here.

We also face challenges in striking the proper balance between sharing information and protecting personal privacy. However, people routinely trade personal information online when seeking product discounts and for the benefit of personalizing applications. So, how and what’s shared, in part, depends on what consumers gain. It’s reasonable to give up some privacy for meaningful rewards, right? There are now health insurance discounts based on shopping habits and information collected by health monitoring wearables. Auto insurance companies are already doing something similar in offering discounts based on devices that monitor driving behavior.

We are entering an era of real-time analysis capabilities. The kicker is that with real-time analysis comes the potential for real-time actionable insights to better serve our clients’ needs.

So, what’s today’s market researcher to do? Evolve. To avoid marginalization, market researchers need to continue to understand client issues and cultivate insights in regard to consumer behavior. To do so effectively in this new world, they need to embrace new and emerging analytical tools and effectively mine data from multiple disparate sources, bringing together the best of data science and knowledge curation to consult and partner with clients.

So, we can say goodbye to “traditional” market research? Yes, indeed. The market research landscape is constantly evolving, and the insights industry needs to evolve with it.

Matt Skobe is a Data Manager at CMB with keen interests in marketing research and mobile technology. When Matt reaches his screen time quota for the day he heads to Lynn Woods for gnarcore mountain biking.    

Topics: data collection, mobile, consumer insights, marketing science, internet of things, data integration, passive data

When Only a #Selfie Stands Between You and Those New Shoes

Posted by Stephanie Kimball

Thu, Aug 13, 2015

mobile, shopping, mobile walletThe next time you opt to skip the lines at the mall and do some online shopping from your couch, you may still have to show your face. . .sort of. MasterCard is experimenting with a new program that will require you to hold up your phone and snap a selfie to confirm a purchase.  MasterCard will be piloting the new app with 500 customers who will pay for items simply by looking at their phones and blinking once to take a selfie. The blink is another feature that ensures security by preventing someone from simply showing the app a picture of your face in an attempt to make a purchase.

As we all know, passwords are easily forgotten or even stolen. So, MasterCard is capitalizing on technology like biometrics and fingerprints to help their customers be more secure and efficient. While security remains a top barrier to mobile wallet usage, concern about security is diminishing among non-users. In addition to snapping a selfie, the MasterCard app also gives users the option to use a fingerprint scan. Worried that your fingerprints and glamour shots will be spread across the web? MasterCard doesn't actually get a picture of your face or finger. All fingerprint scans create a code that stays on your phone, and the facial scan maps out your face, converts it to 0s and 1s, and securely transmits it to MasterCard.

According to our recent Consumer Pulse Report, The Mobile Wallet – Today and Tomorrow, 2015 marks the year when mobile payments will take off. Familiarity and usage have doubled since 2013—15% have used a mobile wallet in the past 6 months and an additional 22% are likely to adopt in the coming 6 months. Familiarity and comfort with online payments has translated into high awareness and satisfaction for a number of providers, and MasterCard wants a slice of that pie. Among mobile wallet users, over a quarter would switch merchants based on mobile payment capabilities.

mobile wallet, wearables

Clearly the mobile wallet revolution is well underway, but the winning providers are far from decided, and MasterCard is taking huge leaps to see how far they can take the technology available. If MasterCard can successfully test and rollout these new features and deliver a product that their customers are comfortable using, they can capture some of the mobile wallet share from other brands like Apple Pay and PayPal.

So what’s next? Ajay Bhalla, President of Enterprise Safety and Security at MasterCard, is also experimenting with voice recognition, so you would only need to speak to approve a purchase. And don’t forget about wearables! While still in the early stages of adoption, wearables have the potential to drive mobile wallet use—particularly at the point of sale—which is why MasterCard is working with a Canadian firm, Nymi, to develop technology that will approve transactions by recognizing your heartbeat.

Since technology is constantly adapting and evolving, the options for mobile payments are limitless. We've heard the drumbeat of the mobile wallet revolution for years, but will 2015 be the turning point? All signs point to yes.

Want to learn more about our recent Consumer Pulse Report, The Mobile Wallet – Today and Tomorrow? Watch our webinar!

Watch Here!

Stephanie is CMB’s Senior Marketing Manager. She owns a selfie stick and isn’t afraid to use it. Follow her on Twitter: @SKBalls

Topics: technology research, financial services research, mobile, Consumer Pulse, retail research

Embracing Mobile Market Research

Posted by Brian Jones

Thu, Jul 23, 2015

Who are the mobile consumers?

mobile research, cmbLet’s get this straight: I am not addicted to my smartphone. Unlike so many of my fellow train commuters who stare zombie-eyed into their small screens, I am not immersed in a personal relationship with pixels. I have an e-Reader for that. But, my smartphone IS my lifeline.I’ve come to depend exclusively on my phone to keep me on-time and on-schedule, to entertain me (when not using my e-Reader), to stay in touch with family and friends, and to keep up-to-date with my work email. It’s my primary source for directions, weather, news, photography, messaging, banking, and a regular source for payment, shopping, and ticketing/reservations. I haven’t purchased a PC in nearly a decade, and I don’t have a landline. I also use my smartphone to take market research questionnaires, and I am far from alone. 

Data around smartphone usage aligns with my personal experience. In a recent CMB online study of U.S. consumers, optimized for mobile devices, 1 in 6 Millennials completed the questionnaire on a smartphone. Other studies report similar results. This example illustrates the issue with representativeness. Major panel vendors are seeing over half of Millennials joining their panels via a mobile device. 

mobile research, cmb

How do we adapt?

Much has been hypothesized about the future of market research under the new paradigm of mobile commerce, big data, and cloud services. New technologies and industry convergence (not just mobile) have brought sweeping changes in consumer behaviors, and market researchers must adapt.

A key component of successful adaptation will be greater integration of primary market research with other data streams. The promise of passive or observational data is captivating, but it is largely still in the formative stages. (For more on passive data, check out our recent webinar.) We still need and will likely always need active “please tell me” research. The shift from phone to online data collection has quickly been replaced with the urgency of a shift to mobile data collection (or at least device agnostic interviewing). Our industry has lagged behind because the consumer experience has become so personalized and the trust/value equation for tapping into their experiences is challenging. Tackling mobile market research with tactical solutions is a necessary step in this transition.

What should we do about it?  

  1. Understand your current audience. Researchers need to determine how important mobile data collection is to the business decision and decide how to treat mobile respondents. You can have all respondents use a mobile device, have some use a mobile device, or have mobile device respondents excluded. There are criteria and considerations for each of these, and there are also considerations for the expected mix of feature phones, smartphones, tablets, and PCs. The audience will determine the source of sample and representation that must be factored into the study design. Ultimately, this has a huge impact on the validity and reliability of the data. Respondent invitations need to include any limitations for devices not suitable for a particular survey.
  2. Design for mobile. If mobile participation is important, researchers should use a mobile first questionnaire design. Mobile optimized or mobile friendly surveys typically need to be shorter in length, use concise language, avoid complex grids and answering mechanisms, and have fewer answer options, so they can be supported on a small screen and keep respondents focused on the activity. In some cases,questionnaire modularization or data stitching can be used to help adhere to mobile design standards.
  3. Test for mobile. All questions, images, etc. need to display on a variety of screen sizes and within the bandwidth capacity of the devices that are being used. Android and iOS device accommodation covers most users. If app based surveys are being used, researchers need to ensure that the latest versions can be downloaded and are bug-free. 
  4. Apply data protection and privacy standards. Mobile market research comes with a unique set of conditions and challenges that impact how information is collected, protected, and secured. Research quality and ethical guidelines specific to mobile market research have been published by CASRO, ESOMAR, the MMRA (Mobile Marketing Research Association), and others.
  5. Implement Mobile Qualitative. The barriers are lower, and researchers can leverage the unique capabilities of mobile devices quite effectively with qualitative research. Most importantly, willing participants are mobile, which makes in-the-moment research possible. Mobile qualitative is also a great gateway to explore what’s possible for mobile quantitative studies. See my colleague Anne Hooper’s blog for more on the future of qualitative methodologies.
  6. Promote Research-on-Research. Experts need to conduct and publish additional research-on-research studies that advance understanding of how to treat mobile respondents and utilize passive data, location tracking, and other capabilities that mobile devices provide. We also need stronger evidence of what works and what doesn’t work in execution of multi-mode and mobile-only studies across different demographics, in B2B studies, and within different countries.

But perhaps the most important thing to remember is that this is just a start. Market researchers and other insight professionals must evolve from data providers to become integrated strategic partners—harnessing technology (not just mobile) to industry expertise to focus on decision-making, risk reduction, and growth.

Brian is a Senior Project Manager for Chadwick Martin Bailey, the photographer of the image in this post, and an 82 percenter—he is one of the 82% of mobile phone owners whose phone is with them always or most of the time. 

Watch our recent webinar that discusses the results of our self-funded Consumer Pulse study on the future of the mobile wallet. 

Watch Here!

Topics: methodology, qualitative research, mobile, research design

Mobile Passive Behavioral Data: Opportunities and Pitfalls

Posted by Chris Neal

Tue, Jul 21, 2015

By Chris Neal and Dr. Jay Weiner

Hands with phonesAs I wrote in last week’s post, we recently conducted an analysis of mobile wallet use in the U.S. To make it interesting, we used unlinked passive mobile behavioral data alongside survey-based data.In this post, I’ve teamed up with Jay Weiner—our VP of Analytics who helped me torture analyze the mobile passive behavioral data for this Mobile Wallet study—to share some of the typical challenges you may face when working with passive mobile behavioral data (or any type of passive behavioral data for that matter) along with some best practices for dealing with these challenges:

  1. Not being able to link mobile usage to individualsThere’s a lot of online passive data out there (mobile app usage ratings, web usage ratings by device type, social media monitoring, etc.) that is at the aggregate level and cannot be reliably attributed to individuals. These data have value, to be sure, but aggregate traffic data can sometimes be very misleading. This is why—for the Mobile Wallet project CMB did—we sourced mobile app and mobile web usage from the Research Now mobile panel where it is possible to attribute mobile usage data to individuals (and have additional profiling information on these individuals). 

    When you’re faced with aggregate level data that isn’t linked to individuals, we recommend either getting some sample from a mobile usage panel in order to understand and calibrate your results better and/or doing a parallel survey-sampling so you can make more informed assumptions (this holds true for aggregate search trend data, website clickstream data, and social media listening tools).
  1. Unstacking the passive mobile behavioral data. Mobile behavioral data that is linked to individuals typically comes in “stacked” form, i.e., every consumer tracked has many different records: one for each active mobile app or mobile website session. Analyzing this data in its raw form is very useful for understanding overall mobile usage trends. What these stacked behavioral data files do not tell you, however, is the reach or incidence (e.g., how many people or the percentage of an addressable market) of any given mobile app/website. It also doesn’t tell you the mobile session frequency or duration characteristics of different consumer types nor does it allow you to profile types of people with different mobile behaviors. 

    Unstacking a mobile behavioral data file can sometimes end up being a pretty big programming task, so we recommend deciding upfront exactly which apps/websites you want to “unstack.” A typical behavioral data file that tracks all smartphone usage during a given period of time can involve thousands of different apps and websites. . .and the resulting unstacked data file covering all of these could quickly become unwieldy.
  1. Beware the outlier! Unstacking a mobile behavioral data file will reveal some pretty extreme outliers. We all know about outliers, right? In survey research, we scrub (or impute) open-ended quant responses that are three standard deviations higher than the mean response, we take out some records altogether if they claim to be planning to spend $6 billion on their next smartphone purchase, and so on. But outliers in passive data can be quite extreme. In reviewing the passive data for this particular project, I couldn’t help but recall that delightful Adobe Marketing ad in which a baby playing with his parents’ tablet repeatedly clicks the “buy” button for an encyclopedia company’s e-commerce site, setting off a global stock bubble. 

    Here is a real-world example from our mobile wallet study that illustrates just how wide the range is of mobile behaviors across even a limited group of consumers: the overall “average” time spent using a mobile wallet app was 162 minutes, but the median time was only 23 minutes. A very small (<1% of total) portion of high-usage individuals created an average that grossly inflated the true usage snapshot of the majority of users. One individual spent over 3,000 minutes using a mobile wallet app.
  1. Understand what is (and what is not) captured by a tracking platform. Different tracking tools do different things and produce different data to analyze. In general, it’s very difficult to capture detailed on-device usage for iOS devices. . .most platforms set up a proxy that instead captures and categorizes the IP addresses that the device transmits data to/from. In our Mobile Wallet study, as one example, our mobile behavioral data did not pick up any Apple Pay usage because it leverages NFC to conduct the transaction between the smartphone and the NFC terminal at the cash register (without any signal ever being transmitted out to the mobile web or to any external mobile app, which is how the platform captured mobile usage).   There are a variety of tricks of the trade to account for these phenomenon and to adjust your analysis so you can get close to a real comparison, but you need to understand what things aren’t picked up by passive metering in order to apply them correctly.
  1. Categorize apps and websites. Needless to say, there are many different mobile apps and websites that people use, and many of these do a variety of different things and are used for a variety of different purposes. Additionally, the distribution of usage across many niche apps and websites is often not useful for any meaningful insights work unless these are bundled up into broader categories. 

    Some panel sources—including Research Now’s mobile panel—have existing mobile website and app categories, which are quite useful. For many custom projects, however, you’ll need to do the background research ahead of time in order to have meaningful categories to work with. Fishing expeditions are typically not a great analysis plan in any scenario, but they are out of the question if you’re going to dive into a big mobile usage data file.

    As you work to create meaningful categories for analysis, be open to adjusting and iterating. A certain group of specific apps might not yield the insight you were looking for. . .learn from the data you see during this process then try new groupings of apps and websites accordingly.
  1. Consider complementary survey sampling in parallel with behavioral analysis. During our iterative process of attempting to categorize mobile apps from reviewing passive mobile behavioral data, we were relieved to have a complementary survey sampling data set that helped us make some very educated guesses about how or why people were using different apps. For example, PayPal has a very successful mobile app that is widely used for a variety of reasons—peer-to-peer payments, ecommerce payments, and, increasingly, for “mobile wallet” payments at a physical point of sale. The passive behavioral data we had could not tell us what proportion of different users’ PayPal mobile app usage was for which purpose. That’s a problem because if we were relying on passive data alone to tell our clients what percent of smartphone users have used a mobile wallet to pay at a physical point of sale, we could come up with grossly inflated numbers. As an increasing number of mobile platforms add competing functionality (e.g., Facebook now has mobile payments functionality), this will remain a challenge.

    Passive tracking platforms will no doubt crack some of these challenges accurately, but some well-designed complementary survey sampling can go a long way towards helping you read the behavioral tea leaves with greater confidence. It can also reveal differences between actual vs. self-reported behavior that are valuable for businesses (e.g., a lot of people may say they really want a particular mobile functionality when asked directly, but if virtually no one is actually using existing apps that provide this functionality then perhaps your product roadmap can live without it for the next launch).

Want to learn more about the future of Mobile Wallet? Join us for a webinar on August 19, and we’ll share our insights with you!

Chris Neal leads CMB’s Tech Practice. He judges every survey he takes and every website he visits by how it looks on his 4” smartphone screen, and has sworn off buying a larger “phablet” screen size because it wouldn’t fit well in his Hipster-compliant skinny jeans.

Dr. Jay heads up the analytics group at CMB. He opted for the 6 inch “phablet” and baggy jeans.  He does look stupid talking to a brick. He’s busy trying to compute which event has the higher probability: his kids texting him back or his kids completing an online questionnaire. Every month, he answers your burning market research questions in his column: Dear Dr. Jay. Got a question? Ask it here!

Want to learn more about combining survey data with passive mobile behavioral data? Watch our recent webinar with Research Now that discusses these findings in depth.

Watch Now!

Topics: advanced analytics, methodology, data collection, mobile, Dear Dr. Jay, webinar, passive data