Voices: the CMB Blog

By Chris Neal and Dr. Jay Weiner

As I wrote in last week’s post, we recently conducted an analysis of mobile wallet use in the U.S. To make it interesting, we used unlinked passive mobile behavioral data alongside survey-based data.In this post, I’ve teamed up with Jay Weiner—our VP of Analytics who helped me torture analyze the mobile passive behavioral data for this Mobile Wallet study—to share some of the typical challenges you may face when working with passive mobile behavioral data (or any type of passive behavioral data for that matter) along with some best practices for dealing with these challenges:

1. Not being able to link mobile usage to individuals. There’s a lot of online passive data out there (mobile app usage ratings, web usage ratings by device type, social media monitoring, etc.) that is at the aggregate level and cannot be reliably attributed to individuals. These data have value, to be sure, but aggregate traffic data can sometimes be very misleading. This is why—for the Mobile Wallet project CMB did—we sourced mobile app and mobile web usage from the Research Now mobile panel where it is possible to attribute mobile usage data to individuals (and have additional profiling information on these individuals). 
   
   When you’re faced with aggregate level data that isn’t linked to individuals, we recommend either getting some sample from a mobile usage panel in order to understand and calibrate your results better and/or doing a parallel survey-sampling so you can make more informed assumptions (this holds true for aggregate search trend data, website clickstream data, and social media listening tools).

2. Unstacking the passive mobile behavioral data. Mobile behavioral data that is linked to individuals typically comes in “stacked” form, i.e., every consumer tracked has many different records: one for each active mobile app or mobile website session. Analyzing this data in its raw form is very useful for understanding overall mobile usage trends. What these stacked behavioral data files do not tell you, however, is the reach or incidence (e.g., how many people or the percentage of an addressable market) of any given mobile app/website. It also doesn’t tell you the mobile session frequency or duration characteristics of different consumer types nor does it allow you to profile types of people with different mobile behaviors. 
   
   Unstacking a mobile behavioral data file can sometimes end up being a pretty big programming task, so we recommend deciding upfront exactly which apps/websites you want to “unstack.” A typical behavioral data file that tracks all smartphone usage during a given period of time can involve thousands of different apps and websites. . .and the resulting unstacked data file covering all of these could quickly become unwieldy.

3. Beware the outlier! Unstacking a mobile behavioral data file will reveal some pretty extreme outliers. We all know about outliers, right? In survey research, we scrub (or impute) open-ended quant responses that are three standard deviations higher than the mean response, we take out some records altogether if they claim to be planning to spend $6 billion on their next smartphone purchase, and so on. But outliers in passive data can be quite extreme. In reviewing the passive data for this particular project, I couldn’t help but recall that delightful Adobe Marketing ad in which a baby playing with his parents’ tablet repeatedly clicks the “buy” button for an encyclopedia company’s e-commerce site, setting off a global stock bubble. 
   
   Here is a real-world example from our mobile wallet study that illustrates just how wide the range is of mobile behaviors across even a limited group of consumers: the overall “average” time spent using a mobile wallet app was 162 minutes, but the median time was only 23 minutes. A very small (<1% of total) portion of high-usage individuals created an average that grossly inflated the true usage snapshot of the majority of users. One individual spent over 3,000 minutes using a mobile wallet app.

4. Understand what is (and what is not) captured by a tracking platform. Different tracking tools do different things and produce different data to analyze. In general, it’s very difficult to capture detailed on-device usage for iOS devices. . .most platforms set up a proxy that instead captures and categorizes the IP addresses that the device transmits data to/from. In our Mobile Wallet study, as one example, our mobile behavioral data did not pick up any Apple Pay usage because it leverages NFC to conduct the transaction between the smartphone and the NFC terminal at the cash register (without any signal ever being transmitted out to the mobile web or to any external mobile app, which is how the platform captured mobile usage).   There are a variety of tricks of the trade to account for these phenomenon and to adjust your analysis so you can get close to a real comparison, but you need to understand what things aren’t picked up by passive metering in order to apply them correctly.

5. Categorize apps and websites. Needless to say, there are many different mobile apps and websites that people use, and many of these do a variety of different things and are used for a variety of different purposes. Additionally, the distribution of usage across many niche apps and websites is often not useful for any meaningful insights work unless these are bundled up into broader categories. 
   
   Some panel sources—including Research Now’s mobile panel—have existing mobile website and app categories, which are quite useful. For many custom projects, however, you’ll need to do the background research ahead of time in order to have meaningful categories to work with. Fishing expeditions are typically not a great analysis plan in any scenario, but they are out of the question if you’re going to dive into a big mobile usage data file.
   
   As you work to create meaningful categories for analysis, be open to adjusting and iterating. A certain group of specific apps might not yield the insight you were looking for. . .learn from the data you see during this process then try new groupings of apps and websites accordingly.

6. Consider complementary survey sampling in parallel with behavioral analysis. During our iterative process of attempting to categorize mobile apps from reviewing passive mobile behavioral data, we were relieved to have a complementary survey sampling data set that helped us make some very educated guesses about how or why people were using different apps. For example, PayPal has a very successful mobile app that is widely used for a variety of reasons—peer-to-peer payments, ecommerce payments, and, increasingly, for “mobile wallet” payments at a physical point of sale. The passive behavioral data we had could not tell us what proportion of different users’ PayPal mobile app usage was for which purpose. That’s a problem because if we were relying on passive data alone to tell our clients what percent of smartphone users have used a mobile wallet to pay at a physical point of sale, we could come up with grossly inflated numbers. As an increasing number of mobile platforms add competing functionality (e.g., Facebook now has mobile payments functionality), this will remain a challenge.
   
   Passive tracking platforms will no doubt crack some of these challenges accurately, but some well-designed complementary survey sampling can go a long way towards helping you read the behavioral tea leaves with greater confidence. It can also reveal differences between actual vs. self-reported behavior that are valuable for businesses (e.g., a lot of people may say they really want a particular mobile functionality when asked directly, but if virtually no one is actually using existing apps that provide this functionality then perhaps your product roadmap can live without it for the next launch).

Want to learn more about the future of Mobile Wallet? Join us for a webinar on August 19, and we’ll share our insights with you!

Chris Neal leads CMB’s Tech Practice. He judges every survey he takes and every website he visits by how it looks on his 4” smartphone screen, and has sworn off buying a larger “phablet” screen size because it wouldn’t fit well in his Hipster-compliant skinny jeans.

Dr. Jay heads up the analytics group at CMB. He opted for the 6 inch “phablet” and baggy jeans.  He does look stupid talking to a brick. He’s busy trying to compute which event has the higher probability: his kids texting him back or his kids completing an online questionnaire. Every month, he answers your burning market research questions in his column: Dear Dr. Jay. Got a question? Ask it here!

Want to learn more about combining survey data with passive mobile behavioral data? Watch our recent webinar with Research Now that discusses these findings in depth.

The explosion of mobile web and mobile app usage presents enormous opportunities for consumer insights professionals to deepen their understanding of consumer behavior, particularly for “in the moment” findings and tracking consumers over time (when they aren’t actively participating in research. . .which is 99%+ of the time for most people). Insight nerds like us can’t ignore this burgeoning wealth of data—it is a potential goldmine. But, working with passive mobile behavioral data brings with it plenty of challenges, too. It looks, smells, and feels very different from self-reported survey data:

• It’s big. (I’m not gonna drop the “Big Data” buzzword in this blog post, but—yep—the typical consumer does indeed use their smartphone quite a bit.)
• It’s messy.
• We don’t have the luxury of carefully curating it in the same way we do with survey sampling. 

As we all find ourselves increasingly tasked with synthesizing insights and a cohesive “story” using multiple data sources, we’re finding that mobile usage and other data sources don’t always play nicely in the sandbox with survey data. Each of them have their strengths and weaknesses that we need to understand in order to use them most effectively. 

So, in our latest in a series of sadomasochistic self-funded thought leadership experiments, we decided to take on a challenge similar in nature to what more and more companies will ask insights departments to do: use passive mobile behavioral data alongside survey-based data for a single purpose. In this case, the topic was an analysis of the U.S. mobile wallet market opportunity. To make things extra fun, we ensured that the passive mobile behavioral data was completely unlinked to the survey data (i.e., we could not link the two data sources at the respondent level for deeper understanding or to do attitudinal + behavioral based modeling). There are situations where you’ll be given data that is linked, but currently—more often than not—you’ll be working with separate silos and asked to make hay.

During this experiment, a number of things became very clear to us, including:

• the actual value that mobile behavioral data can bring to business engagements
• how it could easily produce misleading results if you don’t properly analyze the data
• how survey data and passive mobile behavioral data can complement one another greatly

Interested? I’ll be diving deep into these findings (and more) along with Roddy Knowles of Research Now in a webinar this Thursday, July 16^th at 1pm ET (11am PT). Please join us by registering here. 

Chris leads CMB’s Tech Practice. He enjoys spending time with his two kids and rock climbing.

Watch our recent webinar with Research Now to hear the results of our recent self-funded Consumer Pulse study that leveraged passive mobile behavioral data and survey data simultaneously to reveal insights into the current Mobile Wallet industry in the US.

We all know the consumer purchase journey has changed dramatically since the “mobile web” explosion and continues to evolve rapidly. In order to understand the current state of this evolving journey, CMB surveyed 2,000 recent buyers of tablets in the U.S. We confirmed several things that we expected to see, but we also busted a few myths along the way: 

1. TRUE: “Online media and advertising are now essential to influence consumers.”

• Reading about tablets online and online advertisements are the top ways in which consumers learn about new brands or products. [Tweet this.]
• Nearly everyone we surveyed does some type of research and evaluation online before buying—most commonly using online-only shopping sites (e.g., Amazon, eBay, etc.), general web searches, consumer electronics store websites, review websites (e.g., CNET, Engadget, etc.), or tablet manufacturer websites.

2. TRUE: “The mobile web is becoming more important in the consumer purchase journey.”

• Over half of buyers use the mobile web during the research and evaluation phase, and nearly 40% of buyers do so as a part of the final purchase decision (although very few people actually purchase a tablet using a mobile device). [Tweet this.]

3. FALSE: “Mobile applications are becoming very important in the consumer purchase journey.”

• Although the mobile web is now highly influential, very little purchase journey activity actually happens from within a mobile application per se. This could be because tablet purchasing isn’t something that happens frequently for more individual consumers (high-frequency activities lend themselves better to a dedicated app to expedite and track them). [Tweet this.]

4. FALSE: “Social Media is becoming very important in the consumer purchase journey.”

• The purchase journey for tablets is indeed very “social” (i.e., word-of-mouth and consumer reviews are hugely influential), but precious little of this socialization actually happens on social media platforms in the case of U.S. tablet buyers. [Tweet this.]

5. FALSE: “The Brick and Mortar Retail Store is Dead.”

• The rise of all things online does not spell the death of brick and mortar retail in the consumer electronics category. In-store experiences (including speaking with retail sales associated and doing hands-on demos of tablets) were one of the top sources of influence during the research and evaluation phase, regardless of whether they ultimately bought their tablet in a physical store. 
• Next to ads, in-store experiences were the top source of awareness for new tablet brands and models. 41% of those who learned about new makes/models during the process did so inside of a physical retail store. [Tweet this.]
• Half of all buyers surveyed actually bought their tablet in a physical retail store. [Tweet this.]

6. TRUE: The line between “online” and “offline” purchase journeys is becoming blurred.

• Most people use both online and offline sources during their purchase journey, and they typically influence one another. People doing research online may discover that a tablet model they are interested in is on sale at a particular retailer. At the same time, something a retail sales associate recommends to a shopper in a store may spur an online search in order to read other consumer reviews and see where they can get the recommended model the cheapest and fastest. Smartphone-based activities from within a retail store are just as common as interacting with an actual salesperson face-to-face at this point. 

The mobile web is undoubtedly here to stay, and how consumers go about making various different buying decisions will continue to evolve along with future changes in the mobile web. Here at CMB, we will continue to help companies and brands adapt to these shifts.

Download the full report. 

For more on our mobile stitching methodology, please see CMB's Chris Neal's webinar with Research Now: Watch the Webinar

Chris leads CMB’s Tech Practice. He enjoys spending time with his two kids and rock climbing.

It’s always exciting when we get the opportunity to conduct research that garners interest from everyone from the guy staring at his tablet on the train to the executives of the largest media companies in the world. We got that chance, when CMB partnered with the Council for Research Excellence to lead a study exploring how mobile media devices (tablets, phones, and laptops) impact overall television viewing behavior.

Highlights of the study include:

• Mobile TV viewers tend to be younger (mean age 35), higher income professionals with graduate degrees, and reflect more ethnic diversity than non-mobile-TV users;

• Mobile TV viewers are often heavy overall TV viewers and are more likely than non-mobile-TV viewers to be TV show opinion leaders and to use social media to talk about TV.

• Viewers are more commonly engaged when watching TV on a mobile device than when watching on a television set: they are less commonly doing unrelated tasks on other devices, and more commonly doing activities related to the show they are watching (e.g., looking up info about the show, posting about the show on social networks, etc.) when on a mobile device.

You can download the report here: TV Untethered: Following the Path of Mobile Content

Watch the presentation here: 

Posted by Chris Neal. Chris leads CMB’s Tech Practice. He enjoys spending time with his two kids and rock climbing.

One of the things I love most about my job is that I get to see what’s really going on in the minds of the people who do (or would) actually pay for B2B technology solutions, while at the same time observing industry trade press hype cycles and B2B marketing trends from solution providers. Sometimes these sync; sometimes they don’t. So let’s take a look at a few things currently waxing in the hype cycle that are “real,” and some other current conventional wisdom that doesn’t jive with what I’ve been seeing on the ground.  TREND #1: EVOLUTION OF IT BUYING AUTHORITY AWAY FROM CENTRAL IT DEPARTMENTS

Fact: It is indeed true that decision-making authority at many companies is moving away from central IT and towards non-IT executives or within business units. This is more commonly happening with functionally-specific applications and/or mobile devices. It is not happening in areas like data center infrastructure, networking and IT security. 

Myth: IT departments are actively trying to control all aspects of the IT buying process at companies.

• The truth is, IT Pros I survey or interview are focused on aligning IT with business needs, actively listening and reacting to requests from senior management, LoB managers and end-user employees. They follow up with these requests to do more detailed research of specific solutions and alternatives to present different options with informed recommendations, and the vet any potential new application or device for security and network performance requirements.

 Myth: IT departments think they have total control over all IT buying when in fact much of it happens without their knowledge. 

• IT Pros I survey still think they have more involvement/authority than non-IT executives, LoB managers and end-users say they do, but that “reality perception” gap has been shrinking over the years. Now, many IT Departments acknowledge that identifying the need for new technology solutions and even a lot of the researching and recommendation of specific tools and brands comes from non-IT departments.

TREND #2: THE CONSUMERIZATION OF IT

Fact: The consumerization of IT is accelerating and more employees want to use personal devices, apps and software for work purposes. 

Myth:  IT departments are always fighting the consumerization of IT trend.

• Most IT Departments I investigate now acknowledge (and many actively support) consumerization of IT trends, most commonly helping employees link personal mobile devices to things like corporate email and calendaring accounts. IT is focused on making employees more productive, and this is an easy way to enable this.

TREND #3: TABLETS BECOMING MORE COMMONPLACE AT WORK  

Fact: Tablet penetration is increasing at companies, although it is still relatively rare for most employees to have a company-issued tablet at this point. It is more common for employees to bring in personal tablets and use them for work purposes (see “Trend #2” above 

Myth: Tablets are replacing computers at companies.

• “Hard cannibalization” of company laptops by tablets simply isn’t happening much. It is extremely rare for employees at this point to get rid of their good ‘ol fashioned laptop altogether and go all-tablet, all-the-time. Any employee who needs to produce stuff (e.g., worker-bees) as opposed to consuming things (e.g., senior management reviewing the things that worker-bees produce) still needs and used laptops with larger screens and a quaint QWERTY keyboard.

 Fact: Tablets are extending the refresh cycles of laptops at companies.

• “Soft cannibalization” of company laptops by tablets does indeed happen quite frequently once tablets are in the mix. Employees who use tablets for work tend to use their laptop less for certain tasks, and with less wear-and-tear IT departments are pushing out the refresh cycles of their laptop fleet.

 Myth: Tablets will negate the need for printing at the office.

• Certain tasks and certain documents need to be printed at work. Whenever tablets are used to do these tasks…employees still want to print for them, and IT departments are generally happy to deploy mobile printing solutions if that’s what a critical mass of employees (or even a single, vocal senior executive) want. More computing devices in play generally leads to more printing, not less.  

TREND #4: CLOUD COMPUTING

Fact: Cloud computing is growing by leaps and bounds in corporate America. This trend is indeed real now, after several years where the industry marketing hype did not sync with the volume of deals actually being signed or the proclivity of IT departments to switch to cloud-based app delivery models.

Myth: IT departments are threatened by cloud computing and resisting this trend.

• Initially, IT departments were very skeptical about the security of cloud apps, and distrustful of complex, pay-as-you-go pricing models that could be potential budget-busters. These days, IT departments are more often than not the champions of the shift to the cloud, and executive management sometime puts the kibosh on initiatives because they can involve extra near-term budget (and staffing resources) to make the initial switch.

Myth: Companies are going “all-cloud” and converting their old internal data centers into gyms or rec rooms. 

• It is very rare for companies to have all their apps and storage on the cloud…I’m typically seeing a patchwork of internally-hosted apps, use of some public cloud services, other apps going onto private cloud infrastructures, and hybrid models. Certain apps are difficult to move to cloud provisioning for a variety of reasons (e.g., performance requirements, compliance with regulations, certain app vendors not yet supporting cloud delivery options or the ones they are offering aren’t fully baked yet, app customization needs). What IT departments really need now and for the foreseeable future is better management and security solutions that help them deal with this mixed environment, because it is likely here to stay for quite some time.

IT is changing dramatically and will no doubt look very different 2-5 years from now. The way these trends actually pan out always produce a few surprises, however. So stay tuned to this channel for future episodes of “IT Myth-busters.”

Chris leads CMB’s Tech Practice. He enjoys spending time with his two kids and rock climbing.