Originally posted on Research Access
By Jeffrey Henning
Earlier this month, Chris Neal of Chadwick Martin Bailey shared with members of the New England chapter of the Marketing Research Association tips for running mobile diary studies, based on lessons learned from a recent project.
For the Council for Research Excellence (CRE), CMB studied mobile video usage to understand:
How much time is spent on mobile devices watching TV (professionally produced TV shows)?
Does this cannibalize TV set viewing?
What motivates consumers to watch on mobile?
How can mobile TV viewing be accurately tracked?
The research included a quantitative phase with two online surveys and mobile journaling, followed by a series of home ethnographies. The quant work included a screening survey, the mobile diary, and a final online survey.
The screening survey was Census balanced to estimate market size, with three groups recruited for comparison: those without mobile devices (smartphones or tablets), those with mobile devices who don’t watch TV on them, and those with mobile devices that they watch TV on. The total number of respondents was 5,886.
The mobile diary activity asked respondents to complete their journal 4 times a day for 7 days.
A final attitudinal survey was used to better understand motivations and behaviors associated with decisions about TV watching.
Along the way, CMB learned some valuable best practices for mobile diary studies, including tips for recruiting, incentives, design and analysis. The 10 key lessons learned:
Mobile panels don’t work for low incidence – Take care when using mobile panels – given the small size of many mobile panels, you may have better luck recruiting through traditional online panels, as CMB did. For this study, it was because of the comparatively low incidence of actual mobile TV watching.
Overrecruit – You will lose many recruits to the journaling exercise when it comes time to downloading the mobile diary application. As a general rule, over-recruit by 100% – get twice the promises of participation that you need. Most dropout occurs after the screening and before the participant has recorded a single mobile diary entry. For many members of online survey panels, journaling is a new experience. The second biggest point of dropout was after recording 1 or 2 diary entries.
Keep it short – To minimize this dropout, you have to keep the diary experience as short as possible: no more than 3 to 5 minutes long. The more times you ask participants to complete a diary each day, the greater the dropout rate.
Think small screen – Make sure the survey is designed to provide a good experience on small screens – avoid grids and sum-allocation questions and limit open-ended prompts and use of images. Use vertical scales instead of horizontal scales. “Be wary of shiny new survey objects for smartphone survey-takers,” said Chris. Smartphone users had 5 times the dropout rate of tablet or laptop users in this study. Enable people to log on to their journal from whatever device they were using at the time, including their computer.
Beware battery hogs – When evaluating smartphone apps, be wary of those that drain battery life by constantly logging GPS location. Check the app store reviews of the application.
Keep consistent – Keep the diary questionnaire the same for every time block, to get respondents into the habit of answering it.
Experiment with incentives to maximize participation – Tier incentives to motivate people to stick with the study and complete all time blocks. To earn the incentive for the CMB study, Chris said that respondents had to participate at least once a day for all 7 days, with additional incentives for every journal log entered (participants were reminded this didn’t have to involve actual TV watching, just filling out the log). In the end, 90% of journaling occasions were filled out.
Remind via SMS and email – In-app notifications are not enough to prompt participation. Use email and text messages for each time block as well. Most respondents logged on within 2 hours of receiving a reminder.
Use online surveys for detailed questions – Use the post-journaling survey to capture greater detail and to work around the limits of mobile surveys. You can then use these results to “slice and dice” the journal responses.
Weight by occasions – Remember to weight the data file to total occasions not total respondents. For missing data, leave it missing. Develop a plan detailing which occasion-based data you’re going to analyze and what respondent-level analysis you are going to do. You may need to create a separate occasion-level data file and a separate respondent-level data file.
Properly done, mobile diary studies provide an amazing depth of data. For this project, CMB captured almost 400,000 viewing occasions (mobile and non-mobile TV watching), for over 5 million occasion-based records!
Interested in the actual survey results? CRE has published the results presentation, “TV Untethered: Following the Mobile Path of TV Content” [PDF].
Jeffrey Henning, PRC is president of Researchscape International, a market research firm providing custom surveys to small businesses. He is a Director at Large on the MRA Board of Directors; in 2012, he was the inaugural winner of the MRA’s Impact award. You can follow him on Twitter @jhenning.
By Nick Pangallo
A few weeks ago, I found myself seated at a trendy Mexican restaurant in Minneapolis, an eager participant at one of the more enjoyable business lunches I’ve encountered lately. As you might expect, the topic quickly turned from the vagaries of the marketing life to the weather, summer vacation stories, the gym, far-too-early holiday planning (I’m looking at you, Target), and then, unexpectedly, to dieting. It was there where I learned Jim Garrity, SVP and head of CMB’s Financial Services, Insurance & Healthcare Practice, was quite a fan of Weight Watchers, one of the few truly successfully long-term dieting options out there – it earned Consumer Report’s highest mark for nutrition analysis.
Jim had become a devotee of the Weight Watchers PointsPlus® plan, which, for someone I’d fancied a meat-and-potatoes man like me, came as a bit of a surprise. But as Jim continued to discuss the program and what he enjoyed about it, I realized that PointsPlus® was nothing more than another example of CMB’s brand and product development bread-and-butter: the tradeoff exercise.
For those not familiar with how it works, PointsPlus® assigns a numeric point-value to each meal/snack/dessert/shake/whatever you can buy through the program, based on protein, carbohydrates, fat and fiber content. The system will then give you a daily points target, taking into account your height, weight, age and gender; read more about it here. Simply stay at or under your target, and…well, that’s it.
This, as you might expect, presents our would-be dieter with a daily flurry of choices. Do you have the bagel with cream cheese or the fresh fruit for breakfast? Go with a savory salad or a slim sandwich for lunch? And how do these choices affect what you have “left” at dinner? Sorry to burst your bubble, eager reader, but if you want that red velvet cake, you’re going to have to pass on the cream cheese.
Built on the economic principle of opportunity cost (the idea that to buy a product or undertake an activity, that product/activity must necessarily replace something else you might have bought or engaged in), these sorts of tradeoffs are exactly what we seek to model when researchers help develop brands, products, messaging campaigns, or basically anything else with a give-and-take. You can’t be the industry leader and try harder. If you want to offer premium customer service, you’re going to have to charge a bit more. Anyone who’s ever made a budget, whether on a ledger or www.Mint.com, understands that if you go to the concert, you might have to skip the movies this week.
Our job, then, is to master the usage of methodological techniques which replicate these real-world tradeoffs within a research setting. Almost all of my clients take advantage of Maximum-Difference Scaling, an exercise where participants select which items/features/messages/etc. they like most and least, four options at a time. By forcing this tradeoff, we can accurately prioritize huge lists of information in short order, not only rank-ordering but also sizing the distance between items. Many also use allocations, which allow participants to assign different values to a set of given options, but knowing there are only so many points to go around (often 100). My brand and product development engagements often utilize Discrete Choice (or Conjoint) Modeling, an extremely powerful form of tradeoff exercise where participants must choose between holistic brand positionings or fully-configured products. We can then deconstruct their decisions, analyze the tradeoffs, and find those pesky drivers of decision-making that are the foundation of marketing as we know it.
Think about it this way: if you’re still with me, you’ve traded off the time you could’ve spent watching E!, playing Candy Crush, or checking Facebook in favor of a little lesson in economics and research. So what’ll it be, then–the pasta or the seafood?
Nick is a Project Manager within CMB’s Financial Services, Insurance & Healthcare Practice, as well as a poker-playing behavioral economics and game theory nerd. You can follow him on Twitter @NAPangallo. He would always choose the pasta.
By Caitlin Dailey
A warning to all the college seniors thinking about a career in market research: becoming a market researcher is both a blessing and a curse. Since I began working at CMB, I can’t turn away from a questionnaire or a good read on survey statistics, and having a group of classmates and friends with the same background just adds fuel to the fire. Whenever we come across an interesting study, we just have to share it.
One study came into my inbox just 3 days ago. This particular study was about “what singles want.” I found this particularly interesting given that I’m a single “twenty-something” living in a city with a relatively large number of unmarried women. According to the “what singles want” study of over 5,000 unattached adults 21+, “54% would not date someone with substantial student loan debt.” In a world where a college degree is the norm and post-grad education is increasingly becoming a requirement, I find this statistic hard to swallow. When I first decided to attend Bentley University, I remember standing around in a friend’s kitchen senior year of high school. His mother asked the group of us what colleges we had decided on, and when I said Bentley, she responded (jokingly), “good luck finding a husband with all of those student loans you’re going to have.” This has become a running joke between us. Now that there is a statistic to back it up, the joke has turned into a reality.
And the good news just keeps on coming: “49% would consider getting into a committed relationship with someone who lived at home with parents.” I guess if I still lived at home my student loans would be less of an issue. Call me crazy, but something about living at home and being in a relationship just doesn’t add up. And let’s not neglect to mention the affect that social media has on the dating scene. “38% [of respondents] would cancel a date because of something they found while doing internet research on their date”. So your relationship could be doomed before you even get the chance to meet someone in person. And yet, finding partners online is becoming more and more common. Is your head spinning? Mine is.
So being a market researcher is both a blessing and a curse. I too easily get roped into reading all of these articles that make me contemplate life, and question whether I am in the norm or if I even want to be in the norm, but I also get to learn about market trends and answer real business questions for great clients. My head might be spinning with numbers and statistics, but it certainly makes life interesting.
Caitlin is a senior associate researcher for the Retail/Travel/Entertainment practice. Outside of work she is a company dancer with DanceWorks Boston, and continues her search for ‘Mr. Right’ despite her substantial student loan debt.
Think you've got what it takes? We're growing, take a look at our open positions here.
By Jeannine Rua
Happiness means something a bit different to each of us. To Tony Hsieh, the CEO of Zappos.com, happiness means working for a company that you’re excited about, surrounded by people who feel the same way.
As a researcher, I read his book Delivering Happiness as a subliminal message to anyone working with customer satisfaction or loyalty data. Reading his book, it’s clear Tony set out to highlight his journey, not to write a comprehensive corporate history or autobiography. This is the golden rule in report writing as well – pull out the highlights and tell a story around the most important pieces.
Tony adopted another critical rule of report-writing: write for your audience. Tony admits his book is not a work of grammatical perfection, but it’s written in a way that makes it easy for everyone to read and enjoy. This separates Delivering Happiness from many of its neighbors in the world of “business books” – its informal tone is easy to digest. Throughout the book, Tony seamlessly juxtaposes comical stories of growing up with Asian-American parents with stories of his ambition, failed attempts, and successes.
Beyond the autobiographical elements of the book, I found the managerial guidance in Delivering Happiness also relates to research. The following principals are important context when writing recommendations, but also when thinking about survey design and analysis interpretation. As an added bonus these “rules” might also serve you well in your personal interactions:
1. Remember that you are an n of 1; other people have opinions, too. When Tony was first approached with the idea of starting an internet shoe website, he was skeptical because he himself had never considered purchasing shoes through a catalog and couldn’t imagine people buying shoes without trying them on. My toes are thankful that Tony realized “it didn’t matter whether I would be willing to buy shoes without trying them on first.”
2. Embrace change with an open mind.
Zappos.com was originally based on drop-ship sales, and had shied away from opening a warehouse with inventory because it was not part of their business model. When they realized they were limited in what they could offer their customers, they thought, “If changing our business model is what’s going to save us, then we need to embrace and drive change.”
3. Listen to employee’s feedback. Customer feedback is great – but it’s important to also hear from your employees. Happy employees are critical to delivering a positive customer experience, and employees working in the thick of daily processes often have valuable insight and ideas around what would enable them to better deliver.
4. Pay attention to word of mouth and the lifetime value of customers. It’s important to think about how your company is interacting with customers at every level – one happy customer with a large network of friends may be more valuable than he first appears. Zappos.com trusts their employees and empowers them to help customers in any way they can – even if that means recommending another site for their purchase.
From a research perspective, mobile technology strikes me as the most obvious application for these principals. As mobile technology changes the way consumers shop and interact, we are presented with new opportunities for listening and observing. As you think about your personal and professional goals for the new year, keep an open mind and hopefully happiness will find you.
Jeannine is a Project Manager working with our Tech, eCommerce, and MedTech practice. She finds happiness learning about new places through reading, travelling, and talking with just about anyone she can find.
What's your plan for delivering happiness in 2013?
By Jeff McKenna
The recent research study showing that marketers rank lower than politicians on the “respectability scale” might feel like a kick in the gut for most of us in this role.
From the research: only 13% of consumers agree that marketing benefits society. It’s no surprise that teachers, scientists and engineers are the top of the list, but marketing even falls below bankers (32%), lawyers (34%), and politicians (18%). One point of solace, marketers are tied with actors and dancers; so, we’re not alone.
If we deconstruct the research, we can find plenty to take issue with. What research study isn’t immune to that? For instance, the focus of the research is about online advertising, while the questions about professional respectability come after questions about the effectiveness of different marketing methods. To what extent has this approach primed respondents in a certain direction?
Additionally, when you look more deeply at the results, you find that people still “respect” the need for marketing within business. Most, in fact, consider it “strategic” and necessary for sales.
So, the research findings shouldn’t be taken too personally. As noted earlier, marketers are in the same boat as actors and dancers. It makes me think of Ode by Arthur O’Shaughnessy:
We are the music makers,
And we are the dreamers of dreams,
Wandering by lone sea-breakers
And sitting by desolate streams;—
World-losers and world-forsakers,
On whom the pale moon gleams:
Yet we are the movers and shakers
Of the world for ever, it seems.
Or, as the great Willy Wonka puts it:
Who but a marketer would ever create lickable wallpaper with snozzberry flavor???
So marketers, don’t give up on your role and profession. And remember that without you, the world would be a place with much less flavor and much less fun.
When Jeff's not busy contributing to society at large, he serves as a senior consultant and methodologist for CMB; making sense of big data, and speaking on topics like mobile and the future of market research.
This November we'll be in sunny Boca Raton, for The Market Research Event, presenting with our friends at The Atlantic City Alliance and SunTrust Bank. We'll share findings, challenges, and insights from two very different segmentation studies:
Atlantic City: Building a Foundation for Future Growth
Track: Insight Driven Innovation Track
Time and Date: Tuesday November 13th @ 2:00 pm
Presenters: Judy Melanson, Vice President, Travel and Hospitality Practice, CMB and Rich Mirman, Consultant to the Atlantic City Alliance, Former CMO for Caesars Entertainment, The Mirman Group, LLC
While Atlantic City attracts over 30 million visitors a year, the popular tourist destination faces a number of challenges—including a recession and increased competition. Join us to hear how CMB and The Atlantic City Alliance took a foundational approach to segmentation—identifying and assessing Atlantic City's greatest growth opportunities to decide who to attract, and what to build.
SunTrust: Segmentation as a Change Agent
Track: Insights Leadership & Transformation Track
Time and Date: Wednesday November 14th @ 11:15 am
Presenters: Rich Schreuer, Senior Consultant, CMB; Mark Carr, Co-founder and Managing Partner, The South Street Strategy Group; Jeff VanDeVelde, SVP, Director of Client Experience and Loyalty, SunTrust Bank
Learn how CMB and The South Street Strategy Group partnered with SunTrust to develop and prioritize consumer segments based on goals and needs. This segmentation was central to forming the foundation for SunTrust’s brand strategy, and helping the bank make the critical shift from a product-focused to a customer-centric organization.
We're excited to see you, click here to register, and enter CMB2012 to receive 25% off.
Check out one of our favorite moments from last year: TMRE Highlight: The Art of Choosing
We’re pleased to announce Chadwick Martin Bailey was recently named as a “Top 50” firm in the annual Honomichl report of the top revenue-generating marketing research companies in the US. While we’re happy to be recognized for our revenue, what we are most proud of is our continued commitment to our core values—our clients, our approach, and our people.
We are a client-centered firm. Our clients trust us for the insights we provide them, and we take this responsibility seriously. Our collaborative approach and role as trusted advisors means that insights are translated into practical business advice and solutions.
Our mantra is solving companies’ problems. We provide the right information in the right way so clients gain insight—not confusing or burdensome data dumps. We maintain a methodological center of excellence, and use integrated data sources, rigorous research methods, and leading edge technology to solve real world problems and guide effective business decisions. Our offerings are not “off the shelf,” but rather acknowledge that every company’s situation is different and needs to be treated that way.
From the very beginning, we have strived to build a culture focused on team work, common sense, and excellence. The CMB client experience is so much about our people; our clients benefit not just from the efforts of individuals, but from a team of smart and committed people sharing their high-level skills and experience. We are a strategic learning organization where all of us are expected to be active learners and contributors. I am most proud of the way our consultants, analysts, and methodologists work together.
Learn more about why CMB stands out from the crowd:
Posted by Anne Bailey Berman. Anne is the President of Chadwick Martin Bailey and enjoys volunteering in the community, traveling with her family and spending time in her vegetable garden.
Earlier this month I had the chance to present at the Market Research Technology Event in Las Vegas. Beyond the fact I just could not get accustomed to watching people walk by conference rooms swigging beer and wearing in flip flops; for me the event raised more questions than provided answers.
During the conference, one of the most quoted reports was McKinsey’s: Big data: The next frontier for innovation, competition, and productivity. For me, one of the most striking takeaways from the report was a prediction that by 2018, the US will have a shortage of talent necessary for organizations to take advantage of big data—the US alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis to make effective decisions.
After we market researchers take a moment to celebrate our job security, we should consider that skilled market researchers will be asked to fill the space by taking on more tasks and working longer hours. As the gap widens between the influx of data and the analysts we need to make sense of it, are 80 hour weeks inevitable? Certainly workforce globalization will be a key to filling “big data” needs, but I was very surprised to hear little discussion of how technology will help us deal with this shortage.
I left the conference with the theory that the “new technology” we need is the yet-to-be-realized application of a tool to change a process to yield a quicker, lower cost, or better quality outcome. I think market researchers have yet to focus on how technology can act as a surrogate for the role they play within their organizations
So what might the future hold? I expect technology will allow market researchers to develop “analytical bots” to make sense of the vast ocean of data to answer specific business questions raised by internal clients. Watching Watson and Siri answer questions of fact with extremely high accuracy makes me wonder what our role will be. If these machines “have all the answers” then what purpose do we have? I don’t believe technology will replace market researchers; their skillset and output are still critical for companies to be competitive. The purpose is to create the rules and algorithms that convert the facts into relevant information. This is where market research skills will combine with technology to fill the resource gap.
We’ve heard a lot about expert systems—computer systems that emulate human decision-making. It’s my view that the market researchers who will lead in the next 5 to 7 years will be those who are setting up and managing expert systems, that take all of the facts and computations within large sets of data and apply what is relevant, to make decisions quickly, anywhere, and at any time.
Did you miss us at TDMRE? We'll be at the Audience Measurement Event in Chicago from May 21st to the 23rd. Register for a 25% discount by entering CMB2012 here.
Posted by Jeff McKenna, Jeff is a senior consultant at CMB and team leader for Pinpoint Suite-our innovative Customer Experience Management software. Want to learn more about how Pinpoint Suite can help you make sense of your "Big Data," schedule a demo here.
A new organization in our world recently bore its first fruit, of sorts. The Customer Experience Professionals Association (CXPA) hosted its first Member Insight Exchange in Boston on October 19th and 20th. It was thought-provoking, to say the least.
The CXPA, led by customer experience industry experts Bruce Temkin and Jeanne Bliss, is “a global non-profit organization dedicated to the advancement of customer experience management practices”, filling a role (as Bruce puts it) similar to the American Medical Association and American Bar Association. Now, customer experience (CX) professionals have “a professional organization that will set standards, endorse and share best practices, and raise awareness of the importance of CX and the value of CX professionals.” It seems to have scratched an itch, as it already has nearly 1,000 members, just in the first seven months since starting in March 2011.
I noticed a few research firms in attendance, many more EFM firms, and a plethora of consultancies, peppered in with a sizeable number of client practitioners. There were presentations on exemplary VOC programs, customer experience journey mapping, creating a customer experience organization, types of metrics, and best practices from customer experience executives at B2B and B2C companies.
As the organization is still seeking to establish its role within the industry, the event featured many discussions about the expectations and needs of its members. Since members come from a wide array of job roles and companies, the needs are highly diverse; which indicates to me that the CXPA has a challenge to achieve the goals of “setting standards” and “identifying best practices”. It will be interesting to see how the CXPA defines its niche within the broad landscape of professional organizations, especially when the American Marketing Association has a larger footprint.
On the other hand, the CXPA certainly focuses on a very specific purposeful business activity: that of how companies interact with their customers. And I have to admit that, as companies accept that customers are “assets” with a “lifetime value”, the CXPA positioning speaks more of value than of cost to corporate finance. (Of course, that only works up until the point when someone asks for the actual ROI on customer experience management…)
But what does it matter to us (market researchers)? I kept asking that question to myself throughout the event. Since market researchers function as the information link between customers (and non-customers) and managers who must make decisions that will affect the customers, I remain surprised that only a few research firms attended the event. Meanwhile, all of the presentations addressed solutions we continually provide: segmentation, customer experience measurement, loyalty, and product/service design – but often with different terminology and language to define their application.
It strikes me that market researchers are not always aligned with the people we should be serving. Thus, we can all do a better job of learning and using the different languages required within our business. As practitioners with the strongest (or at least, it should be the strongest) skillset for collecting and converting data (any and all types of data) into useful information, market researchers need to be “in-demand” by all parts of the organization, focused on meeting the customer’s needs. This is easier said than done; but if our task is to provide customer insights, then we should be working harder to meet the needs of our own customers.
How are you working to address and improve the experience of your own customers?
Posted by Jeff McKenna. Jeff leads CMB’s customer experience initiatives and is the chief innovator behind CMB’s new customer experience management software Pinpoint Suite.
Today CMB launched Pinpoint Suite, Customer Experience Management Software designed to assist managers and researchers make better use of customer feedback data by translating the data into actions to improve the customer experience and the bottom line. Find out more.
Over the past few months I (like most marketers) have been reading a lot about social media and its role in marketing plans going forward. We have put out our own reports on what people are doing online and eMarketer and Mashable (among others) put out tons of great content every day.
Through all of my reading and in conversations with clients and prospects I continue to be surprised how little many companies know about the people they are engaging with via social media. Thinking about Facebook specifically, as an admin of a Fan page you get access to limited data, but nothing particularly useful for developing and improving content and engagement strategies. So, as we launch a new research approach to understanding Facebook Fans it seems that there are three things at a minimum that brands should know about their Fans (or “Likers”) to develop a meaningful strategy and get results from their efforts:
1) What are they looking for from you on Facebook? To keep your Fans engaged you need to understand why they have become Fans and what content they are hoping to see from you. Also, how they expect content to differ (or not) from what they might get from you on your corporate website, blog,etc.
2) How well are you meeting Fans’ expectations? Facebook as an engagement or marketing channel doesn’t give you much for mass feedback. You can (and should) read what is posted to your brand's (or brands’) wall and keep an eye on the number of new and departing Fans, but how much do you really know about how you are doing? Asking a few simple questions of your Fans on a periodic basis can give you tremendous insight into where you can improve and what the people who don’t post comments on your wall might be thinking. And because you can’t really segment your audience, you need to understand if you are not giving most of the people what they want before they begin to tune you out.
3) How can you increase engagement and enhance the Fan experience? Once you know if you are hitting the mark with your current content, it is important to know what your Fans think of other potential Facebook initiatives before you invest too heavily. Are they interested in new applications? Contests? Videos? Tools that connect to your brand and products? Testing the waters before you spend valuable development resources can improve your social media ROI.
Facebook offers brands a tremendous opportunity to engage with customers and prospects, but much of it is under leveraged and under-utilized. To really capitalize on the opportunity, you need to tap into your Fans and listen to what they have to say, don’t assume you already know.
Posted by Josh Mendelsohn. Josh is our VP of Marketing and loves live music, tv, great food, market research, New Orleans, marketing, his family, Boston and sports. You can follow him on Twitter @mendelj2.