WELCOME TO OUR BLOG!

The posts here represent the opinions of CMB employees and guests—not necessarily the company as a whole. 

Subscribe to Email Updates

Jared Huizenga

Recent Posts

Swipe Right for Insights

Posted by Jared Huizenga

Wed, Aug 17, 2016

Data collection geeks like me can learn a ton at the CASRO Digital Research Conference. While the name of the event has changed many times over the years, the quality of the presentations and the opportunity to learn from experts in the industry are consistently good.

One topic that came up many years ago was conducting surveys via cellphones with SMS texts. This was at a time when most people had cellphones, but it was still a couple of years before the smartphone explosion. I remember listening to one presentation and looking down at my Samsung flip-phone thinking, “There’s no way respondents will take a CMB questionnaire this way.” For a few simple yes/no questions, this seemed like a fine methodology but it certainly wouldn’t fly for any of CMB’s studies.

For the next two or three years, less than half of the U.S. population owned smartphones (including yours truly). Even so, SMS texting was getting increasing coverage at the CASRO conference, and I was having a really hard time understanding why. Every year was billed as “the year of mobile!” I could see the potential of taking a survey while mobile, but the technology and user experience weren’t there yet. Then something happened that changed not only the market research industry but the way in which we live as human beings—smartphone adoption skyrocketed.
Girl_and_phone.jpg

Today in the U.S., smartphone ownership among adults is 72% according to the Pew Research Center. People are spending more time on their phones and less time sitting in front of a computer. Depending on the study and the population, anywhere from 20%-40% of survey takers are using their smartphones. And if it’s a study with people under 25 years old, that number would likely be even higher. We can approach mobile respondents in three ways:

  • Do nothing. This means surveys will be extremely cumbersome to take on smartphones, to the point where many will abandon during the painful process. This really isn’t an option at all. By doing nothing, you’re turning your back on the respondent experience and basically giving mobile users the middle finger.
  • Optimize questionnaires for mobile. All of CMB’s questionnaires are optimized for mobile. That is, our programming platforms identify the device type a respondent is using and renders the questionnaire to the appropriate screen size.  Even with this capability, long vertical grids and wide horizontal scales will still be painful for smartphone users since they will require some degree of scrolling. This option is better than nothing, but long questions are still going to be long questions.
  • Design questionnaires for mobile. This is the best option, and one that isn’t used often enough. This requires questions and answer options to be written with the idea that they will be viewed on smartphones. In other words, no lengthy grids, no sprawling scales, no drag and drop, minimal scrolling, or anything else that would cause the mobile user angst.  While this option sounds great, one of the criticisms has been that it’s difficult to do advanced exercises like max-diff or discrete choice on smartphones.

One cautionary note if you are thinking that a good option would be to simply disallow respondents from taking a survey on their smartphones.  Did your parents ever tell you not to do something when you were a child?  Did you listen to them or did you try it anyway? What’s going to happen when you tell someone not to take a survey on their mobile device?  Either by mistake or out of sheer defiance, some people will attempt to take it on their smartphone. This happened on a recent study for one of our clients.  These people tried to “stick it to the man,” but alas they were denied entry into the survey. If you want “representative” sample, the other argument against blocking mobile users is that you are blocking specific populations which could skew the results.

The respondent pool is getting shallow, and market research companies are facing increased challenges when it comes to getting enough “completes” for their studies.  It’s important for all of us to remember that behind every “complete” is a human being—one who’s trying to drag and drop a little image into the right bucket or one who’s scrolling and squinting to make sure they are choosing the right option on an 11-point scale in a twenty row grid.  Unless everyone is comfortable basing their quantitative findings off of N=50 in the future, we all need to take steps to embrace the mobile respondent. 

Jared is CMB’s Field Services Director, and has been in market research industry for eighteen years. When he isn’t enjoying the exciting world of data collection, he can be found competing at barbecue contests as the pitmaster of the team Insane Swine BBQ.

Sign up to receive our monthly eZine and never miss a webinar, conference recap, or insights from our self-funded research on hot topics from data integration to Social Currency.

Subscribe Here!

Topics: mobile, research design, Market research

What We’ve Got Here Is a Respondent Experience Problem

Posted by Jared Huizenga

Thu, Apr 14, 2016

respondent experience problemA couple weeks ago, I was traveling to Austin for CASRO’s Digital Research Conference, and I had an interesting conversation while boarding the plane. [Insert Road Trip joke here.]

Stranger: First time traveling to Austin?

Me: Yeah, I’m going to a market research conference.

Stranger: [blank stare]

Me: It’s a really good conference. I go every year.

Stranger: So, what does your company do?

Me: We gather information from people—usually by having them take an online survey, and—

Stranger: I took one of those. Never again.

Me: Yeah? It was that bad?

Stranger: It was [expletive] horrible. They said it would take ten minutes, and I quit after spending twice that long on it. I got nothing for my time. They basically lied to me.

Me: I’m sorry you had that experience. Not all surveys are like that, but I totally understand why you wouldn’t want to take another one.

Thank goodness the plane started boarding before he could say anything else. Double thank goodness that I wasn’t sitting next to him during the flight.

I’ve been a proud member of the market research industry since 1998. I feel like it’s often the Rodney Dangerfield of professional services, but I’ve always preached about how important the industry is. Unfortunately, I’m finding it harder and harder to convince the general population. The experience my fellow traveler had with his survey points to a major theme of this year’s CASRO Digital Research Conference. Either directly or indirectly, many of the presentations this year were about the respondent experience. It’s become increasingly clear to me that the market research industry has no choice other than to address the respondent experience “problem.”

There were also two related sub-themes—generational differences and living in a digital world—that go hand-in-hand with the respondent experience theme. Fewer people are taking questionnaires on their desktop computers. Recent data suggests that, depending on the specific study, 20-30% of respondents are taking questionnaires on their smartphones. Not surprisingly, this skews towards younger respondents. Also not surprisingly, the percentage of smartphone survey takers is increasing at a rapid pace. Within the next two years, I predict the percent of smartphone respondents will be 35-40%. As researchers, we have to consider the mobile respondent when designing questionnaires.

From a practical standpoint, what does all this mean for researchers like me who are focused on data collection?

  1. I made a bold—and somewhat unpopular—prediction a few years ago that the method of using a single “panel” for market research sample is dying a slow death and that these panels would eventually become obsolete. We may not be quite at that point yet, but we’re getting closer. In my experience, being able to use a single sample source today is very rare except for the simplest of populations.

Action: Understand your sample source options. Have candid conversations with your data collection partners and only work with ones that are 100% transparent. Learn how to smell BS from a mile away, and stay away from those people.

  1. As researchers, part of our job should be to understand how the world around us is changing. So, why do we turn a blind eye to the poor experiences our respondents are having? According to CASRO’s Code of Standards and Ethics, “research participants are the lifeblood of the research industry.” The people taking our questionnaires aren’t just “completes.” They’re people. They have jobs, spouses, children, and a million other things going on in their lives at any given time, so they often don’t have time for your 30-minute questionnaire with ten scrolling grid questions.

Action: Take the questionnaires yourself so you can fully understand what you’re asking your respondents to do. Then take that same questionnaire on a smartphone. It might be an eye opener.

  1. It’s important to educate colleagues, peers, and clients regarding the pitfalls of poor data collection methods. Not only does a poorly designed 30-minute survey frustrate respondents, it also leads to speeding, straight lining, and just not caring. Most importantly, it leads to bad data. It’s not the respondent’s fault—it’s ours. One company stood up at the conference and stated that it won’t take a client project if the survey is too long. But for every company that does this, there are many others that will take that project.

Action: Educate your clients about the potential consequences of poorly designed, lengthy questionnaires. Market research industry leaders as a whole need to do this for it have a large impact.

Change is a good thing, and there’s no need to panic. Most of you are probably aware of the issues I’ve outlined above. There are no big shocks here. But, being cognizant of a problem and acting to fix the problem are two entirely different things. I challenge everyone in the market research industry to take some action. In fact, you don’t have much of a choice.

Jared is CMB’s Field Services Director, and has been in market research industry for eighteen years. When he isn’t enjoying the exciting world of data collection, he can be found competing at barbecue contests as the pitmaster of the team Insane Swine BBQ.

Topics: data collection, mobile, research design, conference recap

Follow the Humans: Insights from CASRO’s Digital Research Conference

Posted by Jared Huizenga

Mon, Mar 09, 2015

iStock 000008338677XSmallI once again had the pleasure of attending the CASRO Digital Research Conference this year. It’s the one of the best conferences available to data collection geeks like me, and this year’s presentations did not disappoint. Here are a few key takeaways from this year’s conference.

1. The South shuts down when it snows. After having a great weekend in Nashville after the conference, my flight was cancelled on Monday due to about an inch of snow and a little ice. Needless to say, I was happy to return to Boston and its nine feet of snow.

2. “Big data” is an antiquated term. Over the past few years, big data has been the big buzz in the industry. Much like we said goodbye to traditional “market research,” we can now say adios to “big data.” Good riddance. The term was vague at best. However, that doesn’t mean that the concept is going away. It’s simply being replaced by new, more meaningful terminology like “integrated data” and “multi-sourced data.” But one thing isn’t changing. . .

3. Researchers still don’t know what to do with all that data. What can I say about multi-sourced data that I haven’t already said many times over the past couple years? Clients still want it, and researchers still want to oblige. But this fact remains: adequate tools still do not exist to deliver meaningful integrated data in most cases. We have a long way to go before most researchers will be able to leverage all of this data to its full potential in a meaningful way for our clients.

4. There’s a lot more to mobile research than how a questionnaire looks on a screen. For the past three or four years, it seems like every year is going to be “the year of mobile” at these types of conferences. Because of this, I always attend the mobile-related sessions skeptically. When we talk about mobile, more often than not, the main concern is how the questionnaire will look on a mobile device. But mobile research is much more than that. One of the best things I heard at the conference this year was that researchers should “follow the humans.” This is true on so many levels. Of course, a person can respond to a questionnaire invitation on his/her mobile device, but so much of a person’s daily life, including behaviors and attitudes, is shaped by mobile. Welcome to the world of the ultra-informed consumer. I can confidently say that 2015 is most definitely the year of mobile! (I do, however, reserve the right to say the same thing again next year.)

5. Researchers need to think like humans. It’s easy to get caught up in percentages in our world, and researchers sometimes lose sight of the human aspect of our industry. We like to think that millionaire CEOs are constantly checking their emails on their desktop computers, waiting for their next “opportunity” to take a 45-minute online questionnaire for a twenty-five cent reward. I attended sessions at the conference about gamification, how to make questionnaires more user-friendly, and also how to make questionnaires more kid-friendly by adding voice-to-text and text-to-voice options. All of these things have the potential to ease the burden on research participants, and as an industry, this must happen. We have a long way to go, but. . .

6. Now is the time to play catch-up with the rest of the world. Last year, I ended my recap by saying that change is happening faster than ever. I still think that’s true about the world we live in. With all of the technological advances and new opportunities provided to us, it’s an exciting time to be alive. However, I’m not sure I can honestly say that change is happening faster than ever when it comes to the world of research. I’ve been a part of this industry for a very fulfilling seventeen years, and sometimes my pride in the industry clouds my thinking. Let’s face the facts. The truth is that, as an industry, we are lagging far behind as the world speeds by. Research techniques and tools are evolving at a very slow pace, and I don’t see this changing in the near future. (In our defense, this is true for many industries and not only market research.) I still believe that those of us who are working to leverage the changing world we live in will be much better equipped for success than those who sit idly and watch the world fly.

I’m still confident that my industry is primed and ready for significant and meaningful change—even if we sometimes take the path of a tortoise. As a weekend pitmaster, I know that low and slow is sometimes the best approach. The end result is what really counts.

Jared is CMB’s Field Services Director, and has been in market research industry for seventeen years. When he isn’t enjoying the exciting world of data collection, he can be found competing at barbecue contests as the pitmaster of the cooking team Insane Swine BBQ.

 

Topics: big data, mobile, research design, conference recap

What's the Story? 5 Insights from CASRO's Digital Research Conference

Posted by Jared Huizenga

Wed, Mar 19, 2014

CMB and CASROWho says market research isn’t exciting? I’ve been a market researcher for the past sixteen years, and I’ve seen the industry change dramatically since the days when telephone questionnaires were the norm. I still remember my excitement when disk-by-mail became popular! But I don’t think I’ve ever felt as excited about market research as I do right now. The CASRO Digital Research Conference was last week, and the presentations confirmed what I already knew—big changes are happening in the market research world. Here are five key takeaways from the conference:

  1. “Market research” is an antiquated term. It was even suggested that we change the name of our industry from market research to “insights.” In fact, the word “insights” came up multiple times throughout the conference by different presenters. This makes a lot of sense to me. Many people view market research as a process whereas insights are the end result we deliver to our clients. Speaking for CMB, partnering with our clients to provide critical insights is a much more accurate description of our mission and focus. We and our clients know percentages by themselves fail to tell the whole story, and can in fact lead to more confusion about which direction to take.

  2. “Big data” means different things to different people. If you ask ten people to define big data you’ll probably get ten different answers. Some define it as omnipresent data that follows us wherever we go. Others define it as vast amounts of unstructured data, some of which might be useful and some not. Still others call it an outdated buzzword.  No matter what your own definition of big data is, the market research industry seems to be in somewhat of a quandary about what to do with it. Clients want it and researchers want to oblige, but do adequate tools currently exist to deliver meaningful big data? Where does the big data come from, who owns it, and how do you integrate it with traditional forms of data? These are all questions that have not been fully answered by the market research (or insights) industry. Regardless, tons of investment dollars are currently being pumped into big data infrastructure and tools. Big data is going to be, well, BIG.  However, there’s a long way to go before most will be able to use it to its potential.

  3. Empathy is the hottest new research “tool.” Understanding others’ feelings, thoughts, and experiences allows us to understand the “why behind the what.”  Before you dismiss this as just a qualitative research thing, don’t be so sure.  While qualitative research is an effective tool for understanding the “why,” the lines are blurring between qualitative and quantitative research. Picking one over the other simply doesn’t seem wise in today’s world. Unlike with big data, tools do currently exist that allow us to empathize with people and tell a more complete story. When you look at a respondent, you shouldn’t only see a number, spreadsheet, or fancy graphic that shows cost is the most important factor when purchasing fabric softener. You should see the man who recently lost his wife to cancer and who is buying fabric softener solely based on cost because he has five years of medical bills. There is value in knowing the whole story. When you look at a person, you should see a person.

  4. Synthesizers are increasingly important. I’m not talking about the synthesizers from Soft Cell’s version of “Tainted Love” or Van Halen’s “Jump.” The goal here is to once again tell a complete story and, in order to do this, multiple skillsets are required. Analytics have traditionally been the backbone of market research and will continue to play a major role in the future. However, with more and more information coming from multiple sources, synthesizers are also needed to pull all of it together in a meaningful way. In many cases, those who are good at analytics are not as good at synthesizing information, and vice versa. This may require a shift in the way market research companies staff for success in the future. 

  5. Mobile devices are changing the way questionnaires are designed. A time will come when very few respondents are willing to take a questionnaire over twenty minutes long, and some are saying that day is coming within two years. The fact is, no matter how much mobile “optimization” you apply to your questionnaire, the time to take it on a smartphone is still going to be longer than on PCs and tablets. Forcing respondents to complete on a PC isn’t a good solution, especially since the already elusive sub 25 year old population spends more time on mobile devices than PCs. So what’s a researcher to do? The option of “chunking” long questionnaires into several modules is showing potential, but requires careful questionnaire design and a trusted sampling plan. This method isn’t a good fit for all studies where analysis dictates each respondent complete the entire questionnaire, and the number of overall respondents needed is likely to increase using this methodology. It also requires client buy-in. But it’s something that we at CMB believe is worth pursuing as we leverage mobile technologies.

Change is happening faster than ever. If you thought the transition from telephone to online research was fast—if you were even around back in the good old days when that happened—you’d better hold onto your seat! Information surrounds every consumer. The challenge for insights companies is not only to capture that information but to empathize, analyze, and synthesize it in order to tell a complete story. This requires multiple skillsets as well as the appropriate tools, and honestly the industry as a whole simply isn’t there yet. However, I strongly believe that those of us who are working feverishly to not just “deal” with change but to leverage it, and who are making progress with these rapidly changing technological advances, will be well equipped for success.

Jared is CMB’s Director of Field Services, and has been in market research industry for sixteen years. When he isn’t enjoying the exciting world of data collection, he can be found competing at barbecue contests as the pitmaster of the team Insane Swine BBQ

 

CMB Insight eZine


Sign-up here
for out monthly eZine for the latest Consumer Pulse
reports, case studies conference updates, webinars and more

Topics: qualitative research, big data, mobile, research design, quantitative research, conference recap

International Market Research Starts with Choosing the Right Partner

Posted by Jared Huizenga

Tue, Apr 13, 2010

There's a lot to consider when collecting international data. It's a whole different ball game outside the U.S. market. From methodologies to translations to project management, a lot needs to be taken into consideration-even in other English speaking countries.  The first step to any successful international research project is choosing the right partner.There are so many choices and they are not all created equal. A single data collection partner will never be the right fit for every project. At CMB we have created our own Global Certified Network to ensure we have the most well rounded pool of partners we can team up with on every project.  Having our own certification process has allowed us to hold our data collection providers and partners to the same high level of expectations our clients have come to rely on us for.

To qualify for CMB's Global Certified Network our partners must agree to several strict requirements including industry standards, security requirements, data quality assurance, and project management guidelines. Some examples are...

  • Industry Standards:  All certified partners are required to comply with the ESOMAR International Code on Market and Social Research. In addition, all vendors must comply with national, regional, and local laws. They also must sign Chadwick Martin Bailey's Confidentiality Agreement.
  • Security Requirements:  All partners are required take active measures with regards to respondent privacy. This is especially true when using client-provided sample lists. The partner must be CAN-SPAM compliant and destroy all sample records at the end of a project-or at any time per Chadwick Martin Bailey's request. 
  • Data Quality Assurance:  Partners must demonstrate that procedures are in place to guard against "bad" data and if any issues with data collection arise, partners are required to inform us immediately and offer proactive solutions. Partners must inform us upfront when they are using additional partners for data collection and they must give us the names of those additional partners if issues come up.   
  • Project Management Guidelines: Partners are required to provide a minimum of two points of contact and to respond to queries and requests from the CMB project staff as quickly as possible. Partners must also agree to participate in frequent meetings to give us status updates.

These are just a few ways we ensure our partners share the same commitment and high standards we do when approaching each project. This certification has also allowed us to build an outstanding network of partners with some of the best and brightest companies in our industry.

Posted by Jared Huizenga is CMB's Field Services Manager. Jared is on the New England Barbecue Society's Board of Directors and is the pitmaster on a competition barbecue team.

Topics: international research, B2B research