Voices: the CMB Blog

We’ve all done it . . . made New Year’s resolutions that this will finally be the year to eat healthier, to go on that dream trip to Europe, or to get more organized. Our hopes are high and our intentions are good—and yet, the end of the New Year comes around and our resolutions remain undone.For market researchers and businesses, the New Year puts a focus on planning for new research and evaluating research already in place, including that tracker that’s important to continue but, over time, has turned stale and “tired.” You continue to track, but it isn’t providing the same level of value or insights as it has in the past. It needs to be rejuvenated. 

At Chadwick Martin Bailey, we believe that if your tracker isn’t helping your company grow, stay ahead of the competition, or set strategic priorities, you need to make a change. If one of your resolutions for 2015 is to refresh your tired tracker, here are some things that will help you achieve that goal this year:  

• Evaluate your tracker with a fresh eye to make sure you’re asking the right questions, in the right way, to generate insights that support your business decisions for 2015 and the years to come. Even though most of us shudder at the thought of touching a tracker in any way (how many times have you said the words “but what impact will that have on trending?!”), today’s pace of change in business is remarkable. The landscape that currently exists for your business may be very far from what it was when your tracker began. 

Make an honest assessment of the questions you ask and how you ask them. Would you be gaining deeper, more actionable insights if you made a change? Then, make careful decisions about trade-offs, specifically between improving usefulness vs. losing trendability. If you deem a change necessary, create a transition plan. Conduct a parallel pilot test of the change when possible. Have discussions with your stakeholders to ensure everyone understands the trade-offs that will be made.

• Focus on the strategic and tactical decisions the business needs to make from this tracking research. Have conversations with your stakeholders and information users. Find out which results from the existing tracker are actively used and which results are never touched. 
  • If you find some results are no longer used or cannot provide insights to drive action, consider cutting the questions. 
  • If you find new information needs have arisen that require tracking, add questions that will address them. For any new needs that don’t need to be tracked over time, consider incorporating a “rotating” module into your tracker (a short section of questions open to change wave-to-wave). This helps leverage the tracker to address specific related questions without undergoing the cost and time of a separate research effort. 

What information does your business currently need in order to take action? Knowing the answer to this question and keeping your tracker current to address those needs elevates its usefulness to drive action and decision-making.

• Ensure your tracker deliverables are telling a story that is relevant to each audience. You should be delivering the right insights to the right stakeholders, and these insights should be in a form that allows them to act. This means no data overload. It’s hard to identify insights when they sit somewhere within a 100 page deck. It’s harder to digest business-changing recommendations when you only have 20 minutes on the calendar to review them before the stakeholders are off to their next meeting. It’s even harder for your stakeholders to decide what type of action they should take when the information is delivered in a “one size fits all” format. 

It’s important to think about how to customize your tracker deliverables in a succinct way that readily speaks to each stakeholder’s role and what decisions they need to make, so you don’t fall into the trap of just delivering updates on the same set of metrics wave after wave.

Topline reports may work well for certain audiences while scorecards and dashboards might work better for others. Don’t be afraid to deliver results creatively and in a visually-compelling format. At CMB, we often include dynamic deliverables such as easy-to-digest infographics, one-pagers, posters, and video/motion graphics. These dynamic deliverables are all focused on communicating the story (not the data!) in a way that is relevant and useful for enabling action across our clients’ organizations. 

So if you’ve made a commitment that this year will finally be the year that you rejuvenate that tired tracker, consider the areas above when setting it up to support confident, strategic decision-making in 2015 and beyond.

Wendy is the Account Director of CMB’s Financial Services practice. She has two children, and she loves Cape Cod, the Boston Celtics, and refreshing tired trackers. Her 2015 New Year’s resolution is to finally make this the year she actually keeps her resolution about kicking her daily Diet Coke habit.

Have you ever considered the origins of marketing research? Recently I’ve been pondering this. Some professions, such as construction, have been in existence since the dawn of civilization, meeting the basic human need of shelter. The (relatively) recent rise of the computer programmer marks its starting point in the early 1980s with the advent of the personal computer. But what about market research?I did some digging in order to answer my question, which led me to a book entitled A New Brand of Business: Charles Coolidge Parlin, Curtis Publishing Company, and the Origins of Market Research by Douglas Ward. This book focuses on Charles Coolidge Parlin (1872-1942), who is recognized today as the “Father of Marketing Research.” Parlin worked for Curtis Publishing Company, which was one of the most successful and influential American publishing companies of the early 20th century.

The pivotal moment for creating formalized marketing research was when Curtis Publishing made a principle-driven choice to ban medical, cosmetic, financial, and cigarette advertisements—and thus their accompanying revenue—from its magazines. To make up for this lost revenue, the company adopted a smarter business approach that focused only on the company’s existing clients, which would allow Curtis Publishing to become experts in its clients’ businesses. This novel idea went as follows: if Curtis Publishing could better serve its clients, those clients could in turn benefit Curtis Publishing with increased advertising revenue. In order to do this, the company sought to learn as much as possible about each client’s profit margins, territories, possibilities for expansion, and competition. In short, Curtis Publishing needed a clear view of each client’s marketplace.

With this impetus, Curtis Publishing created the Division of Commercial Research (1911) right here in Boston in what was formally known as Pemberton Square and is now known as Government Center. This was the first marketing research organization in the United States. The company had the notion to move forward on logical and statistical rule rather than intuition, and it strived to gauge public sentiment, evaluate changes in consumer tastes, and turn consumer wants into corporate profits. This newly founded “market research” would eventually become "the rudder on the ship of modern corporate capitalism.”

Parlin’s studies at Curtis Publishing led him to remarkable conclusions that were not readily apparent otherwise. For instance, Parlin calculated the strong influence that women had over family automobile purchases and foresaw that the automobile industry needed to reduce the number of models offered. Insights such as these eventually led to increased—and smarter—advertising as companies attempted to stay ahead of the curve.          

Interestingly, marketing research has the same purpose today as it did back then—it provides a way to improve marketing and business decision making. Parlin’s studies were typically hundreds of pages long with hand drawn charts, maps, and graphs bound in black or red leather with gold embossed lettering, and while we might do things a little differently now, we still need to create an informative narrative backed with charts and graphs aimed at getting to the heart of business decision making. Ever increasing amounts of information are available today, but distilling the most interesting and the most useful facts remains the ultimate challenge. I think Charles Parlin would agree, don’t you?

Matt Skobe is a Senior Data Manager at CMB. His passions include spending time with his wife and kids and mountain biking (day and night).

Want to help us craft the future of market research? Join our team!

In a few weeks I’ll be taking a cruise to the Caribbean—a cruise that I have spent 9 months planning. Needless to say, I’ve been a little preoccupied making sure everything is in place to ensure a flawless vacation. And as I sorted through all of these details, I couldn’t help but notice the similarities between vacation planning and how we at CMB prepare for a smooth, successful research project. You might be thinking “this is a woman who really needs a vacation.” But hear me out.

The first step of vacation planning is to put together a list of possible locations for a trip and select an appropriate timeframe. Planning a successful research study works on the same principles, every project starts with taking the time to define and understand the main decisions that need to be made from the research—we use tools like our Business Decision Worksheet—which directly ties the questionnaire, analysis and reporting to the business decisions, letting us identify and gain consensus on the most pressing decisions, and ensuring the results are actionable.

We also know how critical it is to develop (and stick to) a schedule that aligns with our clients’ needs. One of the first things that we at CMB do at the beginning of each project is put together a schedule outlining each key milestone of the process, all the way up to delivery of the final results. Putting together a detailed schedule helps us align resources and ensure we stay on track to meet our client deadlines. Knowing how much our clients rely on our research makes the scheduling a crucial part of the process and an important key to our success in executing projects.

Once the schedule is set, the project kicks off and the exploratory phase begins. I personally did lots of exploratory research before selecting my specific cruise line, ship, and date. Through this exploratory research, I was able to drill down and identify what aspects were most important in making my decision. Exploratory phases are also crucial for determining what will be most important to measure in the questionnaire and which areas are “nice to haves,” but not necessary to be included for the project.

Exploratory research also helps generate new ideas that may not have been previously considered. Similar to the many resources available for cruise planning (cruise line website, message boards, etc.), exploratory research for a project can span several platforms, including a review of secondary research, conducting in-depth interviews or focus groups, or hosting online discussion boards.

Sometimes the exploratory phase of a project gets less attention/recognition than is deserved because it doesn’t come across as being as “glamorous” as the analysis and insights that will come from the quantitative research. However, all market researchers know that the level of planning can make or break a project. CMB’s focus on planning allows us to try and anticipate what potential issues may come up down the road so that we can troubleshoot effectively and properly set expectations with our clients. Of course just like you can’t predict a rogue wave, there are times when the unexpected happens. When this happens we know we need to remain flexible enough to make course corrections and steer us back to the business decisions that our clients are trying to make.

I know we can only take the analogy so far; when all is said and done, often the only tangible evidence of having been on a vacation are the pictures. While the deliverables we produce for our clients are polished and shiny, they’re hardly the end “goal” of the research. Successful research is useful and used, and that starts well before a questionnaire is designed.

Cara is a Research Manager at CMB. She enjoys spending time with her husband Brett, her dog Nala, and planning her next vacation.

As a recent graduate, and entrant into the world of professional market research, I have some words of wisdom for college seniors looking for a career in the industry. You may think your professors prepared you for the “real world” of market research, but there are some things you didn’t learn in your Marketing Research class. So what’s the major difference between research at the undergrad level and the work of a market researcher? In the real world, context matters, and there are real consequences to our research. One example of this is how we approach testing for statistical significance.Starting in my freshman year of college, I was taught to abide by a concept that I came to think of as the “Golden Rule of Research.” According to this rule, if you can’t be 95% or 90% confident that a difference is statistically significant, you should consider it essentially meaningless.

Entering the world of Market Research, I quickly found that this rule doesn’t always hold when the research is meant to help users make real business decisions. Although significance testing can be a helpful tool in interpreting results, ignoring a substantial difference simply because it does not cross the thin line into statistical significance can be a real mistake.

Our Chief Methodologist, Richard Schreuer, gives this example of why this “Golden Rule” doesn’t always make sense in the real world:

Imagine a manager gets the results of a concept test in which a new ad outperforms the old by a score of 54% to 47%; sig testing shows our manager can be 84% confident the new ad will do better than the old ad. The problem in the market research industry is that we typically assess significance at the 95% or 90% level, if the difference between scores doesn’t pass this strict threshold, then it is often assumed no difference exists.

However, in this case, we can be very sure that the new ad is not worse than the old (there’s only a 1% chance that the new ad’s score is below the old). So, the manager has an 84% chance of improving her advertising and a 1% chance of hurting it if she changes to the new creative—pretty good odds. The worst scenario is that the new creative will perform the same as the old. So, in this case, there is real upside in going with the new creative and little downside (save the production expense). But if the manager relied on industry-standard significance testing, she would likely have dismissed the creative immediately.

At CMB, it doesn’t take long to get the sense that there is something much bigger going on here than just number crunching. Creating useable, meaningful research and telling a cohesive story require more than just an understanding of the numbers themselves; it takes creativity and a solid grasp on our clients’ businesses and their needs. As much as I love working with the data, the most satisfying part of my job is seeing how our research and recommendations support real decisions that our clients make every day, and that’s not something I ever could have learned in school.

Amy is a recent graduate from Boston College, where she realized that she had a much greater interest in statistics than the average student. She is 95% confident that this is a meaningful difference.

Join CMB' Amy Modini on February 20th, at 12:30 pm ET, to learn how we use discrete choice to better position your brand in a complex changing market. Register here.

A lot of time and money is wasted on segmentation studies. Here’s why, and what to do about it.

Last November I partnered with a banking client for a conference presentation on a segmentation study we conducted to help guide his organization towards greater customer-centricity. The study provided market insight to help transition from a product-based to a customer-centric organization by identifying need, attitude, and behavior-based segments.  The results helped them develop value propositions customized for each segment, which addressed products, messaging and customer experiences. 

The study was a great success. It’s used by our client in many ways, and was “actionable” in every sense of the word.  But rather than dwelling on our very great success, it got me thinking about why segmentation studies are often not acted upon.  In my 25 years of market research experience, I have found that segmentation studies are often found “interesting” but not “actionable.”  And it’s often not a function of the quality of research.  Poorly executed studies are never actionable.  But even well executed studies may not be actionable.  (And, by the way, when a client finds a study “interesting,” for me, that’s code for “I don’t want to hurt your feelings, but you failed.”)

Back to the conference presentation…at the start of our talk I asked the audience how many had worked on  well-executed segmentation studies (either as a supplier or a client) that were ultimately deemed “not useful.”  I knew the situation was bad, but I was shocked when about four-fifths of the audience raised their hand. So, here are a number of things we at CMB have learned over years about how to make segmentation actionable.  Note they don’t have anything to do with the mechanics of execution.

1. It’s the process, stupid (apologies to James Carville)
   While any good market research firm can write a decent questionnaire, structure a sound sample, and use state-of-the art analysis techniques, it’s the process that usually determines the project’s fate.  Simply soliciting client input, executing the study and presenting results is not enough.  The study will be a success if the process involves making information-users partners by capturing their definition of success, upcoming decisions and hypotheses, and then including these partners in selection the final segmentation solution.

2. Articulate and agree on business decisions
   Our experience shows that while, many research consumers are good at listing information needs, few actually identify the decisions they intend to make with this information.  Most seem to believe that if they have enough information they will find insights to help make as yet undetermined decisions.  This problem is especially acute in segmentation studies, because different types of decisions (product development vs. messaging vs. targeting) require different type questions and measurement techniques.

3. Many options, but no silver bullet
   Over many years and many studies I have never had an engagement where one segmentation solution worked equally well for all decisions.  For example, solutions that are stronger for targeting will typically be weaker for messaging.   At CMB, our process involves examining and rejecting up to 50 solutions, and then presenting four or five really good ones to our client. This is where management art blends with science.  By understanding competing decisions at the start, we make rational tradeoffs to select the best solution.

4. Real work begins when the study ends
   A segmentation study is typically treated as a discrete project with a beginning and end date.  If the final presentation is well-received the supplier and client may have celebratory drinks or dinner, if not the supplier quietly slinks off to the airport.  But the reality is that no matter how positive the initial reaction, segmentation studies can die on the vine if planning for implementation doesn’t occur before the final presentation.  In successful segmentation engagements, the final presentation is not “the end,” but rather “the end of the beginning.”  Segmentation often requires managers to think differently about the market, and this can’t occur without a process to support and reinforce this way of thinking.  We typically use a set of cross-functional workshops in which participants work with the information and participate in exercises to develop plans with input and support from the group

If you can internalize and act on these principles you’ll never have to slink back to airport after a final presentation. 

Rich is Senior VP and Chief Methodologist at CMB, he also knows the secrets of raising chickens, and the lost art of ski ballet.

You didn’t think we’d give away all our secrets did you? Join us this Wednesday the 27^th at noon to learn more secrets to successful segmentation.