WELCOME TO OUR BLOG!

The posts here represent the opinions of CMB employees and guests—not necessarily the company as a whole. 

Subscribe to Email Updates

BROWSE BY TAG

see all

Laurie McCarthy

Recent Posts

My Data Quality Obsession

Posted by Laurie McCarthy

Tue, Jan 12, 2016

3d_people_in_a_row.jpgYesterday I got at least 50 emails, and that doesn’t include what went to my spam folder—at least half of those went straight in the trash. So, I know what a challenge it is to get a potential respondent to even open an email that contains a questionnaire link. We’re always striving to discover and implement new ways to reach respondents and to keep them engaged: mobile optimization is key, but we also consider incentive levels and types, subject lines, and, of course, better ways to ask questions like highlighter exercises, sliding scales, interactive web simulations, and heat maps. This project customization also provides us with the flexibility needed to communicate with respondents in hard-to-reach groups.

Once we’ve got those precious respondents, the question remains: are we reaching the RIGHT respondents and keeping them engaged? How can we evaluate the data efficiently prior to any analysis?

Even with the increased methods in place to protect against “bad”/professional respondents, the data quality control process remains an important aspect of each project. We have set standards in place, starting in the programming phase—as well as during the final review of the data—to identify and eliminate “bad” respondents from the data prior to conducting any analysis.

We start from a conservative standpoint during programming, flagging respondents who fail any of the criteria in the list below. These respondents are not permanently removed from the data at this point, but they are categorized as an incomplete and are reviewable if we feel that they provide value to the study:

  • “Speedsters”Respondents who completed the questionnaire in 1/5 of the overall median time or less. This is applied to evaluate the data collected after approximately the first 20% or 100 completes, whichever is first.
  • “Grid Speedsters”:When applicable, respondents who, for two or more grids of ten or more items, has a grid speed less than 2 standard deviations from the mean for the grid. Again, this is applied after approximately the first 20% or 100 completes, whichever is first.
  • “Red-Herring”We incorporate a standard scale question (0-10), which is programmed at or around the estimated 10-minute mark in the questionnaire, asking the respondent to select a number on the scale. Respondents who do not select the appropriate number are flagged.

This process allows us to begin the data quality review during fielding, so that the blatantly “bad” respondents are removed prior to close of data collection.

However, our process extends to the final data as well.  After the fielding is complete, we review the data for the following:

  • Duplicate respondents: Even with unique links and passwords (for online), we review the data based on the email/phone number provided and the IP Address to remove respondents who do not appear to be unique.
  • Additional speedsters: Respondents who completed the questionnaire in a short amount of time. We take into consideration any brand/product rotation as well (evaluating one brand/product would take less time than evaluating several brands/products). 
  • Straight-liners: Similar to the grid speeders above, we review respondents who have selected only one value for each attribute in a grid. We flag respondents who have straight-lined each grid to create a sum of “straight-liners.” We review this metric on its own as well as in conjunction with overall completion time. The rationale being that if respondents are only selecting one value throughout the questionnaire and appear in the straight-lining flag, these individuals will also have sped through the questionnaire.
  • Inconsistent response patterns: In grids, we can sometimes have attributes that would use the reverse scale, and we review those to determine if there are contradictory responses. Another example might be a respondent who indicates he/she uses a specific brand, and, later in the study, the respondent indicates that he/she is not aware of that brand.

While we may not eliminate respondents, we do examine other factors for “common sense”:

  • Gibberish verbatims: Random letters/symbols or references that do not pertain to the study across each open ended response
  • Demographic review: Review of the demographic information to ensure that they are reasonable and in line with the specifications of the study

As part of our continuing partnership with panel sample providers, we provide them with the panel ID and information of those respondents who have failed our quality control process. In some instances, in which the client or the analysis require that certain sample sizes are collected, this may also necessitate replacing bad respondents. Our collaboration allows us to stand behind the quality of the respondents we provide for analysis and reporting, while also meeting the needs of our clients in a challenging environment.

Our clients rely on us to manage all aspects of data collection when we partner with them to develop a questionnaire, and our stringent data quality control process ensures that we can do that plus provide data that will support their business decisions. 

Laurie McCarthy is a Senior Data Manager at CMB. Though an avid fan of Excel formulas and solving data problems, she has never seen Star Wars. Live long and prosper.

We recently did a webinar on research we conducted in partnership with venture capital firm Foundation Capital This webinar will help you think about Millennials and their investing, including specific financial habits and the attitudinal drivers of their investing preferences.

Watch Here!

 

Topics: Chadwick Martin Bailey, methodology, data collection, quantitative research

The Building Blocks of Job Loyalty

Posted by Laurie McCarthy

Tue, Mar 03, 2015

loyalty, cmbAn old friend recently remarked how crazy it is that I’ve been at the same company for over 16 years (and I’m not even that old!). I hadn’t really thought it was a big deal, but among my peers, I’m an outlier. A recent article claims Millennials are putting a “time-limit” on their time at a company—91% expect to stay in a job for less than three years. Kids these days!Of course, economic, cultural, and social reasons all play into the shift away from long-term employee tenure. But when I think about why I’ve stayed at CMB, I keep coming back to the role loyalty plays. If employers want to keep and grow their most valued employees in an increasingly competitive job market, they need to take a lesson from the brand loyalty playbook. Here are 5 principles worth developing:

Value: 

If you want your smart, curious, and driven employees to stay, you need to offer opportunities to keep their skills fresh and relevant. When I was interviewing at CMB out of grad school, I was excited that analysts were exposed to all aspects of project work (sampling, questionnaire design, data collection, reporting, and analysis). Over the years, we’ve also developed the CMB University (CMBU) program, which includes weekly seminars on the latest techniques, case studies, and innovations. All this emphasis on staying competitive means I’m never bored and my skillset is constantly refreshed.

Pride: 

It doesn’t matter whether your company is a small firm, a global giant, or somewhere in between—employees who are proud of their work and their culture are going to stick around. Just this past year, CMB was named to the AMA Gold Top 50 US Market Research Organizations and the 2014 Top 100 Women-Led Business in Massachusetts by the Boston Globe. I take pride in knowing my contributions to the company (small and large) helped win these awards. Bottom line: as an organization grows, so does the degree to which an employee can thrive. 

Advocacy:

Word of mouth is an essential component to hiring and sustaining a good employee base, as well as for promoting a brand. When employees feel valued by an organization, they’re going to tell family and friends. The key is to leverage that advocacy to work in favor of the organization. At CMB, there is a referral program. Our employees are the face of our company, and our referral program encourages employees to reach out to peers as potential candidates and talk about their positive experiences at CMB.

Passion: 

79% of Millennials say it’s more important to genuinely enjoy their job than to make a lot of money. They want to love what they are doing and not just see it as work. I am a data dork and proud of it. Syntax, formulas, advanced design, efficiencies gained when brainstorming with colleagues—love ‘em all! My passion is digging into the data, wrapping my head around a problem, and troubleshooting . . . all of which come with my role as a data manager. 

But during my time at CMB, it’s not just the work I’ve enjoyed, it’s the sense of community. There’s a camaraderie in our work environment—we work hard together, and then we play together. Throughout my years here, I’ve been privileged to form friendships in addition to great working relationships. 

Trust:

Loyalty is definitely a two-way street—when an organization believes in you, it really inspires you to believe in the organization. As an employee, you have confidence that the company will do right by you and that you will earn both hard benefits (401K, health insurance, vacation, etc.) and soft benefits (flexibility of hours, virtual commuting, working from home when you are sick, etc.). Having a 401(k) is important, but retirement plans are offered at most companies in some form or another. Here’s what really makes me happy: not missing my child’s first grade holiday concert, and CMB makes sure I won’t.

So after almost two decades at the same organization, maybe I am an exception . . . but I wouldn’t have it any other way.

Laurie McCarthy is a Senior Data Manager, and she has been with CMB for over 16 years. She’s had 9 desks, sat on 4 floors, had 2 kids, and has gotten married once.

Want to join our team? Check out our current openings. 

Open Positions

Topics: Chadwick Martin Bailey, our people, customer experience and loyalty