WELCOME TO OUR BLOG!

The posts here represent the opinions of CMB employees and guests—not necessarily the company as a whole. 

Subscribe to Email Updates

BROWSE BY TAG

see all

Laura Dulude

Recent Posts

How Advanced Analytics Saved My Commute

Posted by Laura Dulude

Wed, Aug 22, 2018

commuter

I don’t like commuting. Most people don’t. If you analyzed the emotions that commuting evokes, you’d probably hear commuters say it made them: frustrated, tired, and bored. To be fair, my commute experience isn’t as bad as it could be: I take a ~20-minute ride into Boston on the Orange Line, plus some walking before and after.

Still, wanting to minimize my discomfort during my time on the train and because I am who I am, I tracked my morning commute for about 10 months. I logged the number of other people waiting on the platform, number of minutes until the next train, time spent on the train, delays announced, the weather, and several other factors I thought might be related to a negative experience.

Ultimately, I decided the most frustrating part about my commute is how crowded the train is—the less crowded I am, the happier I feel. So, I decided to predict my subjective crowd rating for each day using other variables in my commuting dataset.

In this example, I’ve used a TreeNet analysis. TreeNet is the type of driver modeling we do most often at CMB because it’s flexible, allows you to include categorical predictors without creating dummy variables, handles missing data without much pre-processing, resists outliers, and does better with correlated independent variables than other techniques do.

TreeNet scores are shown in comparison to each other. The most important input will always be 100, and every other independent variable is scaled relative to that top variable. So, as you see in Figure 1, the time I board the train and the day of the week are about half as important as the number of people on the platform when I board. That means that as it turns out, I probably can’t do all that much to affect my commute, but I can at least know when it’ll be particularly unpleasant.

Importance to Crowding_commuter

What this importance chart doesn’t tell you is the relationship each item has to the dependent variable. For example, which weekdays have lower vs. higher crowding? Per-variable charts give us more information:

Weekday and Crowding_commuter

Figure 2 indicates that crowding lessens as the week goes on. Perhaps people are switching to ride-sharing services or working from home those days.

For continuous variables, like boarding time, we can explore the relationships through line charts:

Boarding Time and Crowding_commuter

Looks like I should get up on the earlier side if I want to have the best commuting experience! Need to tackle a thornier issue than your morning commute? Our Advanced Analytics team is the best in the business—contact us and let’s talk about how we can help!

 Laura Dulude is a data nerd and a grumpy commuter who just wants to get to work.

Topics: advanced analytics, EMPACT, emotional measurement, data visualization

I, for one, welcome our new robot...partners

Posted by Laura Dulude

Tue, Oct 17, 2017

 

iStock-841217582.jpg

Ask a market researcher why they chose their career, and you won't hear them talk about prepping sample files, cleaning data, creating tables, and transferring those tables into a report. These tasks are all important parts of creating accurate and compelling deliverables, but the real value and fun is deriving insights, finding the story, and connecting that story to meaningful decisions.

So, what’s a researcher with a ton of data and not a lot of time to do? Hello, automation!

Automation is awesome.

There are a ton of examples of automation in market research, but for these purposes I'll keep it simple. As a data manager at CMB, part of my job is to proofread banner tables and reports, ensuring that the custom deliverables we provide to clients are 100% correct and consistent. I love digging through data, but let’s be honest, proofing isn’t the most exciting part of my role. Worse than a little monotony is that proofing done by a human is prone to human error.

To save time and avoid error, I use Excel formulas to compare two data lists and automatically flag any inaccuracies. This is much more accurate and quicker than checking lists against one another manually—it also means less eye strain.

As I said, this is a really simple example of automation, but even this use case is an incredible way to increase efficiency so I have more time to focus on finding meaning in the data.

Other examples include:

  • Reformatting tables for easier report population using Excel formulas
  • Creating Excel macros using VBA
  • SPSS loops and macros

I’m a huge proponent of automation, whether in the examples above or in myriad more complex scenarios. Automation helps us cut out inefficiencies and gives us time to focus on the cool stuff

Automation without human oversight? Not awesome.

Okay, so my proofreading example is quite basic because it doesn’t account for:

  • Correctness of labels
  • Ensuring all response options in a question are being reported on
  • Noting any reporting thresholds (e.g. only show items above 5%, only show items where this segment is significantly higher than 3+ other segments, etc.)
  • Visual consistency of the tables or report
  • Other details that come together to create a truly beautiful, accurate, and informative deliverable.

Some of the bullet points above can also be automated (e.g. thresholds for reporting and correctness of labels), but others can’t. On top of that, automation is also prone to human error—we can automate incorrectly by misaligning the data points or filtering and/or weighting the data incorrectly. Therefore, it’s imperative that, even after I automate, I review to catch any errors—flawless proofing requires a human touch.

When harnessed correctly, automation maximizes efficiency, alleviates tediousness, and reduces error to free up more time for insights. Before you start arming yourself against a robot takeover, remember: insights are an art and a science, and machines haven’t taken over the world just yet.

Topics: Market research Automation,, Artificial Intelligence, quantitative research