WELCOME TO OUR BLOG!

The posts here represent the opinions of CMB employees and guests—not necessarily the company as a whole. 

Subscribe to Email Updates

BROWSE BY TAG

see all

Flying High on Predictive Analytics

Posted by Amy Maret

Thu, Jul 27, 2017

pexels-photo-297755_resized-1.jpgBuying a plane ticket can be a gamble. Right now, it might be a good price, but who’s to say it won’t drop in a day—a week? Not only that, it may be cheaper to take that Sunday night flight instead of Monday morning. And oh—should you fly into Long Beach or LAX? As a frequent traveler (for leisure and work!) and deal seeker, I face dilemmas like these a lot.

The good news is that there are loads of apps and websites to help passengers make informed travel decisions. But how? How can an app—say, Hopper—know exactly when a ticket price will hit its lowest point? Is it magic? Is there a psychic in the backroom predicting airline prices with her crystal ball?

Not quite.

While it seems like magic (especially when you do land that great deal), forecasting flight prices all comes down to predictive analytics—identifying patterns and trends in a vast amount of data. And for the travel industry in particular, there’s incredible opportunity to use data in this way. So, let’s put away the crystal ball (it won’t fit in your carry on) and look at how travel companies and data scientists are using the tremendous amount of travel data to make predictions like when airfare will hit its lowest point.

In order to predict what will happen in the future (in this case, how airfare may rise and fall), you need a lot of data on past behaviors. According to the Federal Aviation Administration (FAA), there are nearly 24,000 commercial flights carrying over two million passengers around the world every day. And for every single one of those travelers, there’s a record of when they purchased their ticket, how much they paid, what airline they’re flying, where they’re flying to/from, and when they’re traveling. That’s a ton of data to work with!

As a researcher, I get excited about the endless potential for how that amount of historical data can be used. And I’m not the only one. Companies like Kayak, Hopper, Skyscanner, and Hipmunk are finding ways to harness travel data to empower consumers to make informed travel decisions. To quote Hopper’s website: their data scientists have compiled data on trillions of flight prices over the years to help them make “insightful predictions that consistently perform with 95% accuracy”.

 While the details of Hopper are intentionally vague, we can assume that their team is using data mining and predictive analytics techniques to identify patterns in flights prices. Then, based on what they’ve learned from these patterns, they build algorithms that let customers know when the best time to purchase a ticket is—whether they should buy now or wait as prices continue to drop leading up to their travel date. They may not even realize it, but in a way those customers are making data-driven decisions, just like the ones we help our clients make every day.

As a Market Researcher, I’m all about leveraging data to make people’s lives easier. The travel industry’s use of predictive modeling is mutually beneficial—consumers find great deals while airlines enjoy steady sales. My inner globetrotter is constantly looking for ways to travel more often and more affordably, so as I continue to discover new tools that utilize the power of data analytics to find me the best deals, I’m realizing I might need some more vacation days to fit it all in!

So the next time you’re stressed out about booking your next vacation, just remember: sit back, relax, and enjoy the analytics.

Amy M. is a Project Manager at CMB who will continue to channel her inner predictive analyst to plan her next adventure.

Topics: big data, travel and hospitality research, predictive analytics

And the award goes to… Predictive Analytics!

Posted by Frances Whiting

Wed, Feb 22, 2017

Oscars-1.jpg

It doesn’t take a data scientist to predict some of what will happen at Sunday’s Oscars—beautiful people will wear expensive clothes, there will be a handful of bad jokes, a few awkward speeches, and most likely some tearful and touching ones as well. But in terms of the actual award outcomes, well, that takes a bit more analysis, and as quick search suggests, there’s no shortage of that online.   

These predictions come at an interesting time in the context of recent world events. In 2016 a few world events shook the predictive analytics world (and beyond) with outcomes so unexpected that even the most respected pollsters failed to predict them. And while many of the unanticipated polling outcomes occurred within politics (think Brexit and the U.S. presidential election), the implications for predictive analytics are also relevant to the market research industry.

As CMB’s president and co-founder, Anne Bailey Berman, recently said in Research Business Report’s Predictions issue, “ the market research industry will face many of the same questions regarding surveys and predictive analytics that are facing pollsters and data scientists in the aftermath of the election.”

Let’s bring it back to Sunday's Academy Awards. Since people love to predict the winners of awards like “Best Picture” and “Best Actress,” the awards show offers pollsters a chance to reflect on what went wrong in 2016 and to test refined predictive models in a much lower stakes context than a presidential election.

For example, popular polling site FiveThirtyEight has an ongoing tracker for Oscar winners. Typically, FiveThirtyEight bases its Oscar prediction model on the outcome of guild and press prizes that precede the Academy Awards. FiveThirtyEight watches who wins these other awards, like the Golden Globes or the Screen Actors Guild Awards, and then tries to figure out how much (how predictive) those awards matter.

First they look at historical data and pull all guild/press winners from the last 25 years, assuming these winners are representative of the Academy’s thinking. Based on the percentage of those awards that actually went to the corresponding Oscar, they assign a certain score for each award (e.g., if 17 of the last 25 winners for the Academy Award for best supporting actor also won the Globe, there’s a 68% correlation between the two).

Then they turn each award percentage into a score by squaring the original fraction and multiplying by 100. In doing this, weak scores get weaker and strong scores stay strong. FiveThirtyEight pollsters then consider other factors, like if the award is voted on by people who are also part of the Academy Award electorate or if the nominee loses. Both factors impact each prize’s “score”.

After reviewing FiveThirtyEight’s predictive modeling I've learned that even low-stakes polling for events like award shows depends on historical voting patterns and past outcomes. But is there danger in relying too much on historical data? If there’s one thing the 2016 US presidential election taught us it’s that predictive models can be susceptible to failure when they place too much weight on historic outcomes and trends. [twitter-129.pngTweet this]

The main problem with the predictive polls in 2016 was that they weren’t fully representative of the actual voting population. Unlike previous elections, there were A LOT of voters who turned out to cast their ballot on Election Day who predictive polls had missed throughout the campaign. Ultimately the polls failed to accurately predict the actions of these “anonymous voters,” perhaps in large part because they failed to account for the changing cultural, demographic, and economic social contexts impacting peoples’ decisions today. But that’s an exploration from another time. The point is, the 2016 predictive polls–based largely on historical trends–misrepresented the actual voting population.

Similar to the actual 2016 voting population, Academy members who vote on the Oscars are generally anonymous and can't be polled in advance of the event. This anonymity  forces pollsters to get creative and base their predictive models on a combination of historical guild and press prize outcomes. As market researchers and political pollsters know, even if voters are polled before the vote, there’s no guarantee they will actually act accordingly.  

This leaves us researchers with a serious conundrum: how can we get into anonymous respondents’ heads and predict their actual decisions/voting behaviors without relying too much on historical data?

One solution might be to emphasize behavioral datainformation gathered from consumers’ actual commercial behaviors–over their stated preferences and beliefs. For Oscar predictions, behavioral data might include:

  • Compiling social media mentions and search volume (via Google or Bing) for particular movies, actors, actresses, directors, etc.
  • Considering the number of social media followers nominees have and levels of online engagement
  • Tracking box office sales, movie downloads, and movie reviews

Based on the surprising outcome of the 2016 presidential election and Brexit, we learned that there was a huge cohort of unaccounted voters–voters who indeed turned out on voting day–that skewed traditional predictive models.

If pollsters hadn’t relied solely on historical data, and instead used an integrated approach that included current behavioral data, perhaps the predictions would have been more successful. There were plenty of voters on all sides who voiced their opinions on traditional and untraditional platforms, and capturing and accounting for those myriad of voices was a missed opportunity for pollsters.

Though the Oscars are a much lower stakes scenario, hopefully researchers continue to learn from 2016 and expand their modeling practices to include a combination of measures. Instead of a singular approach, researchers should consider combining historical trends and current behavioral data.

Interested in learning more about predictive analytics? Check out Dr. Jay’s recent blog post on trusting predictive models after the 2016 election.

 Frances Whiting is an Associate Researcher at CMB who is looking forward to watching the 89th Academy Awards and the opportunity to try her hand at predictive analytics!

Topics: television, predictive analytics, Election

Dear Dr. Jay: HOW can we trust predictive models after the 2016 election?

Posted by Dr. Jay Weiner

Thu, Jan 12, 2017

Dear Dr. Jay,

After the 2016 election, how will I ever be able to trust predictive models again?

Alyssa


Dear Alyssa,

Data Happens!

Whether we’re talking about political polling or market research, to build good models, we need good inputs. Or as the old saying goes: “garbage in, garbage out”.  Let’s look at all the sources of error in the data itself:DRJAY-9-2.png

  • First, we make it too easy for respondents to say “yes” and “no” and they try to help us by guessing what answer we want to hear. For example, we ask for purchase intent to a new product idea. The respondent often overstates the true likelihood of buying the product.
  • Second, we give respondents perfect information. We create 100% awareness when we show the respondent a new product concept.  In reality, we know we will never achieve 100% awareness in the market.  There are some folks who live under a rock and of course, the client will never really spend enough money on advertising to even get close.
  • Third, the sample frame may not be truly representative of the population we hope to project to. This is one of the key issues in political polling because the population is comprised of those who actually voted (not registered voters).  For models to be correct, we need to predict which voters will actually show up to the polls and how they voted.  The good news in market research is that the population is usually not a moving target.

Now, let’s consider the sources of error in building predictive models.  The first step in building a predictive model is to specify the model.  If you’re a purist, you begin with a hypotheses, collect the data, test the hypotheses and draw conclusions.  If we fail to reject the null hypotheses, we should formulate a new hypotheses and collect new data.  What do we actually do?  We mine the data until we get significant results.  Why?  Because data collection is expensive.  One possible outcome from continuing to mine the data looking for a better model is a model that is only good at predicting the data you have and not too accurate in predicting the results using new inputs. 

It is up to the analyst to decide what is statistically meaningful versus what is managerially meaningful.  There are a number of websites where you can find “interesting” relationships in data.  Some examples of spurious correlations include:

  • Divorce rate in Maine and the per capita consumption of margarine
  • Number of people who die by becoming entangled in their bedsheets and the total revenue of US ski resorts
  • Per capita consumption of mozzarella cheese (US) and the number of civil engineering doctorates awarded (US)

In short, you can build a model that’s accurate but still wouldn’t be of any use (or make any sense) to your client. And the fact is, there’s always a certain amount of error in any model we build—we could be wrong, just by chance.  Ultimately, it’s up to the analyst to understand not only the tools and inputs they’re using but the business (or political) context.

Dr. Jay loves designing really big, complex choice models.  With over 20 years of DCM experience, he’s never met a design challenge he couldn’t solve. 

PS – Have you registered for our webinar yet!? Join Dr. Erica Carranza as she explains why to change what consumers think of your brand, you must change their image of the people who use it.

What: The Key to Consumer-Centricity: Your Brand User Image

When: February 1, 2017 @ 1PM EST

Register Now!

 

 

Topics: methodology, data collection, Dear Dr. Jay, predictive analytics

Big Data Killed the Radio Star

Posted by Mark Doherty

Wed, Jun 29, 2016

It’s an amazing time to be a music fan (especially if you have all those Ticketmaster vouchers and a love of '90's music). While music production and distribution was once controlled by record label and radio station conglomerates, technology has “freed” it in almost every way. It’s 200542299-001_47.jpgnow easy to hear nearly any song ever recorded thanks to YouTube, iTunes, and a range of streaming sources. While these new options appear to be manna from heaven, for music lovers, they can  actually create more problems than you’d expect. The never-ending flow of music options can make it harder to decide what might be good or what to play next. In the old days (way back in 2010 :)), your music choices were limited by record companies and by radio station programmers. While these “corporate suits” may have prevented you from hearing that great underground indie band, they also “saved” you from thousands of options that you would probably hate. 

That same challenge is happening right now with marketers’ use of data. Back in the day (also around 2010), there was a limited number of data sets and sources to leverage in decisions relating to building/strengthening a brand. Now, that same marketer has access to a seemingly endless flow of data: from web analytics, third-party providers, primary research, and their own CRM systems. While most market information was previously collected and “curated” through the insights department, marketing managers are often now left to their own devices to sift through and determine how useful each set of data is to their business. And it’s not easy for a non-expert to do due diligence on each data source to establish its legitimacy and usefulness. As a result, many marketers are paralyzed by a firehose of data and/or end up trying to use lots of not-so-great data to make business decisions.

So, how do managers make use of all this data? It’s partly the same way streaming sources help music listeners decide what song to play next: predictive analytics. Predictive analytics is changing how companies use data to get, keep, and grow their most profitable customers. It helps managers “cut through the clutter” and analyze a wide range of data to make better decisions about the future of their business. It’s similarly being used in the music industry to help music lovers cut through the clutter of their myriad song choices to find their next favorite song. Pandora’s Musical Genome Project is doing just that by developing a recommendation algorithm that serves up choices based on the attributes of the music you have listened to in the past. Similarly, Spotify’s Discover Weekly playlist is a huge hit with music lovers, who appreciate Spotify’s assistance in identifying new songs they may love.

So, the next time you need to figure out how to best leverage the range of data you have—or find a new summer jam—consider predictive analytics.

Mark is a Vice President at CMB, he’s fully embracing his reputation around the office as the DJ of the Digital Age.

Did you miss our recent webinar on the power of Social Currency measurement to help brands activate the 7 levers that encourage consumers to advocate, engage, and gain real value? You're not out of luck:

 Watch Here

 

Topics: advanced analytics, big data, data integration, predictive analytics