Earlier this month I had the chance to present at the Market Research Technology Event in Las Vegas. Beyond the fact I just could not get accustomed to watching people walk by conference rooms swigging beer and wearing in flip flops; for me the event raised more questions than provided answers.

During the conference, one of the most quoted reports was McKinsey’s: Big data: The next frontier for innovation, competition, and productivity.  For me, one of the most striking takeaways from the report was a prediction that by 2018, the US will have a shortage of talent necessary for organizations to take advantage of big data—the US alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis to make effective decisions.

After we market researchers take a moment to celebrate our job security, we should consider that skilled market researchers will be asked to fill the space by taking on more tasks and working longer hours.  As the gap widens between the influx of data and the analysts we need to make sense of it, are 80 hour weeks inevitable? Certainly workforce globalization will be a key to filling “big data” needs, but I was very surprised to hear little discussion of how technology will help us deal with this shortage.

I left the conference with the theory that the “new technology” we need is the yet-to-be-realized application of a tool to change a process to yield a quicker, lower cost, or better quality outcome.  I think market researchers have yet to focus on how technology can act as a surrogate for the role they play within their organizations

So what might the future hold? I expect technology will allow market researchers to develop “analytical bots” to make sense of the vast ocean of data to answer specific business questions raised by internal clients. Watching Watson and Siri answer questions of fact with extremely high accuracy makes me wonder what our role will be.  If these machines “have all the answers” then what purpose do we have?  I don’t believe technology will replace market researchers; their skillset and output are still critical for companies to be competitive.  The purpose is to create the rules and algorithms that convert the facts into relevant information.  This is where market research skills will combine with technology to fill the resource gap.

We’ve heard a lot about expert systems—computer systems that emulate human decision-making. It’s my view that the market researchers who will lead in the next 5 to 7 years will be those who are setting up and managing expert systems, that take all of the facts and computations within large sets of data and apply what is relevant, to make decisions quickly, anywhere, and at any time.

Did you miss us at TDMRE? We'll be at the Audience Measurement Event in Chicago from May 21st to the 23rd. Register for a 25% discount by entering CMB2012 here.

Posted by Jeff McKenna, Jeff is a senior consultant at CMB and team leader for Pinpoint Suite-our innovative Customer Experience Management software. Want to learn more about how Pinpoint Suite can help you make sense of your "Big Data," schedule a demo here.