Hi Dr. Jay –
The city of Boston is trying develop one key measure to help officials track and report how well the city is doing. We’d like to do that in house. How would we go about it?
-Olivia
Hi Olivia,
This is the perfect tie in for big data and the key performance index (KPI). Senior management doesn’t really have time to pour through tables of numbers to see how things are going. What they want is a nice barometer that can be used to summarize overall performance. So, how might one take data from each business unit and aggregate them into a composite score?
We begin the process by understanding all the measures we have. Once we have assembled all of the potential inputs to our key measure, we need to develop a weighting system to aggregate them into one measure. This is often the challenge when working with internal data. We need some key business metric to use as the dependent variable, and these data are often missing in the database.
For example, I might have sales by product by customer and maybe even total revenue. Companies often assume that the top revenue clients are the bread and butter for the company. But what if your number one account uses way more corporate resources than any other account? If you’re one of the lucky service companies, you probably charge hours to specific accounts and can easily determine the total cost of servicing each client. If you sell a tangible product, that may be more challenging. Instead of sales by product or total revenue, your business decision metric should be the total cost of doing business with the client or the net profit for each client. It’s unlikely that you capture this data, so let’s figure out how to compute it. Gross profit is easy (net sales – cost of goods sold), but what about other costs like sales calls, customer service calls, and product returns? Look at other internal databases and pull information on how many times your sales reps visited in person or called over the phone, and get an average cost for each of these activities. Then, you can subtract those costs from the gross profit number. Okay, that was an easy one.
Let’s look at the city of Boston case for a little more challenging exercise. What types of information is the city using? According to the article you referenced, the city hopes to “corral their data on issues like crime, housing for veterans and Wi-Fi availability and turn them into a single numerical score intended to reflect the city’s overall performance.” So, how do you do that? Let’s consider that some of these things have both income and expense implications. For example, as crime rates go up, the attractiveness of the city drops and it loses residents (income and property tax revenues drop). Adding to the lost revenue, the city has the added cost of providing public safety services. If you add up the net gains/losses from each measure, you would have a possible weighting matrix to aggregate all of the measures into a single score. This allows the mayor to quickly assess changes in how well the city is doing on an ongoing basis. The weights can be used by the resource planners to assess where future investments will offer the greatest pay back.
Dr. Jay is fascinated by all things data. Your data, our data, he doesn’t care what the source. The more data, the happier he is.
After nearly a decade working on highly complex and strategic research projects, I’ve learned the one thing you can count on when dealing with massive amounts of data is Murphy’s Law—anything that can go wrong, will go wrong. No matter how much planning we do (and we take planning very seriously), the nature of market research means there’s bound to be a hiccup or two along the way.One of the best ways to deal with Murphy's Law is to accept that issues will arise but to make sure they don’t get in the way of the end goal—actionable insights. At CMB, our ability to seamlessly execute projects hinges on our capacity to adjust and course correct (when needed) to keep things on track. We put a lot of preparation and time in putting together solid project plans, focusing on business decisions, and conducting stakeholder interviews, but we also place a lot of emphasis on hiring and training strong problem solvers. We do this because we know that even the best laid out plans can still go awry, which is why it's important to manage problems proactively. For example, CMB firmly believes in conducting stakeholder interviews at the beginning of nearly all research engagements. This allows us to proactively re-shape/re-think the questionnaire design based on the information we’re hearing from the stakeholders. This helps prevent getting to the final presentation and delivering insights that are not relevant or useable for the key stakeholders.
