When business goals backfire: How to adjust to unintended consequences
What are the values most important to a company? MIT's innovation expert Michael Schrage shares his thoughts on how to approach Key Performance Indicators.
Michael Schrage examines the various roles of models, prototypes, and simulations as collaborative media for innovation risk management. He has served as an advisor on innovation issues and investments to major firms, including Mars, Procter & Gamble, Google, Intel, BT, Siemens, NASDAQ, IBM, and Alcoa. In addition, Schrage has advised segments of the national security community on cyber conflict and cybersecurity issues. He has presented workshops on design experimentation and innovation risk for businesses, organizations, and executive education programs worldwide. Along with running summer workshops on future technologies for the Pentagon's Office of Net Assessment, he has served on the technical advisory committee of MIT's Lincoln Laboratory. In collaboration with the Center for Strategic and International Studies (CSIS), Schrage helped launch a series of workshops sponsored by the Department of Defense on federal complex systems procurement. In 2007, he served as a judge for the Industrial Designers Society of America's global International Design Excellence Awards.
Michael Schrage: There are several questions, essential questions that leaders and managers should ask themselves when they look at how they want KPIs to have an impact and influence on their organizations.
The most important one is obvious: What’s most important to the organization? Where is value really created? Where is value really created for us internally and for our clients, for our customers, for our users, whether they be business or consumers? How do we want to measure that? What kind of performance leads to those kinds of outcomes?
The big challenge, the difficult challenge, the challenge that my colleagues and I and my clients and I and my students and I debate and argue the most is: when are KPIs means, and when are KPIs ends?
Sometimes we want to have key performance because that’s what excellent means. We’re always greeting customers with a smile. We’re always trying to provide the easiest most convenient, the best possible user experience.
There are other times when the outcome is really what matters the most which is we get a profit on this. We get a sale for this.
What are means and ends?
Sometimes the KPIs link. Here’s how: “We’ve dramatically increased our sales.” That’s fantastic. Sales, sales KPI. “We’ve outperformed our sales KPI and expectation.” Oh, “three weeks later, three months later, the returns for what we’ve sold are larger than ever before; The customer satisfaction scores or the net promoter scores have dropped precipitously!” Oh my gosh, we have to manage tradeoffs between KPIs?
The leadership challenge, the management challenge is not just “how do we do a better job of identifying, cultivating, deploying KPIs,” it’s “how do we identify and manage the tensions and sometimes conflicts between them?”
Again, that’s why this is such a fun subject, because sometimes there can be changes where the KPIs converge or a new KPI emerges from your observation of means and ends.
So there are many stories and anecdotes that can be told about KPIs that really positively and constructively transform how an organization behaves. And there are also sadly (but obviously) examples of what one might call pathological KPIs, KPIs that make sense in the moment but when you review their ultimate impact internally and externally – bad, bad.
A classic example from 10-15 years ago, back when we had call centers instead of contact centers, was “time spent on calls.” That many call centers were compensated. The key performance indicator was “how do we keep the call down to two minutes, three minutes, five minutes?”
And so throughput calls per hour, calls per day were a dominant KPI because that’s what productivity was going to be. What did you end up with? High throughput, very unhappy customers, even oftentimes if their situation was resolved because the feedback was “I felt rushed. I felt they were interrupting a lot. It was a bad UX. It was a bad customer experience.”
There was also emerging at that time the notion of “first touch,” first call resolution. How can we get things resolved at that time, at the first call without making the person call back or without your calling them back?
Interestingly if I may digress, you know what really transformed that KPI was chat programs.
Because what people could do was they could do a chat and say “hey, I’m having this kind of problem. What should I do with this person?” And they were able to use their colleagues or a database or a Wiki to get an answer to do first call resolution. So this ties into the notion of not just a dominant KPI but how do we get a productive tension between KPIs?
One is “time spent on call.” The second is “effective resolution” and the third is “customer satisfied.”
You’re never going to get them all right but there’s going to be a sweet spot there. And in this case triangulating those KPIs to create a sweet spot was transformative for a lot of call centers that subsequently became contact centers.
So remember, if you pick the wrong dominant KPI you may end up with perverse outcomes because you are incentivizing perverse behaviors. If you’ll forgive me for making a bad acronymic pun, one of the ways you shock people into this recognition is that you say “KPI doesn’t just stand for ‘key performance indicator,’ it stands for ‘key performance incentive’.”
So make sure you’re recognizing and rewarding the right things, not things that can lead to perverse or counterproductive outcomes.
What are the values most important to a company? Can they be told by Key Performance Indicators (KPIs) alone or can the wrong incentive lead to the wrong outcome for a company? MIT's innovation expert Michael Schrage shares his thoughts on how to approach KPIs, using a classic example from the call center industry to show how dangerous an inexact business performance goal can be.
A pragmatic approach to fixing an imbalanced system.
- Intentional or not, certain inequalities are inherent in a digital economy that is structured and controlled by a few corporations that don't represent the interests or the demographics of the majority.
- While concern and anger are valid reactions to these inequalities, UCLA professor Ramesh Srinivasan also sees it as an opportunity to take action.
- Srinivasan says that the digital economy can be reshaped to benefit the 99 percent if we protect laborers in the gig economy, get independent journalists involved with the design of algorithmic news systems, support small businesses, and find ways that groups that have been historically discriminated against can be a part of these solutions.
Is there a way for more human-centered algorithms to prevent potentially triggering interactions on social media?
- According to a 2017 study, 71% of people reported feeling better (rediscovery of self and positive emotions) about 11 weeks after a breakup. But social media complicates this healing process.
- Even if you "unfriend", block, or unfollow, social media algorithms can create upsetting encounters with your ex-partner or reminders of the relationship that once was.
- Researchers at University of Colorado Boulder suggest that a "human-centered approach" to creating algorithms can help the system better understand the complex social interactions we have with people online and prevent potentially upsetting encounters.