Senior executives are often highly skeptical of predictive analytics. Some insist on exclusively trusting their own judgment, yet their “gut” often fails them—and their shareholders.
When initiating discussions with the business about the potential value of predictive analytics, CIOs may be met with resistance from business leaders who are wary of using data-driven methods to help make strategic business decisions. These skeptics have trouble imagining how a statistical process or mathematical model could possibly offer better answers to pressing business questions, such as where to open a store or which insurance risks will turn profits for the company, than they could, with their years of experience and refined business acumen. To make a case against predictive analytics, they may point to a high-profile incident where a computer made a one-off error that no competent human expert would ever make.
Notably, this sort of skepticism can correlate with seniority; the individuals who tend to be the most dubious of predictive analytics and other “evidence-based” management and decision-making techniques are often senior leaders in an organization. Their skepticism, coupled with their power and authority, can hinder the progress of predictive analytics projects. Convincing these leaders of the value of predictive analytics requires understanding their misconceptions about analytic solutions and their decision-making processes.
Making the Case for Predictive Analytics
It’s no wonder some executives remain unconvinced of the benefits of predictive analytics. The discipline is often wrapped in lofty promises about “predicting the future.” These vague claims sometimes lead to misconceptions about the goal of predictive analytics. For example, many people imagine that predictive analytics purports to somehow replace human judgment by building models that provide definitive answers.
In fact, the goal of analytics projects is generally less to uncover “Truth with a capital T” so much as to improve expert decision-making by converting raw data into insights, inferences, or predictive models that can aid operational processes. Professional judgment and domain knowledge are critical in this process and come into play at two points: First, they should be applied to frame, prioritize, and inform specific steps in the process of analyzing data to build predictive models. Second, no predictive model or decision rule is complete or infallible 100 percent of the time. Human judgment is required to decide when to use, temper, or simply ignore a model’s indications.
What’s more, the statistical models that underpin analytics solutions are not static. Analytics is best viewed as an iterative process where predictive models are continually refined and improved. The most effective analytics projects benefit from stakeholders who understand that both data-driven analytics and expert decision-making have strengths as well as limitations.
For the benefit of their businesses, CIOs should confront misconceptions about predictive analytics head-on and reset executives’ expectations about what predictive analytics realistically can do well. They’ll also have to play up the critical role that executives’ expert judgment should play in helping predictive models work their seeming magic. In short, CIOs should reframe the analytics debate from “Man vs. Machine” to “Man and Machine.”
Why Executives Are So Enamored of Their Experience
In domain after domain, many predictive models have been shown to be effective in helping human specialists—from doctors to baseball scouts to insurance underwriters to retailers—make decisions more consistently, correctly, and economically. One example comes from Cook County Hospital: A controlled experiment in the 1990s showed that a data-driven decision-rule protocol was markedly more accurate than the unaided judgment of physicians in determining which patients entering the emergency room complaining of chest pains should be sent to intensive care, intermediate care, or home.¹
Decades of academic research and business experience suggest that data-driven methods can help even highly trained domain experts make better decisions. This is not just because our databases are now so deep and rich or that we possess powerful analytical tools and techniques. It is also because we human beings are so surprisingly bad at weighing evidence, juggling probabilities, and making consistent, coherent decisions in the face of uncertainty.
Consider two types of mental processes people use to make decisions. The economics Nobel laureate Daniel Kahneman distinguishes between “Type 1” and “Type 2” mental processes. Type 1 mental processes are fairly automatic, effortless, and place a premium on “narrative coherence.” In contrast, “Type 2″ mental processes are controlled, effortful, and place a premium on logical coherence.²
Kahneman notes that although we fancy ourselves as primarily Type 2 creatures, most of our mental processes are in fact Type 1 in nature. It turns out that our Type 1 minds are very poor at statistical reasoning. We over-generalize from personal experience, exaggerate the likelihood of scenarios that vividly come to mind, inconsistently weigh evidence, and simply get tired. So far are we from being naturally statistical thinkers and rational decision-makers that Kahneman characterizes the human mind as “a machine for jumping to conclusions.”
The prevalence of Type 1 thinking helps to explain why senior executives can be so insistent on using their gut to make decisions, so confident in their judgments, and so skeptical of predictive analytics: Such executives have had the longest time to form a body of Type 1-style “narratively coherent” beliefs and decision rules pertaining to their domains.
Their degrees of confidence about which patient to treat, which insurance policy to write, which employee to hire, or which ad to run are generally not based on supporting statistical evidence. Rather, their confidence is based on the vividness and coherence in the narratives they construct to summarize their experiences and observations. And these narratives are often easier to communicate to employees and shareholders than statistical analyses of the evidence.
Perhaps another reason some executives trust their professional judgment over analytics is simply that they’ve had to rely solely on their instincts and domain experience until now. Decision-makers in many domains have had little or no access to data or statistical analyses to aid in their decision-making, yet they’ve managed to build successful careers. While understandable, this sentiment neglects the shifting competitive landscape due to the ascendance of data-driven decision-making.
Converting the Disbelievers
Given people’s misconceptions and the mental processes they use to make decisions, convincing a skeptical CEO of the value of analytics is no small task. Understanding why CEOs distrust analytics, how they make decisions, and why they place so much confidence in their professional judgment will help the CIO’s cause. CIOs may also be able to sway skeptical CEOs by sharing with them studies that show how human decision-making processes are inherently flawed and how predictive models can help counterbalance the cognitive biases that lead us to make bad decisions.
What’s more, not every decision today can be made based on past experience and observations. The business world is too complicated to make decisions based on gut feeling and professional judgment alone. Companies should consider shifting to more fact-based decision-making or risk being out maneuvered by their competitors who do.
—by James Guszcza, national predictive analytics lead in the Advanced Analytics & Modeling practice, Deloitte Consulting LLP and John Lucker, principal, Deloitte Consulting LLP, Deloitte’s global advanced analytics & modeling market offering leader, and a US leader in Deloitte Touche Tohmatsu Limited’s Deloitte Analytics Institute.