Better Decision Making with Objective Data is Impossible

In the popular 1950s and 1960s television show Dragnet, Sargent Joe Friday, played by Jack Webb, craved objective facts, famously asking witnesses to provide “just the facts.”

Managers today similarly crave facts. The potential positives of working from objective facts are enticing. It’s expected that improved performance follows from basing decisions on facts, whether in traditionally heuristics-based industries such as health care or in causally imprecise contexts such as business strategy.

But our world is awash in data, and data is not the same thing as facts. Facts are much harder to come by than data. While data seems to promise objectivity, instead it requires analysis — which is replete with subjective interpretation.

Assuredly, having data is a necessary step toward making objective decisions. Yet the objectivity of data is a myth. Modern analytical methods afford creative and flexible uses of data that can support multiple perspectives and competing analyses about the same data sets.

For instance, more data makes it easier to find support for virtually any position — because more data provides more options, limited only by the creativity of the analyst. Analysis could choose to focus on a subset that shows the “correct” results. Or, data that counters a desired position could be filtered out as “erroneous.” Sophisticated tools support many different modeling methods and options; one is bound to find the “right” answer. Just keep adding and dropping variables or observations until the known “truth” shines through.

Using data and analytics to support pre-existing beliefs is called “confirmation bias.” This is a particularly acute problem for modern analytics due to the potent combination of access to massive amounts of data, sophisticated methods and the seeming irreproachability of data-based decisions.

Confirmation bias can advance personal and political agendas or technical outcomes in ways that are difficult to detect. It can take the form of looking only for evidence that supports a desired outcome.

Alternatively, another way confirmation bias manifests itself is by having a preconceived idea about when to stop data analysis. If analysis stops because early results appear “appropriate,” there’s still a possibility that more analysis would find different results. For example: Jack Webb’s famous line on Dragnet, “Just the facts, ma’am,” was something he never actually said on the show (the character used a similar line, “All we want are the facts, ma’am.”). The idea here is that more information can show that what we “know” to be “true” is not factual.

Yet disconfirmation may be a better result than confirmation. Managers who are open to learning from analytics may find unexpected results to be more valuable than the expected. Findings that counter current thinking provide organizations with opportunity for distinction, differentiation and advantage.

Three roles can help organizations reduce the subjectivity and bias in analytical decision making: dictator, antagonist and goon. Despite these unflattering labels, organizations that have the talent and cultural wherewithal to develop this data triumvirate may get closer to learning “just the facts.”

Data Dictator

Different starting points create opportunity for data to be shaped to fit biases. A data dictator can unify frames of reference. At The Coca-Cola Company, one of their first steps was to get a common, shared understanding of the numbers behind each report so that time could be spent productively on managerial decisions rather than discussing from different reference points. Intermountain Healthcare developed a shared vocabulary, reducing ambiguity.

Data Antagonist

It can be difficult to espouse decisions that counter prevailing thinking. But this is necessary to avoid the trap of seeking (and finding) only confirmation. Warren Buffett, for example, encourages critics and contradictions. Analysts can be specifically charged to find evidence that contradicts the current algorithms that identify potential customers, failing machines or risky patients.

Data Goon

Relying on others for analytical skills reduces the ability for managers to understand how analysis has shaped the raw data. Managers need to find an analyst they can trust or, better yet, become that data goon themselves in order to participate.