AI’s Prediction Problem

AI developments often focus on new capabilities to “detect and predict.” Progress on prediction is certainly impressive. Even for two things we can be sure of (death and taxes), advances in AI can help us know better when these certain events will happen, how much they will cost , and when people are trying to avoid them.

In business, AI has considerable potential as a crystal ball to predict. Already, AI is dramatically improving the abilities and reducing the costs of prediction across many functional areas — for example, supply chain disruptions, job candidate potentials, likely customer purchases, and nefarious computer activity.

Yes, the idea of seeing into the future is exciting. But, AI, you had me at “detect.” Despite the appeal of prediction, business can get ample value just from improved detection. Prediction can follow later.

The Importance of Smoke-Detector Predictability

Prediction is more uncertain, difficult, and expensive than detection. And most businesses (to massively understate) are not so well-run that better detection and faster reaction alone aren’t hugely valuable. Although the value may be less for detection than prediction, the ROI can be greater.

Changing perspective from crystal balls to smoke detectors can help the serious data-driven manager in several ways:

Smoke detectors encourage action. Smoke detectors provide early signals of what is already happening, not what might happen. Smoke detectors don’t predict fires; they alert us immediately when there is one. We have existing processes for fire: escape or extinguish. Similarly, businesses can still benefit from detecting issues quickly, even if they are unpredicted. The opportunity to prevent may have passed, but managing existing business processes can start sooner. And just like extinguishing is easier when smoke detectors alert quickly, managers have more options for better outcomes if they have more time.

Smoke detectors make sense. We don’t know how crystal balls work. Most managers don’t know how AI works either. AI has hidden layers that make it difficult for managers to know why AI delivers a given prediction. “Knowing” an outcome without understanding the underlying reasoning makes it harder to trust the results. Blind acceptance is a romantic, not a rational, approach.

In contrast, smoke detectors make sense. Even without a detailed understanding, people know they need to be centrally located with access to airflow — no one would expect a smoke detector in a box in a closet to do much detecting. By similarly thinking about how AI could provide early detection of business issues, managers can then naturally think about the data that could inform the detection. Where are data signals currently missing? Where are data signals low quality? Where are data signals giving false alarms? These questions naturally lead to practical steps to improve the data that fans the flames of modern AI.

Smoke detectors are themselves predictable. As managers recommend investment in solutions that involve AI, the investments are based on expected value. Expected value includes not only the amount but also uncertainty. Prediction is likely more uncertain than detection. It will take a lot of extra value from prediction to offset that uncertainty.

Worse, measurement problems may doom investment targeted at knowing the future from the outset. On the basis of prediction of the future, proactive management will act to change that future. When the predicted event doesn’t happen, is it because the prediction was incorrect or because the managerial action was successful?

For example, instead of trying to predict which customers will churn, managers can shift to better detect which customers are dissatisfied. The implications may be similar, but changes in satisfaction are measurable while customers who were going to leave but didn’t are not.

Prediction might be more valuable, but at a greater cost and uncertainty that can result in lower ROI.

Smoke detectors fit many places. Most businesses have many issues spread throughout the organization that could benefit from earlier detection. While people most likely don’t need more than one crystal ball, smoke detectors are useful in many places. If managers think about specific instances where they would like earlier detection, they will most likely find many different uses.

Value From AI Today

Yes, AI-enabled prediction is a fascinating and useful longer-term organizational goal. Companies are making considerable progress in prediction in many areas. But while the lure of having a crystal ball is certainly appealing, prediction is inherently difficult and may yield a low ROI. As a smoke detector, AI can provide real value for organizations today.