Artificial intelligence is becoming more capable of predicting customer behaviour, but accurate results do not always translate into profits. New judgement models such as Jev promise to help businesses make predictions without training customised AI systems. However, experts warn that companies focusing on technical accuracy rather than financial returns risk missing the technology’s actual value.
Accuracy does not guarantee profits
Forbes predictive AI expert Eric Siegel argued that businesses often measure the wrong indicators when evaluating AI systems. Metrics such as accuracy and AUC may demonstrate technical performance without revealing whether a model makes money.
Consider a system designed to identify customers planning to cancel their subscriptions. When only 20% of messages indicate cancellation intentions, predicting that nobody will leave produces 80% accuracy. Despite that impressive figure, the system fails to identify a single at-risk customer.
Jev offers another approach through zero-shot outcome prediction, removing the need to train a dedicated predictive model. Its creator recently announced $870 million in funding at a $7.5 billion valuation, claiming that 29.4% of Fortune 500 companies already use the technology.
Finding the profitable balance
Siegel recommends using profit curves to determine which AI predictions justify business action. His example involves offering customers a $250 subscription discount to prevent cancellations, potentially generating $700 in net future earnings for each successful retention. Based on a synthetic dataset scaled to 500,000 messages, targeting the riskiest 10% could generate approximately $4.5 million. Expanding offers to 17% raises estimated profits to $5.2 million, while reaching 46% brings returns close to zero.
Changing the assumed retention benefit to $800 produces different results. Under that scenario, targeting nearly 30% of customers could generate $9.3 million. These figures are illustrative projections rather than verified business earnings. Ultimately, Siegel argues that companies should evaluate deployment costs, financial benefits and uncertainty before deciding how to use predictive AI.
