Why AI Costs Aren't Unpredictable, They're Non-Deterministic | Yarken
AI Economics
Why AI Costs Aren’t Unpredictable, They’re Non-Deterministic
Run the same AI agent on the same task twice, and the bill can swing by up to 30x.
That is not a forecasting problem. It is not a prompt engineering problem. It is a fundamentally different kind of cost, and most finance and IT teams are still trying to manage it with tools built for something else entirely.
If your AI spend keeps surprising you even after you have tightened prompts, capped context windows, and switched to cheaper models, this post explains why, and what the teams who have actually gotten ahead of it are doing instead.
What does “non-deterministic cost” mean?
Definition
Non-deterministic cost is spend that changes each time a workload runs, even when the input, the model, and the task are identical. Unlike a cloud server with a fixed hourly rate, an AI agent’s token consumption depends on the reasoning path it takes that run, so the same request can cost 20,000 tokens one time and 2 million the next.
That distinction matters more than it sounds. “Unpredictable” implies you just need better data to forecast it. Non-deterministic means the outcome genuinely varies by nature of how the system works, no matter how good your data gets.
Why the same agent can cost 30x more on a rerun
An AI agent does not follow a fixed script. Each run, it decides how deep to search, how many steps to take, and when it has enough information to answer. Small differences in the reasoning path, one extra tool call, one longer chain of thought, compound fast.
The models generating that spend cannot even predict it themselves. Ask a model to estimate its own token usage before it starts, and it consistently underestimates. That is worth sitting with: the system creating the cost cannot reliably tell you what the cost will be. Enterprise buyers comparing agents head to head report variance climbing as high as 300x between different models on comparable tasks. Same task, same intent, wildly different bills depending on which model does the reasoning.
Why traditional FinOps was not built for this
Traditional cloud FinOps assumes deterministic, allocated resources. A virtual machine costs what it costs per hour. A storage tier has a known rate. You forecast by multiplying usage by price, and the price does not move on its own.
AI spend breaks that model at the root. The “resource” is a reasoning process, and reasoning processes are probabilistic by design. AI FinOps has to manage cost that moves every time the workload runs, which means the old playbook of rate negotiation and usage caps only gets you partway there. You need a different lever, and it is not a cheaper model.
The fix is not a cheaper model, it’s a smaller reasoning surface
The teams making real progress are not chasing lower per-token rates or squeezing prompts tighter. They are restructuring the work itself, splitting it into two categories:
- Judgment work: ambiguous, high-variance tasks that genuinely need an AI agent to reason through them.
- Repeatable work: known steps, stable logic, the same decision every time, that do not need to be re-reasoned through on every run.
Routing the second category into governed, deterministic automation instead of an open-ended agentic loop does two things at once. It cuts the unnecessary reasoning cycles that drive the worst cost spikes, and it shrinks what is left down to spend you can actually forecast, because non-deterministic cost concentrated in a smaller, well-scoped surface is far easier to model than the same variance spread across an entire workflow.
What the visibility gap is actually costing enterprises
This is not a theoretical problem. KPMG’s Global AI Pulse survey of more than 2,100 senior leaders found only 35% of organizations have full, actively monitored visibility into their AI operating costs. Organizations with that full visibility reported established ROI at roughly five times the rate of those without it.
The other half of that finding is sharper. Nearly half of organizations, 49%, said they had scaled back, delayed, or paused AI agent deployments because costs began to outweigh the value they were generating. Not because the technology failed. Because nobody could see the spend clearly enough to trust it.
That is the real cost of treating non-deterministic spend like a deterministic budget line. Leaders do not pull back from AI because it does not work. They pull back because they cannot see where the money goes, and an unmeasured cost is an ungovernable one.
Where Yarken fits
Yarken’s approach starts from the same principle driving the teams ahead of this curve: govern the parts of the workflow that should behave the same way every time, so your budget goes toward the reasoning that actually earns it.
Bringing AI spend into a unified view alongside cloud, SaaS, and infrastructure means you are not managing a new cost category in a spreadsheet on the side. You are managing one dollar of technology spend, visible end to end, wherever it is generated.