Key takeaways
Inference, not training, is where the money goes. Industry estimates put inference at 70 to 90% of total AI compute spend over a model's production life, because training happens once and inference happens on every single call, forever.
Through 2028, at least 50% of GenAI projects will overrun their budgeted costs, and the root cause Gartner names is architectural: poor design choices and no operational visibility into how cost scales (Gartner).
The unit price is falling and the total is still rising. Inference on a trillion-parameter model is projected to cost providers over 90% less by 2030 than in 2025 (Gartner), while token consumption grows faster than the price drops.
The bill grows unexpectedly because nobody owns the run cost. Someone owned the build. No one was on the hook for what it costs to operate at scale.
The mistake: budgeting for the pilot, not the production bill
Inference is what your AI system costs every time it answers, and it is a recurring, per-call cost, not a one-time build cost. A pilot serves a handful of internal users a few hundred times a day. That bill is small enough to ignore, which is exactly the problem. The pilot number becomes the mental anchor, and then the system goes to production, usage multiplies, and the anchor was never real.
The mechanics are usually some mix of these:
A negligible per-call cost, multiplied by real production volume. A cost that rounds to zero for one user does not round to zero across ten thousand users and a hundred use cases.
Agentic workloads that call the model many times per task. One user request can trigger dozens of model calls behind the scenes, so cost scales with steps, not with users.
Retries, retrieval, and long context that nobody metered. Oversized prompts, redundant retrieval, and silent retry loops each multiply token count per request.
No visibility into which feature, team, or workflow is spending. The invoice arrives as one number, so there is nothing to optimize and no one to hold accountable. That vacancy is the subject of the companion guide, who owns your AI run cost.
The pilot proved it works. It did not prove it is affordable at scale, and those are different questions asked at different times.
The mistake: budgeting for the pilot, not the production bill
Inference is what your AI system costs every time it answers, and it is a recurring, per-call cost, not a one-time build cost. A pilot serves a handful of internal users a few hundred times a day. That bill is small enough to ignore, which is exactly the problem. The pilot number becomes the mental anchor, and then the system goes to production, usage multiplies, and the anchor was never real.
The mechanics are usually some mix of these:
A negligible per-call cost, multiplied by real production volume. A cost that rounds to zero for one user does not round to zero across ten thousand users and a hundred use cases.
Agentic workloads that call the model many times per task. One user request can trigger dozens of model calls behind the scenes, so cost scales with steps, not with users.
Retries, retrieval, and long context that nobody metered. Oversized prompts, redundant retrieval, and silent retry loops each multiply token count per request.
No visibility into which feature, team, or workflow is spending. The invoice arrives as one number, so there is nothing to optimize and no one to hold accountable. That vacancy is the subject of the companion guide, who owns your AI run cost.
The pilot proved it works. It did not prove it is affordable at scale, and those are different questions asked at different times.
Why does the bill grow even as prices fall?
The per-token price is collapsing fast. Gartner projects the cost of inference on a trillion-parameter model will drop more than 90% between 2025 and 2030. So why is your bill going up?
Because you are buying far more tokens than the price is dropping. Cheaper inference reads like a saving, but in practice it is permission to use more, and organizations take that permission faster than the price falls. Economists call this the Jevons paradox: make a resource cheaper and total consumption rises enough to more than absorb the saving. The falling unit price is the reason the total keeps climbing, not a reason to wait for it to fall on its own.
What does an unmanaged inference bill actually cost?
The cost of ignoring inference economics is not hypothetical. It shows up in recent, specific numbers, the kind that get left out of a pilot business case.
Most GenAI projects overrun, and the cause is architectural
Gartner's guidance is direct: through 2028, at least 50% of generative AI projects will overrun their budgeted costs, driven by poor architectural choices and a lack of operational know-how. Organizations underestimate operating expense because they have no visibility into how cost scales, so a project that looked viable in proof of concept becomes, in Gartner's words, a budget black hole in production.
So what? Half the AI initiatives being funded this year are on track to blow their number, not because the technology fails, but because no one modeled what it costs to run once real usage arrives. That is a planning failure, and planning failures are preventable.
Inference is the majority of the bill, and it never stops
Training a model is a one-time capital cost. Inference is the marginal cost of every answer the system gives, and it recurs for the life of the system. Industry estimates put inference at 70 to 90% of total AI compute spend for organizations that have shipped to production. The cost you spent months negotiating on the build is the smaller half. The half that grows is the one that arrives every month.
The point of governing inference is to price and control that recurring liability before it compounds, instead of meeting it on an invoice.
Token consumption is on track to rival payroll
The clearest signal of where this goes: Gartner predicts that AI coding token costs will surpass the average developer's salary by 2028 as token consumption surges. The anecdotes behind that forecast are already real, with individual developers reported consuming $20,000 of tokens in a single month and business users reaching $32,000. A cost that was a rounding error in the pilot becomes a line item that competes with headcount.
It has already happened at named-company scale. Uber exhausted its entire 2026 AI budget by April, four months into the year (Fortune), after agentic coding tools reached roughly 5,000 engineers at an average of $150 to $250 per engineer per month, with power users hitting $2,000. Adoption jumped from 32% of engineers to 84% in a single month, exactly the usage curve no pilot-anchored budget survives, and the company's answer was to cap employee AI spending. Meta imposed a similar internal token cap the same quarter.
So what? The variable you did not budget is growing toward the size of the salaries you budget carefully. It deserves the same discipline.
Agentic AI makes the cost curve steeper, and cancellations follow
The move to agentic systems, where the model plans and calls itself repeatedly to complete a task, multiplies calls per task and therefore cost per task. Gartner predicts over 40% of agentic AI projects will be canceled by the end of 2027, with escalating costs among the leading causes. Pricing a full task at production volume is also one of the seven screens in our guide to spotting agent-washing before you sign. The projects being killed are often the ones delivering value, killed anyway because the cost to run them was never owned.
The contrast: unmanaged inference vs. governed inference
Speed made it easy to ship an AI feature without pricing it. That is exactly why the run cost surprises you. The faster you can ship, the more it matters that someone owns what the thing costs to operate.
Budget anchored on the pilot's cost
Cost modeled at production volume before launch
One invoice, no idea what is driving it
Spend attributed by feature, team, and workflow
Biggest available model for every call
Right-sized model per task, cheapest that clears the bar
Long prompts, redundant retrieval, silent retries
Caching, prompt discipline, retry and loop governance
Agent free to call itself without limit
Step and cost budgets enforced per task
Someone owns the build
Someone owns the run cost, with a number attached
The right-hand column is not about using AI less. It is about knowing what each call costs and buying only the ones worth buying. The difference is where the cost lands: a modeled, attributed, controllable line item, or a monthly surprise nobody can explain.
What we see in the field
Across our AI-enablement and data pods, the inference bill is almost never high because the workload is high. It is high because the architecture was never designed for cost. The estimate the client walks in with was built on pilot usage, and the production system is making calls the pilot never made.
The first move is to instrument the spend and attribute it, so the invoice stops being one opaque number and starts pointing at a cause. The same few culprits turn up: a large share of token spend going to a handful of workflows, oversized models doing work a smaller model would do for a fraction of the cost, retrieval pulling far more context than the answer needs, and agent loops that retry without a limit. Fixing the architecture, not throttling usage, is what brings the number down, and it typically takes token spend down 40 to 60% while the system does the same work.
The number drops because the waste is gone, not because the AI does less. You stop paying for the calls that were never worth making.
Executive takeaway
Run the decision through the three things you translate everything into.
Modeling and instrumenting inference cost up front is days of work. The alternative is quarters spent explaining a bill that keeps growing, then re-architecting under pressure once finance flags it.
The unit price is falling and your total is still rising, so the lever is consumption and architecture, not waiting for cheaper tokens. Governed inference routinely cuts spend 40 to 60% for the same output, and it makes next month's number predictable instead of a guess.
This is the one that matters most at your level. An AI feature that gets canceled for cost after it shipped and worked is a reputational risk with your name on the sponsor line. Owning the run cost, with visibility and a number, is how you keep a working system funded instead of watching it get killed by its own invoice.
AI made inference cheap per call. It did not make the total predictable, and it did not put anyone on the hook for it. Accountability means knowing what the system costs to run before the invoice tells you.
Frequently asked questions
Why are my AI inference costs growing unexpectedly?
Because production usage looks nothing like the pilot. A per-call cost that rounds to zero for a few internal users multiplies across real volume, agentic workloads that make many calls per task, oversized prompts, redundant retrieval, and retries. The bill grows fastest where nobody has visibility into what is driving it.
Why does my AI bill go up when the price per token is falling?
Because consumption grows faster than the price drops. Gartner projects inference on a trillion-parameter model will cost over 90% less by 2030, but cheaper tokens invite far more usage, so the total keeps climbing even as each call gets cheaper.
Is inference or training the bigger cost?
Inference, by a wide margin, over a system's production life. Training is a one-time cost, while inference recurs on every call. Industry estimates put inference at roughly 70 to 90% of total AI compute spend for systems in production.
How do you reduce AI inference costs without cutting features?
Instrument spend so you can see what drives it, right-size the model to each task, cut wasted context and redundant retrieval, cache repeated calls, and put cost budgets on agent loops. The savings come from removing waste, not from using AI less, which is why governed inference commonly cuts spend 40 to 60% at the same output.
See where your AI run cost is exposed
If your AI bill is climbing and the invoice does not tell you why, a 30-minute inference-cost review gets you a whiteboard read on where the spend is concentrated and what a governed architecture would change. No obligation.
If you are earlier, weighing an AI or modernization initiative, the Cloud Migration Readiness Scorecard covers the cost and architecture checks to run before you commit, no form.
Knowing why the bill grows is half the problem. The other half is whose name is on it: Who Owns Your AI Run Cost?
AI made inference fast and cheap per call. NorthStar makes it accountable.
A vetted, dedicated pod that owns the outcome inside your systems, including what it costs to run, without the Big-4 bill or the offshore babysitting.
By NorthStar Group. 12+ years, 45+ clients, delivery you can hold someone to.
Sources
Gartner, "Gartner Predicts AI Coding Costs Will Surpass Average Developer's Salary by 2028 as Token Consumption Surges" (June 24, 2026): coding token costs to overtake average developer salary by 2028; reported spend of $20,000/month per developer and $32,000/month for some business users. gartner.com
Gartner, "10 Best Practices for Optimizing Generative and Agentic AI Costs": through 2028, at least 50% of GenAI projects will overrun their budgeted costs due to poor architectural choices and lack of operational know-how. (Summarized via Gartner cost-optimization guidance.)
Gartner, "Gartner Predicts That by 2030, Performing Inference on an LLM With 1 Trillion Parameters Will Cost GenAI Providers Over 90% Less Than in 2025" (March 25, 2026). gartner.com
Gartner, "Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027" (June 25, 2025): escalating costs among the leading causes. gartner.com
Fortune, "Uber burned through its entire 2026 AI budget in four months" (May 26, 2026): ~5,000 engineers, $150 to $250/month average per engineer, power users to $2,000, adoption 32% to 84% in one month. fortune.com Corroborated: TechCrunch (June 2, 2026). techcrunch.com
Meta internal AI token spending cap (2026): as reported by industry press with HN discussion; primary company confirmation not located. Directional; verify before quoting in a client-facing asset. mlq.ai
Inference share of AI compute spend (70 to 90% over a model's production life): industry-consensus estimate compiled from AI infrastructure and FinOps analyses (e.g., Spheron, Mirantis, Epoch AI). Directional; verify the specific figure before quoting in a client-facing asset.