Somewhere around month two of running agents in production, a bill shows up that nobody can explain. Not because anything is broken — because nothing is. The agents did exactly what they were built to do: loop, call tools, re-reason, retry. Every iteration re-sent a growing context window at list price, and the meter did what meters do.
I’ve written that the context window is a budget, not a bucket, and that autonomy itself should be a budget an agent spends. This post completes the set. Agentic systems run on three budgets — reliability, attention, and dollars — and the third one is the least instrumented, despite being the only one that arrives as an invoice.
Why agent economics broke your chatbot intuitions
The 2026 cost picture in three data points:
- Inference dominates. Roughly 80% of AI GPU spend is now inference, not training. Whatever you spent getting models is now dwarfed by what you spend running them. This is an operations bill, which makes it an SRE problem.
- Agents multiply tokens. Gartner’s 2026 analysis puts agentic workloads at 5–30× the tokens per task of a standard chatbot exchange. A task isn’t a request; it’s 10–20 model calls, each re-sending accumulated context.
- The growth is superlinear. Because each loop iteration carries the context of the previous ones, the cost of call n grows with n. Long agent runs are expensive precisely at the end — the retries and re-reasoning steps you added for reliability are your most expensive tokens.
That last point is the one that catches teams. The reliability patterns we reach for by instinct — retry on failure, add a validation pass, re-check the plan — all cost tokens, and they cost the most tokens exactly when they fire on a bloated context. Reliability and cost aren’t separate concerns in agentic systems; they’re the same loop, billed twice.
There’s also a structural asymmetry worth naming: always-on monitoring agents consume continuously, whether or not anything is wrong. A human on-call engineer idles cheaply. An agent watching your alerts is spending your budget every polling cycle — which is fine, as long as somebody decided that on purpose.
The wrong metric will make your bill look great
The instinctive response is a cost-per-token crusade: cheaper model, shorter prompts, celebrate the graph going down. The FinOps Foundation’s framing is the correction: cost-per-token is an input metric. Cost-per-successful-output is the unit economics.
The failure mode is concrete. Swap your triage agent to a model that’s 4× cheaper per token, and if its diagnosis acceptance rate drops from 90% to 60%, every rejected diagnosis triggers a retry loop, an escalation, or — worst — a human redoing the work with the agent’s wrong answer as anchor. Your cost-per-token improved. Your cost per resolved incident got worse, and it got worse in a column nobody was watching.
Computing the right metric has a prerequisite that’s pure engineering: cost telemetry and quality signals have to land in the same trace. If you followed the OTel GenAI conventions post, you already have the shape — gen_ai.client.token.usage per span, a quality score on the task’s root span. Divide one by the other, grouped by agent and task class. That query is your actual AI P&L. Until it exists, every optimization is a guess wearing a dashboard.
The lever stack, ranked by effort-to-impact
The good news: the levers are real, they compound, and combined they routinely cut managed-API spend 50–90% without touching model quality. The better news for this audience: they rank cleanly.
1. Prompt caching — take the free money first. Your agents re-send the same system prompt, tool schemas, and few-shot examples on every single call. Prompt caching discounts those repeated prefix tokens 50–90% depending on provider. For loop-heavy agents where the stable prefix dominates the context, this is the single cheapest win in the stack — usually a code change measured in hours. Watch your cache-read token metrics; a hit rate below ~70% on an agent workload means your prompt assembly is shuffling the prefix and burning the discount.
2. Model routing — stop paying frontier prices for routine steps. Inside one agent task, the plan step might need your best model while the extract-fields step needs something 20× cheaper. Route by step difficulty, with an escalation path when the small model’s confidence or validation fails. This is where cost-per-successful-output earns its keep: route aggressively where quality holds, and let the metric — not the price sheet — tell you where it doesn’t.
3. Context discipline — the compounding lever nobody funds. Every technique from context engineering — compress, select, isolate — is also a cost lever, because tokens are the unit of both attention and billing. Summarize tool outputs before appending them. Evict resolved threads from the loop. Cap iterations with a budget the agent can see. This is real engineering work rather than a config flag, which is exactly why context bloat is the dominant unmanaged cost in most agent fleets.
4. Self-hosted levers — quantization and the KV cache. If you run your own inference, quantizing KV caches to INT8/FP8 cuts VRAM 30–50% on long contexts, which converts directly into concurrency on the same hardware. Batching does the same for anything async. These are platform-team levers with platform-team payback periods — worth it at volume, premature before it.
Run the budget like a budget
Here’s where SRE gets to feel smug: the operating model for the dollar budget already exists, because it’s the error budget with a different unit.
Set the envelope. Expected tokens-per-task per agent, per action class. You’ll be wrong at first; the point is having a number to be wrong about.
Alert on burn rate, not on the invoice. A prompt regression, a runaway loop, or a silent model update that doubles token consumption should page within hours. Month-end cost review is the equivalent of discovering an outage from the quarterly uptime report. Burn-rate alerting on token spend catches one more thing, too: it’s a drift detector. A step change in tokens-per-task with no deploy is often the first observable sign that something upstream changed.
Make the spend attributable. Per-agent, per-task, per-tenant, from span-level telemetry — not allocated after the fact from a shared API key’s bill. (If all your agents share one key, you have more than a cost-attribution problem.)
Honest caveats. Cost-per-successful-output requires a definition of “successful” that survives contact with your grading pipeline — garbage quality signals make the metric confidently wrong. The 5–30× multiplier is a range, not your number; measure your own. And there’s a real risk of over-rotating: teams that turn token thrift into a KPI start starving agents of the context they need, buying a cheap bill and an unreliable fleet. The budget’s job is to make trade-offs visible, not to win every trade.
What to do Monday
- Get tokens into your traces. Per-span token usage with agent and task attribution. Everything else in this post is a query on top of that data.
- Turn on prompt caching and measure the hit rate. Hours of work, double-digit percentage savings on loop-heavy agents. There is no cheaper lever.
- Compute cost-per-successful-output for one agent. Your most expensive one. The number will either confirm the spend or start the right argument.
- Set a burn-rate alert on tokens-per-task. Even a crude threshold catches runaway loops and silent model changes months before the invoice does.
- Give your longest-running agent a visible iteration budget. A loop that knows it has 12 calls left plans differently than one that thinks tokens are free — and so, it turns out, do teams.
Reliability got its budget twenty years ago. Attention got one when context windows became the binding constraint. The dollar budget is the newest and the crudest — but it’s the one the CFO reads, and in 2026 it’s increasingly the number that decides whether your agent program expands or gets quietly wound down. Treat it with the same discipline as the other two, and it stops being a surprise and starts being a control surface.