In November 2025, four LangChain agents entered an infinite loop and ran for 11 days. The bill was $47,000. The team had a monitoring dashboard running the entire time — and nobody noticed until it was over. The post-mortem found no per-agent budget caps and no mechanism to terminate a session before the next API call completed. (That incident is covered in depth in The $47,000 Agent Loop →.)

There's a mental model mismatch that makes this kind of disaster predictable, and it comes down to this: engineers think in requests. Agents run in loops.

When you're building a traditional API-backed feature, cost math is simple. One user action equals one API call. One API call has a known cost. You multiply by your DAU and you have your monthly bill, approximately. Budget accordingly.

When you're building agents, this model falls apart. An agent doesn't make one call. It reasons, retrieves, calls tools, observes the results, reasons again, calls more tools, synthesizes — and each of those steps is a context window, and each context window grows as the conversation goes on because everything that came before gets appended. The token count isn't static. It compounds.

AI agent cost control is the practice of defining and enforcing token budgets, session limits, and spend guardrails at the infrastructure layer — not inside agent code. Unlike traditional API cost management (or cost observability, which tracks what was spent), agent cost control prevents future spend: it enforces a ceiling before the next API call initiates, regardless of where the agent is in its reasoning process. Agent costs don't scale linearly with requests: they compound with context accumulation, loop behavior, and tool call overhead, making proactive enforcement essential. A 5-step agent loop serving 200 concurrent users can cost 10× what naive per-request estimates suggest. (See also: What is agentic governance →)

A March 2026 Gartner survey of 353 data and AI leaders found that only 44% of organizations have adopted financial guardrails or AI FinOps practices. IDC's FutureScape 2026 projects that G1000 organizations will face up to a 30% rise in underestimated AI infrastructure costs by 2027, driven specifically by the "opaque consumption models" of agentic workloads. Most teams are flying blind until the bill arrives.

Why Do AI Agent Costs Multiply With Each Loop?

Take a modest agent. It handles customer support queries. For a typical question, it goes through this cycle: initial reasoning call, two tool calls (retrieve relevant docs, look up account status), a synthesis call to compose the response. Four LLM calls. Say your context window at each step is, on average, 4,000 tokens. That's 16,000 tokens for one resolved query.

That doesn't sound terrible. At GPT-4 pricing, that's maybe 50 cents for a complex query. You're building a support product. You're charging for it. Fine.

Now add scale. You're serving 200 concurrent users. You're now at 3,200,000 tokens per "round" of queries. If an average session involves five exchanges before it's resolved, you're at 16,000,000 tokens per hour of support traffic.

Now add the thing nobody budgets for: variance. Some queries don't resolve in five exchanges. Some agents get into loops — they call a tool, the tool returns something unexpected, they reason about it, call the tool again, get a similar result, reason again. Without a hard stop, a single looping session can consume 100× the token budget of a normal one. You don't notice it until it's in the tail of your cost distribution, and by the time it's notable in your aggregate, it's happened hundreds of times.

This is not a hypothetical. Every company that has shipped agents at scale has a version of this story.

Where Do AI Agent Cost Spirals Actually Come From?

Context accumulation. The longer a conversation runs, the more tokens get included in every subsequent call. A 20-turn conversation doesn't cost 20× a 1-turn conversation — it costs significantly more, because every turn includes the entire history. If you're not actively managing context window size (through summarization, pruning, or hard turn limits), you're letting cost compound with every exchange.

Tool call overhead. Each tool call requires the agent to explain what it's calling and why, get the result back, and reason about the result — all of which adds tokens. Agents that call tools aggressively, or that call tools whose results are verbose, pay a significant overhead per call. A tool that returns 500 tokens when 50 would do is costing you 10× more than necessary.

Retries. When a tool call fails, or when the model's output doesn't pass a validation step, the agent retries. Retries are full calls at full cost. An agent that retries aggressively on flaky tools can run up a large bill in a short time.

Concurrent sessions at peak. If your agent is customer-facing, you have peak hours. During peak, the number of concurrent sessions multiplies your per-session costs. If your agent also tends to take longer (more reasoning steps, more tool calls) on high-complexity queries, and high-complexity queries cluster at peak... you can see where this goes.

What Are the Most Effective Strategies for Controlling AI Agent Costs?

1. Session-level token budgets with hard stops. The most direct intervention. Set a maximum token budget per session. When a session approaches the limit, either summarize the conversation so far and continue with a compressed context, or gracefully terminate and hand off to a human. The key word is "hard stop" — a soft warning that the agent can ignore doesn't help. The cost enforcement layer needs to be able to terminate a session before it burns past the ceiling.

This sounds aggressive. In practice, a well-tuned budget catches the outlier sessions (the loops, the unusually complex queries) without affecting the median session at all. The $47,000 loop incident happened because the team had no mechanism to terminate sessions at the execution layer — only monitoring that fired alerts nobody acted on in time. For a deeper breakdown of why alerts aren't enforcement, see The $400M AI FinOps Gap →.

2. Context pruning and compression. Rather than including the full conversation history in every call, summarize older turns. After every N exchanges, run a lightweight summarization call that compresses the conversation history, replace the raw history with the summary, and proceed. You lose some fine-grained context but retain the semantic substance. For most agent tasks, this tradeoff is highly favorable.

The cost of the summarization call is paid back immediately in the reduced token count of every subsequent call. For long sessions, the savings compound.

3. Tiered model routing. Not every step in your agent's reasoning requires your most capable (and most expensive) model. Tool selection, parameter extraction, routine classification — these can often be handled by smaller, faster, cheaper models. Reserve GPT-4-class models for reasoning steps that genuinely require them: complex synthesis, nuanced judgment calls, multi-step reasoning chains.

This requires knowing, at design time, which steps in your agent's workflow actually need heavy reasoning. It's worth the analysis. The cost difference between a frontier model and a smaller model on simple tasks is often 10× or more.

4. Real-time spend visibility with alerting before thresholds. This sounds obvious. It isn't standard practice. Most teams find out about cost anomalies after the billing period closes. You need to know about a spiraling session while it's happening — specifically, before it hits a threshold that matters.

Track agent costs in real time with a per-session alert at 60% of your budget ceiling. Set a fleet-level alert when aggregate spend for the hour is trending beyond your daily allocation. These alerts should go somewhere someone is actually watching, not just into a log. And remember: alerting is not enforcement. The session that blew past $47,000 was generating alerts. Enforcement means the session stops automatically.

What Do AI Agent Budget Guardrails Look Like in Practice?

A budget guardrail isn't a feature you add to your agent code. It's a policy at the infrastructure layer — something that observes session costs in real time, tracks against a defined budget, and takes action when thresholds are reached.

The action options are: warn (send an alert but let the session continue), compress (trigger a context compression step to reduce future call costs), cap (refuse to initiate new LLM calls until the session ends), or terminate (end the session and hand off appropriately).

Which action is right depends on your product. A customer support agent might compress first, then cap; termination would be a last resort because it creates a bad experience. An internal automation agent might terminate cleanly — the job can be requeued when budget resets.

The important thing is that these decisions are made at policy definition time, not ad hoc in the moment. The enforcement policies define the decision. The governance layer enforces it.

Who Owns AI Agent Cost Governance?

One more thing worth naming: cost governance for agents requires cross-functional alignment that most organizations haven't established yet.

Engineering sets the technical budget ceiling. Finance needs to understand that agent costs are variable in ways traditional SaaS infrastructure costs are not. Product needs to understand the cost implications of features that increase session depth or tool call frequency. Without this alignment, you get a cost spike, a panicked response, and a retroactive patch — instead of a thoughtful policy that was in place from the start.

The time to build that alignment is before the spike. It's a boring conversation to have. Have it anyway. If you're thinking about how this fits into a broader operational discipline, AgentOps: The Discipline Missing From Your AI Deployment Stack covers the full operational stack.

Agent costs are controllable. The math isn't mysterious. The tools exist. What's usually missing is the governance layer to enforce the policies that translate knowledge into behavior — automatically, in production, without requiring an engineer to be watching.

How Waxell handles this: Waxell's Cost policy type enforces session-level token budgets in real time — tracking spend as it accumulates and triggering configurable responses (compress, cap, or terminate) when sessions approach defined thresholds. Real-time spend telemetry gives live visibility into per-session token consumption across your agent fleet. Enforcement policies are defined once at the governance layer and apply to every session without touching agent code — the session terminates before the next call goes out, not after the damage compounds. Fleet-level spend alerts fire before your daily allocation is consumed, not after. Start free →

Frequently Asked Questions

Why do AI agent costs spiral in production?
Agent costs spiral because they compound rather than scale linearly. Each turn in a conversation extends the context window, making every subsequent LLM call more expensive. Tool calls add overhead at each step. Loops — where an agent retries a failing operation — can run up 100× the normal session cost. Without a governance layer enforcing budget ceilings, a single outlier session can consume as much as hundreds of typical sessions. The $47,000 November 2025 incident (4 LangChain agents, A2A protocol, 11-day loop) is the clearest documented example of this dynamic at production scale.

How do you calculate the real cost of an AI agent session?
Multiply the average number of LLM calls per session by the average context window size at each call. For a typical 4-step agent handling a query with a 4,000-token average context per step, that's 16,000 tokens per resolved query. At 200 concurrent users, 5 exchanges per session, you're at roughly 16 million tokens per hour of traffic — before accounting for variance from long sessions or loops. Most teams' initial estimates are off by 5–10× before they do this math explicitly.

What is a token budget for AI agents?
A token budget is a hard limit on the total tokens a single agent session is allowed to consume. When the session approaches the limit, the governance layer triggers a predefined response: summarize the conversation history and continue with compressed context, stop accepting new LLM calls until the session ends, or gracefully terminate and hand off to a human. A well-tuned budget catches outlier sessions — loops, unusually long conversations — without affecting typical sessions at all.

What causes AI agent cost explosions?
Four mechanisms cause most cost explosions: context accumulation (full conversation history included in every subsequent call), tool call overhead (verbose tool responses inflating every context window), retry behavior (failed calls at full cost), and concurrent session peaks (multiplied per-session costs during high traffic). Any one of these is manageable. When they compound — a peak traffic moment with verbose tools and some agents in retry loops — costs can spike dramatically in minutes.

How do you stop an AI agent from going over budget?
Budget enforcement has to be at the infrastructure layer, not in the agent's system prompt. The governance layer observes token spend in real time, compares against the defined budget, and triggers hard stops — not suggestions the model can override. Effective implementations alert at 60% of the budget ceiling (while there's still time to act), compress context at 80%, and cap or terminate at the ceiling. The key word is "hard stop": a soft warning the model can reason around does not constitute cost governance.

What's the difference between AI agent cost monitoring and cost enforcement?
Cost monitoring tracks what was spent — dashboards, per-session breakdowns, alerting when thresholds are approached. It is asynchronous: by the time an alert fires, the spend has already occurred. Cost enforcement intercepts execution before the next API call and evaluates it against a budget ceiling. If the ceiling is reached, the session terminates before the call goes out. The $47,000 incident in November 2025 happened on a team that had monitoring. They did not have enforcement. A March 2026 Gartner survey found that only 44% of organizations have adopted financial guardrails at all — most are relying on monitoring alone.

Sources

• Medium / CodeOrbit, Our $47,000 AI Agent Production Lesson: The Reality of A2A and MCP (November 2025) — https://medium.com/@theabhishek.040/our-47-000-ai-agent-production-lesson-the-reality-of-a2a-and-mcp-60c2c000d904

• TechStartups.com, AI Agents Horror Stories: How a $47,000 AI Agent Failure Exposed the Hype and Hidden Risks of Multi-Agent Systems (November 14, 2025) — https://techstartups.com/2025/11/14/ai-agents-horror-stories-how-a-47000-failure-exposed-the-hype-and-hidden-risks-of-multi-agent-systems/

• Hacker News, We spent 47k running AI agents in production (November 2025) — https://news.ycombinator.com/item?id=45802430

• Gartner, Gartner Identifies Three Pillars for Deriving Value from AI (March 9, 2026) — https://www.gartner.com/en/newsroom/press-releases/2026-03-09-gartner-identifies-three-pillars-for-deriving-value-from-ai (44% of organizations have adopted financial guardrails or AI FinOps practices)

• IDC, FutureScape 2026: Moving into the Agentic Future (2026) — https://www.idc.com/resource-center/blog/futurescape-2026-moving-into-the-agentic-future/ (G1000 organizations will face up to 30% rise in underestimated AI infrastructure costs by 2027)

• FinOps Foundation, State of FinOps 2026 — https://data.finops.org/ (98% of FinOps practices now manage some form of AI spend, up from 31% two years prior; 1,192 respondents, $83B+ in annual cloud spend)