In May 2026, Gabriel Anhaia published a production trace that will look familiar to anyone who has run a multi-step agent in production.

A customer service agent called a database lookup tool. The tool returned a truncated JSON blob — an upstream gateway had a 4KB response cap nobody had documented. The model correctly identified the response as broken and decided to retry. The tool returned the same truncated payload. The model retried again. And again. Seventeen times. Each turn a full prompt round-trip with growing context, accumulating tokens, running up costs — all because nothing between the tool and the model translated "this response is malformed" into actionable signal.

The model wasn't wrong. It was doing exactly what it was trained to do: retry on apparent transient failure. The failure wasn't the model. It was the gap between what the tool returned and what the agent could reason about.

That gap is where most AI agent output quality problems originate — and it's the last place most teams look.

Tool Misuse Is the Most Common AI Agent Failure in Production

According to Latitude's 2026 observability framework report, tool misuse is the most common agent-specific failure mode in production. Not hallucination. Not reasoning failures. Not inadequate context. The most common failure is agents calling tools with wrong arguments, missing required fields, or incorrect data types — and then failing in ways that don't generate obvious error signals.

This matters because it inverts the usual diagnostic instinct. When an agent produces a bad output, the first question is almost always: what did the model get wrong? The more productive question, based on production failure data, is: what happened at the tool interface?

The structural reason is straightforward. A well-designed agent isn't just a model generating text — it's a chain of tool invocations connected by model reasoning. Each link in that chain has specific schema expectations: exact field names, exact data types, exact nesting structures. The model must satisfy every one of these on every call. At context window lengths typical of production multi-step workflows, models fail at schema compliance more often than benchmark scores suggest — because benchmarks typically measure final-output accuracy, not intermediate tool call fidelity.

A single malformed argument at step 2 silently corrupts every subsequent step that depends on that output. The corruption doesn't necessarily produce a hard failure. It continues through the pipeline, gets passed to the next step, and eventually surfaces as a wrong answer, an incomplete result, or a behavior the agent would have handled correctly if the tool had returned clean data.

Three Classes of Tool Call Failure — and Why They Each Require Different Handling

Not all tool call failures look the same, and the agent's right move depends heavily on which class it's dealing with.

Schema mismatch is the most dangerous class. The tool returned data, but it doesn't conform to the contract the agent was promised: wrong field types, missing required keys, invalid JSON, truncated payloads. The retry-loop trap documented in Anhaia's trace falls here. The right response is not to retry with the same arguments — the tool itself is broken, and identical retries will produce identical garbage. Agents without explicit schema validation logic have no way to distinguish "transient network error" from "structurally broken response," and default to retrying both.

Partial data is retryable, but with different parameters. The tool returned valid, well-formed data, but it's incomplete: pagination cut off mid-result, a timeout returned what was available so far, an external API rate-limited and returned 3 of 47 records. The same call won't help. A modified call — smaller page, narrower filter, different time window — might.

Semantic garbage is the hardest to catch. The tool returned valid, well-formed, complete data, and the data is wrong in ways schema validation can't detect. An agent passed a customer name in a field that expected an ID. A search API returned zero results because a filter value was semantically adjacent to the right one but not syntactically correct. The response looks fine. The content makes no sense relative to what was asked.

Each class produces a different downstream failure pattern. Without tool-level instrumentation that captures the argument, the schema context, and the response in sequence, distinguishing these three classes after the fact requires manual reconstruction of execution traces — which doesn't scale.

Why Existing Output Quality Evaluation Misses This

Most teams approach AI agent output quality with one of two evaluation approaches: LLM-as-judge for semantic quality, or static output gates for format and content validation. Both evaluate the terminal step — the final response — and neither reaches into intermediate tool calls.

This is the structural blind spot. The Hacker News thread "AI agent benchmarks are broken" — which generated 86 comments when it ran in July 2025 — made this structural argument in a different form. A recurring observation in the thread: agents evaluated only on final-output quality can pass 38% of tasks by doing nothing at all, and the evaluation architecture used by most benchmarks (LLMs judging LLM outputs) shares the same blind spots as the thing under test. What the thread didn't quite name was the corollary: if final-output evaluation is a poor proxy for agent quality at the benchmark level, it's an even poorer proxy at the production level, where tools fail in ways synthetic benchmarks never exercise.

Offline evaluation can't close this gap. Production tool call failures emerge from specific context window states, specific data payloads, and the interaction effects between a particular model version and a particular external API's current behavior. A model that handles a tool schema correctly at 2,000 tokens of context can generate approximate or partially correct arguments at 12,000 tokens — even with identical instructions. The degradation is context-sensitive, distribution-sensitive, and invisible to any evaluation pipeline that doesn't operate on production traces.

The problem isn't that teams aren't evaluating. It's that evaluation is happening at the wrong level.

Silent Failures: When the Output Looks Right but the Path Was Wrong

There's a related failure mode that's even harder to catch: the agent produces the correct final output through an incorrect intermediate process.

The agent references last year's report instead of this year's. It queries a deprecated API endpoint that happens to return data that was valid six months ago. It infers an intermediate value — one that happens to be right in this instance — rather than querying for it. The output passes final-output validation. It gets used. The failure is invisible until the same behavior on different data produces a wrong answer.

This class of failure — correct output, incorrect process — requires trajectory evaluation to detect. You need to see not just what the agent produced, but how it navigated there: which tool calls it made, in what order, with what arguments, and what each returned. Without that trace, process failures are invisible by definition.

How Waxell Handles This

Waxell Observe captures every model call and tool use in order — the full trace of an agent run from first inference through final output, with every tool invocation argument and response, in sequence, in context.

This is the structural difference between a final-output dashboard and tool-call-level governance. With output monitoring at the tool interface, teams can detect schema violations and argument format errors in the execution trace — not just in the terminal response. They can identify which specific tool calls are producing degraded outputs across runs, compare execution paths across context window lengths, and catch the correlation between long-context states and schema compliance drift before it becomes a production incident.

Output validation policies extend this into pre-execution enforcement: flagging intermediate tool call outputs that violate defined schema contracts mid-run, not only runs where the terminal answer fails. This shifts quality enforcement upstream — from after-the-fact detection to mid-run intervention.

Waxell Observe initializes with 2 lines of code and auto-instruments 200+ libraries, which means teams don't need to manually add instrumentation to each tool call. Every call is traced. Every argument is captured. The 50+ policy categories available include content, quality, and reasoning policies that operate at the individual step level, not just the final response. Policy checks run at 0.045ms p95 latency — tool-level enforcement adds no meaningful overhead to production pipelines.

For teams testing output quality before production, Waxell's testing environment lets you run governed agent workflows against defined quality contracts before promoting to production — so tool-level schema regressions surface in testing, not in customer-facing failures.

FAQ

What is tool argument rot in AI agents?
Tool argument rot is the progressive generation of malformed, incorrect, or schema-violating arguments when AI agents call tools — producing truncated JSON, missing required fields, wrong data types, or syntactically invalid payloads. It's the most common agent-specific production failure mode according to observability practitioners, and it's more insidious than final-output hallucination because it doesn't necessarily produce immediate hard failures. A bad tool call at step 2 corrupts the context for every subsequent step that depends on it.

Why does AI agent output quality degrade in production but not in testing?
Tool call failures in production emerge from specific context window states, specific external API behaviors, and the interaction effects between model version and real data distributions that synthetic test cases don't exercise. A model that handles a tool schema correctly at 2,000 tokens of context can generate approximate arguments at 12,000 tokens with identical instructions. This context-sensitive degradation is invisible to offline evaluation because it requires production trace data to surface.

What's the difference between tool misuse and tool failure?
Tool misuse is an agent-side problem: the agent called the tool with incorrect arguments, selected the wrong tool for the task, or failed to handle a tool error correctly. Tool failure is a tool-side problem: the tool returned an error, an empty response, or malformed data. Both produce bad downstream outputs. The distinction matters for the fix: tool misuse requires changing what the agent sends; tool failure requires changing how the agent interprets and handles what it receives.

How do you detect AI agent tool call quality issues in production?
The only reliable approach is to instrument every tool call — capturing the argument, the schema context, and the response — in the full trace of each agent run. Post-hoc final-output evaluation misses process failures because it doesn't see how the agent navigated to the output. Real-time tool-call tracing with schema validation policies allows teams to catch malformed arguments in-run, compare execution paths across context window lengths, and identify which specific tool calls are introducing quality degradation.

What are silent failures in AI agent pipelines?
Silent failures are runs where the agent produces the correct final output through an incorrect intermediate process — referencing stale data, using a deprecated API endpoint, or inferring a value that should have been queried. The output passes validation and gets acted on, but the agent navigated there incorrectly. Silent failures accumulate without triggering error signals. The only way to catch them is trajectory evaluation: tracing every step of the agent's execution, not just checking the terminal response.

Why don't LLM-as-judge evaluations catch tool call failures?
LLM-as-judge evaluation operates on the agent's final response and has no visibility into the tool calls that produced it. It can detect a wrong answer but can't distinguish between "wrong answer because the model reasoned incorrectly" and "wrong answer because a tool returned bad data at step 3." Both produce the same signal to a final-output evaluator. Catching tool-level failures requires step-level instrumentation, not response-level evaluation.

Sources

1. Gabriel Anhaia, "When Your Tool Returns Garbage, Agents Loop Forever. Here's the 30-Line Guard." — DEV Community, May 2026. Production trace: 17 identical retries on truncated JSON. Three-class taxonomy of tool failures.
   https://dev.to/gabrielanhaia/when-your-tool-returns-garbage-agents-loop-forever-heres-the-30-line-guard-5b09

2. César Migueláñez, "Detecting AI Agent Failure Modes in Production: A Framework for Observability-Driven Diagnosis" — Latitude, March 26, 2026. Primary source for "tool misuse is the most common agent-specific failure mode" claim.
   https://latitude.so/blog/ai-agent-failure-detection-guide

3. "AI agent benchmarks are broken" — Hacker News, July 11, 2025. 185 points, 86 comments. Discussion of benchmark blind spots, final-output evaluation gaps, and the 38% do-nothing pass rate.
   https://news.ycombinator.com/item?id=44531697

4. Andrey Podivilov et al., "AgentLens: Production-Assessed Trajectory Reviews for Coding Agent Evaluation" — arXiv:2607.06624 [cs.AI], submitted July 7, 2026, revised July 14, 2026. Trajectory-level benchmark for coding agents.
   https://arxiv.org/abs/2607.06624