AI AGENTS PRODUCTION
FAILURES 9 Ways Your Agent Will Betray You in Production

An AI agent making hundreds of requests per minute to an external API looks, from the API’s perspective, like a denial-of-service attack. Rate limiters do not distinguish between a well-intentioned agent and an attacker — they block based on request patterns, user-agent strings, and velocity. The agent gets throttled or banned, and your production workflow silently stalls.

The compounding problem is agent loop behaviour under rate limits. When a request is rejected, the agent retries. The retry triggers another rejection. The exponential backoff logic — if it exists — may not be long enough. Without proper circuit-breaking, the agent enters a retry death spiral that burns your request quota, generates cost, and produces nothing. In multi-agent systems, one agent hitting a rate limit can cascade — blocking downstream agents waiting for data that never arrives.

Detection also extends beyond simple rate limiting. Websites with bot protection (Cloudflare, DataDome) fingerprint agent behaviour through timing patterns, TLS fingerprints, header ordering, and browser automation tells. An agent that successfully retrieved data from a web source in testing may be blocked entirely in production once those protections update their signatures.

Every large language model has a finite context window — the total number of tokens it can hold in memory simultaneously. In multi-turn agent conversations with RAG pipelines, each turn adds: the new user message, retrieved document chunks, the model’s previous response, tool call results, and system prompt boilerplate. This accumulates fast.

Researcher Drew Breunig has documented four distinct failure modes within context window management. Context Poisoning is when a hallucination makes it into the context and is repeatedly referenced — as observed by DeepMind’s team in their Pokémon-playing Gemini agent, where hallucinated game-state information “poisoned” subsequent reasoning, causing the agent to fixate on impossible goals. Context Distraction occurs when the context overwhelms the model’s training — long histories cause the model to over-attend to recent context rather than pre-trained knowledge. Context Confusion and Context Clash arise from superfluous or contradictory information within a single context window.

When the context window fills completely, something must be dropped. Naive implementations drop the oldest context — which may include the original task definition. An agent that forgets its goal mid-execution is not a broken agent. It is an agent doing exactly what its architecture allows.

Fine-tuning an LLM for a specific domain — legal, medical, financial — seems like the obvious path to specialisation. It often destroys generalisation. Catastrophic forgetting is the well-documented phenomenon where a model trained on new data overwrites the weight patterns representing previous knowledge, degrading performance on tasks it could previously handle with competence.

Research published in January 2026 provides the mechanistic detail: forgetting operates through three primary channels — gradient interference in attention weights, representational drift in intermediate layers, and loss landscape flattening around prior task minima. Crucially, 15–23% of attention heads in lower layers show severe disruption, correlating with early forgetting. The non-linear temporal profile is deceptive: the first 1–2 epochs of fine-tuning cause minimal degradation, but forgetting accelerates dramatically in epochs 3–5 once the model begins truly converging on the new task.

An Oxford and Google DeepMind study named this at the model level “fidelity decay” — specifically “semantic drift” (where the meaning of a concept subtly shifts) and “semantic collapse” (where two distinct concepts merge into one, erasing nuance). A model fine-tuned on a harmless Q&A dataset showed measurable degradation in its understanding of “fairness” and “sycophancy” — concepts entirely unrelated to the fine-tuning data.

Text hallucinations are embarrassing. Function hallucinations are dangerous. When an agentic system equips a model with real tools — file system access, API calls, database writes, code execution — and that model hallucinates a function call, the result is not incorrect text. It is an incorrect real-world action, executed with the full permissions of the agent.

The model invents a function that does not exist, or calls a real function with fabricated parameters, or misidentifies which tool to call for a given task. Hallucination rates on specialised tasks range from 28–40% for medical review tasks (Stanford research) and 69–88% for legal citation generation. These rates do not improve simply because the model now has access to tools — they carry into tool selection and tool parameter generation. An agent choosing a “delete_customer_records” function when it should have called “archive_customer_records” is not making a reasoning error. It is producing the same category of stochastic error that text hallucination produces, with real-world consequences.

The Amazon Q Developer incident of 2025 illustrated the danger clearly: a compromised pull request injected a prompt that tricked the AI coding assistant into generating code that would delete local files and destroy AWS cloud infrastructure. The distinction between prompted and hallucinated harmful execution collapses when the agent acts without human confirmation.

When an agent generates synthetic data — summaries, transformed records, generated content — and that synthetic data feeds back into the training pipeline without verification, errors compound with each iteration. The model trains on its own mistakes, which amplify into the next generation’s outputs, which feed back again. This is recursive model collapse: the agent becoming its own source of increasingly distorted signal.

Multi-agent architectures amplify this risk. If a data retrieval agent is compromised or begins to hallucinate, it feeds corrupted data to downstream agents. Those downstream agents, trusting the input, make decisions that amplify the error across the system — creating cascading failures that propagate at machine speed with invisible lineage. In traditional systems, you can trace data lineage. With agents, the chain of reasoning is opaque. You see the final bad decision, but cannot easily rewind to find which agent introduced the corruption.

The Lakera AI memory injection research (November 2026) demonstrated a persistent variant: an attacker plants false information into an agent’s long-term memory store. The agent recalls this planted instruction in future sessions, defends it as correct when questioned by humans, and acts on it weeks later. One well-placed injection compromises months of agent interactions.

LLMs trust anything that can send them convincing-sounding tokens. This makes them fundamentally vulnerable to confused deputy attacks — where malicious instructions are embedded in content the agent is legitimately asked to process: web pages, documents, GitHub issue titles, email subjects. The agent follows the injected instruction as faithfully as if it came from the original operator.

The Clinejection attack of February 2026 is the most detailed documented example: a prompt injection hidden in a GitHub issue title tricked an AI triage bot into executing arbitrary code, triggering cache poisoning and credential theft that led to a compromised npm package installing a second AI agent on 4,000 developer machines. The attack chain began with a single issue title crafted to look like a performance report while containing an embedded instruction to install a package from a typosquatted repository.

Research confirms that just five carefully crafted documents can manipulate AI agent responses 90% of the time through RAG poisoning. When the agent’s retrieval pipeline fetches externally-sourced content — web pages, documents, emails — every retrieved chunk is a potential injection vector. GitHub Copilot’s CVE-2025-53773 allowed remote code execution through prompt injection with a CVSS score of 9.6, potentially affecting millions of developer machines.

LLMs are probabilistic. The same input does not produce the same output every time — the model samples from a probability distribution over possible next tokens. Most of the time, this variance is cosmetic: different wording, slightly different structure, similar meaning. But at the tail of that distribution, there are catastrophic outputs. The same input that produced a correct tool call 999 times will produce a harmful tool call on the 1000th — not because anything changed, but because probability.

The DEV Community documented the canonical example: an AI sales agent told a major customer it would receive a 50% discount — a commitment the company honoured at significant cost. Nothing in the system had changed. No adversarial input. No context overflow. Pure non-deterministic output from a model operating exactly as designed. This is not a bug. It is a feature of probabilistic systems operating in deterministic business contexts.

Non-determinism is manageable in chatbot contexts where a slightly different answer is acceptable. In agentic contexts — where the agent executes real transactions, sends real emails, makes real API calls, generates real financial commitments — tail-event outputs are not cosmetic. They are operational failures that traditional regression testing cannot catch because the test suite will pass 99.9% of the time.

An agent with a vector knowledge base — a RAG system with embedded documents — builds its “map of the world” at embedding time. The real world changes. Documents update. Concepts evolve. New products replace old ones. The vector space remains fixed, representing reality as it was when the embeddings were generated. As the gap between the vector map and current reality grows, retrieval accuracy degrades silently.

Oxford and Google DeepMind’s research on “fidelity decay” identifies semantic drift at both the model level (fine-tuning causes conceptual shift) and the retrieval level (embedding model updates cause search quality crashes). B2B data decays at up to 22.5% annually — meaning a knowledge base embedded a year ago contains roughly a quarter of stale information. The agent retrieves confidently, reasoning on facts that are no longer facts.

Semantic drift is particularly insidious because it produces failures that look like model reasoning errors. The retrieved context is wrong, but the model’s reasoning given that context is correct. Debugging suggests a model problem when the actual problem is a retrieval problem — leading teams to invest in prompt engineering or model upgrades when what they need is embedding refresh and retrieval monitoring.

Every production agent eventually faces the same architectural cliff: the thinking agent that reasons carefully is too slow to be useful; the fast agent that responds quickly is too reckless to be safe. This is not a parameter-tuning problem. It is a structural tradeoff baked into how inference works.

Reasoning models (o1, o3, Claude-3.7-Sonnet-thinking) produce higher-quality outputs through extended chain-of-thought. They also consume significantly more tokens, incur higher latency, and generate substantially higher API costs. The latency-cost tradeoff becomes a spiral when the application requires both accuracy and responsiveness: increasing reasoning depth increases both quality and cost; reducing reasoning depth reduces cost but increases failure rates; failure rates increase retry counts, which increases cost without improving quality.

Agents that make 3–10× more LLM calls than simple chatbots — as documented in production agent benchmarks — have their cost and latency problems multiplied by every additional hop. A single user request triggering planning, tool selection, execution, verification, and response generation can cost $5–8 per task in API fees at frontier model prices. At scale, this arithmetic becomes operationally unsustainable before the product achieves meaningful adoption.