AI Agent Error Handling: Graceful Degradation Patterns for Production Systems

Why AI Agents Fail Differently Than Traditional Software

Traditional software fails predictably. A database timeout throws an exception, a null pointer crashes a function, and a 404 means the resource is gone. AI agents fail in ways that are fundamentally harder to anticipate — an LLM returns confidently wrong output, a tool call succeeds but produces semantically incorrect results, or the agent enters an infinite reasoning loop that burns through your API budget.

Production AI agent systems need error handling strategies that go beyond try-catch blocks. They need graceful degradation — the ability to provide reduced but still useful functionality when components fail.

The Error Taxonomy for AI Agents

Before building error handling, you need to categorize the failure modes your agent can encounter.

flowchart TD
    CALL(["Inbound Call"])
    HEALTH{"Primary<br/>agent healthy?"}
    PRIMARY["Primary agent<br/>LLM provider A"]
    SECONDARY["Hot standby<br/>LLM provider B"]
    QUEUE[("Persisted<br/>call state")]
    HUMAN(["Live human<br/>fallback"])
    DONE(["Caller served"])
    CALL --> HEALTH
    HEALTH -->|Yes| PRIMARY
    HEALTH -->|Timeout or 5xx| SECONDARY
    PRIMARY --> QUEUE
    SECONDARY --> QUEUE
    PRIMARY --> DONE
    SECONDARY --> DONE
    SECONDARY -->|Both fail| HUMAN
    style HEALTH fill:#f59e0b,stroke:#d97706,color:#1f2937
    style PRIMARY fill:#4f46e5,stroke:#4338ca,color:#fff
    style SECONDARY fill:#0ea5e9,stroke:#0369a1,color:#fff
    style HUMAN fill:#dc2626,stroke:#b91c1c,color:#fff
    style DONE fill:#059669,stroke:#047857,color:#fff

Transient Infrastructure Failures

These are the easiest to handle: API rate limits, network timeouts, and temporary service outages. Standard retry logic with exponential backoff works well here.

import tenacity

@tenacity.retry(
    wait=tenacity.wait_exponential(multiplier=1, min=2, max=60),
    stop=tenacity.stop_after_attempt(5),
    retry=tenacity.retry_if_exception_type(
        (RateLimitError, TimeoutError, ConnectionError)
    ),
)
async def call_llm(prompt: str, model: str) -> str:
    return await client.chat.completions.create(
        model=model, messages=[{"role": "user", "content": prompt}]
    )

Semantic Failures

The LLM returns a valid response, but the content is wrong, incomplete, or nonsensical. These are harder to detect because no exception is thrown. Defense strategies include output validation schemas, confidence scoring, and cross-model verification for high-stakes decisions.

Cascade Failures

One agent in a multi-agent pipeline fails, and the bad output propagates downstream. A planning agent produces an invalid plan, the execution agent tries to follow it, and the entire workflow derails. Circuit breakers and inter-agent validation checkpoints prevent this.

Core Degradation Patterns

Pattern 1: Model Fallback Chain

When your primary model is unavailable or producing poor results, fall back to alternatives.

MODEL_CHAIN = ["gpt-4o", "claude-3-5-sonnet", "gpt-4o-mini"]

async def resilient_completion(prompt: str) -> str:
    for model in MODEL_CHAIN:
        try:
            result = await call_llm(prompt, model)
            if passes_quality_check(result):
                return result
        except (RateLimitError, TimeoutError):
            continue
    return generate_fallback_response(prompt)

Pattern 2: Scope Reduction

When the agent cannot complete the full task, reduce scope rather than failing entirely. If a research agent cannot access three of its five data sources, it should return partial results with clear attribution of what sources were available, rather than returning nothing.

Pattern 3: Human Escalation with Context

For critical failures, escalate to a human operator but package the full context — what the agent was trying to do, what failed, what partial results exist, and what the agent recommends as next steps.

Pattern 4: Checkpoint and Resume

Long-running agent workflows should checkpoint intermediate state so that failures do not require restarting from scratch. This is especially important for multi-step processes like document analysis pipelines or complex research tasks.

class CheckpointedAgent:
    async def run(self, task_id: str, steps: list[Step]):
        checkpoint = await self.load_checkpoint(task_id)
        for i, step in enumerate(steps):
            if i < checkpoint.last_completed:
                continue
            try:
                result = await step.execute()
                await self.save_checkpoint(task_id, i, result)
            except AgentError as e:
                await self.handle_step_failure(task_id, i, e)
                break

Circuit Breakers for Agent Systems

The circuit breaker pattern from microservices architecture adapts well to AI agents. Track failure rates per tool and per model. When failures exceed a threshold, open the circuit and route requests to fallback paths instead of continuing to hit failing services.

A good implementation tracks three states: closed (normal operation), open (all requests go to fallback), and half-open (periodic test requests to check if the service has recovered).

Monitoring Degradation in Production

Every degradation event should be logged with structured metadata: which component degraded, what fallback was used, what capability was lost, and the estimated impact on output quality. This data feeds into dashboards that show the real-time health of your agent system — not just uptime, but quality-adjusted availability.

Sources:

• https://www.anthropic.com/research/building-effective-agents
• https://learn.microsoft.com/en-us/azure/architecture/patterns/circuit-breaker
• https://docs.llamaindex.ai/en/stable/understanding/agent/error_handling/