The Router Pattern: Building AI Agents That Intelligently Direct Requests

Why Routing Matters in Multi-Agent Systems

When you build a system with multiple specialized agents — one for billing questions, another for technical support, a third for scheduling — you need a mechanism that examines each incoming request and sends it to the right handler. This is the Router pattern: a central classification layer that inspects a request, determines which agent or pipeline should handle it, and forwards it accordingly.

Without a router, you either force a single monolithic agent to do everything (reducing quality) or rely on the user to manually select the right department (reducing usability). A well-designed router gives you the best of both worlds: specialized agents that excel at their domain, with seamless automatic dispatch.

Core Components of the Router Pattern

The Router pattern consists of three elements:

flowchart LR
    INPUT(["User intent"])
    PARSE["Parse plus<br/>classify"]
    PLAN["Plan and tool<br/>selection"]
    AGENT["Agent loop<br/>LLM plus tools"]
    GUARD{"Guardrails<br/>and policy"}
    EXEC["Execute and<br/>verify result"]
    OBS[("Trace and metrics")]
    OUT(["Outcome plus<br/>next action"])
    INPUT --> PARSE --> PLAN --> AGENT --> GUARD
    GUARD -->|Pass| EXEC --> OUT
    GUARD -->|Fail| AGENT
    AGENT --> OBS
    style AGENT fill:#4f46e5,stroke:#4338ca,color:#fff
    style GUARD fill:#f59e0b,stroke:#d97706,color:#1f2937
    style OBS fill:#ede9fe,stroke:#7c3aed,color:#1e1b4b
    style OUT fill:#059669,stroke:#047857,color:#fff

1. Classifier — Analyzes the incoming request and produces a category label with a confidence score
2. Route Table — Maps category labels to specific agent handlers
3. Fallback Route — Catches requests where no route matches confidently enough

Here is a complete implementation:

from dataclasses import dataclass
from enum import Enum
from typing import Callable, Any
import openai

class RequestCategory(Enum):
    BILLING = "billing"
    TECHNICAL = "technical"
    SCHEDULING = "scheduling"
    GENERAL = "general"

@dataclass
class ClassificationResult:
    category: RequestCategory
    confidence: float
    reasoning: str

@dataclass
class Route:
    category: RequestCategory
    handler: Callable[[str], Any]
    min_confidence: float = 0.7

class AgentRouter:
    def __init__(self, routes: list[Route], fallback: Callable[[str], Any]):
        self.routes = {r.category: r for r in routes}
        self.fallback = fallback
        self.client = openai.OpenAI()

    def classify(self, message: str) -> ClassificationResult:
        categories = ", ".join([c.value for c in RequestCategory])
        response = self.client.chat.completions.create(
            model="gpt-4o",
            messages=[
                {
                    "role": "system",
                    "content": (
                        f"Classify the user message into one of: {categories}. "
                        "Respond with JSON: {"category": "...", "
                        ""confidence": 0.0-1.0, "reasoning": "..."}"
                    ),
                },
                {"role": "user", "content": message},
            ],
            response_format={"type": "json_object"},
        )
        import json
        data = json.loads(response.choices[0].message.content)
        return ClassificationResult(
            category=RequestCategory(data["category"]),
            confidence=data["confidence"],
            reasoning=data["reasoning"],
        )

    def route(self, message: str) -> Any:
        result = self.classify(message)
        route = self.routes.get(result.category)

        if route and result.confidence >= route.min_confidence:
            print(f"Routing to {result.category.value} "
                  f"(confidence: {result.confidence:.2f})")
            return route.handler(message)

        print(f"Falling back — category: {result.category.value}, "
              f"confidence: {result.confidence:.2f}")
        return self.fallback(message)

Wiring Up Specialized Handlers

Each handler is a function (or agent) that processes the request within its domain:

def billing_agent(message: str) -> str:
    return f"[Billing Agent] Processing: {message}"

def technical_agent(message: str) -> str:
    return f"[Technical Agent] Processing: {message}"

def scheduling_agent(message: str) -> str:
    return f"[Scheduling Agent] Processing: {message}"

def general_agent(message: str) -> str:
    return f"[General Agent] Processing: {message}"

router = AgentRouter(
    routes=[
        Route(RequestCategory.BILLING, billing_agent, min_confidence=0.7),
        Route(RequestCategory.TECHNICAL, technical_agent, min_confidence=0.6),
        Route(RequestCategory.SCHEDULING, scheduling_agent, min_confidence=0.75),
    ],
    fallback=general_agent,
)

# Usage
response = router.route("I was charged twice on my last invoice")
# Output: Routing to billing (confidence: 0.95)

Tuning Confidence Thresholds

Setting min_confidence per route lets you control the tradeoff between precision and recall. A high threshold (0.85+) means the router only forwards when it is very certain, sending ambiguous requests to the fallback. A lower threshold (0.5-0.6) routes more aggressively but risks misclassification.

Start with 0.7 across all routes, then analyze misrouted requests to adjust per category. Categories with costly errors (like billing) benefit from higher thresholds.

FAQ

How do I handle requests that match multiple categories equally well?

Return the top-N classifications from your classifier and check if the top two scores are within a small delta (e.g., 0.05). When they are too close to call, route to the fallback agent or ask the user a clarifying question before proceeding.

Should I use an LLM or a traditional classifier for routing?

For prototyping and systems with fewer than 10 categories, an LLM classifier works well and requires no training data. For high-throughput production systems processing thousands of requests per second, a fine-tuned BERT or logistic regression model will be faster and cheaper. Many teams start with LLM routing and graduate to a trained classifier once they have labeled data from production traffic.

How do I add a new route without redeploying the entire system?

Store your route table in a configuration file or database rather than hardcoding it. The router reads the config at startup (or periodically refreshes it), so adding a new category and handler becomes a configuration change rather than a code deployment.

---

#AgentDesignPatterns #RouterPattern #Python #AgenticAI #SoftwareArchitecture #LearnAI #AIEngineering