Shipping a Claude Agent End-to-End: A Real Walkthrough

Most writing about agents stops at the architecture diagram. This post does the opposite: it follows one realistic build from the moment a problem lands on an engineer's desk to the moment the agent is quietly doing useful work in production, and it dwells on the unglamorous decisions in between — the ones that determine whether the thing actually ships or dies as a demo. The example is a contract-review triage agent for a mid-sized company's legal operations team, but the shape of the journey generalizes to almost any Claude agent you'd build in 2026.

The problem, stated honestly

The legal ops team receives roughly a hundred inbound contracts a week — vendor agreements, NDAs, order forms — and a small team of reviewers is drowning. Most contracts are routine and could be approved against a standard playbook; a minority contain non-standard clauses that genuinely need a human lawyer. The reviewers spend the bulk of their time on the routine ones just to find the few that matter. The business ask, as first stated, was "can AI review our contracts." That framing is a trap. An agent that tries to fully review contracts inherits enormous liability and a fuzzy success criterion.

The reframe that made the project shippable: don't review contracts, triage them. The agent's job is to read each incoming contract, compare it against the company's standard playbook, flag every deviation with a citation to the specific clause, and route the contract into one of three buckets — clean (auto-approvable), minor deviations (fast human check), or material deviations (full legal review). The agent never approves anything; it sorts and explains. That narrowing turned an unbounded, scary problem into a bounded, measurable one, and it's the single most important decision in the whole project.

Designing the agent

With the job defined, the architecture follows. The core is a Claude agent given three things: a skill that encodes the company's contract playbook (the standard clauses, the acceptable variations, the red-flag language), a set of tools via MCP to fetch the contract text, look up the relevant playbook section, and write the triage result back into the legal ops system, and a tight specification of the output format — a structured verdict with per-clause findings and citations.

We deliberately kept it single-agent at first. Multi-agent orchestration was tempting — a subagent per contract section — but it would have multiplied token cost several times for a problem a single capable model handles well, so we left that option on the shelf until evals proved we needed it. The early discipline was: simplest thing that could work, measured against a real eval set.

flowchart TD
  A["Contract arrives"] --> B["Agent fetches text via MCP"]
  B --> C["Load playbook skill"]
  C --> D["Compare clauses to standard"]
  D --> E{"Deviations found?"}
  E -->|"None"| F["Bucket: clean"]
  E -->|"Minor"| G["Bucket: fast check"]
  E -->|"Material"| H["Bucket: full legal review"]
  F --> I["Write verdict + citations"]
  G --> I
  H --> I

The eval set came before the prompt. We took eighty historical contracts the legal team had already triaged, captured their human verdicts as ground truth, and held them out. Every change to the skill or prompt got scored against those eighty: did the agent's bucket match the human's, and when it flagged a deviation, was the citation correct. This eval harness is what let two engineers iterate confidently instead of arguing about whether the latest prompt tweak helped.

The messy middle: where most projects stall

The first version scored well on clean contracts and badly on edge cases — exactly the contracts that matter most. Reading the failing transcripts (observability earning its keep) showed two recurring problems. The playbook skill was too terse, so the agent didn't know that a particular indemnity phrasing was acceptable; we expanded the skill with explicit examples of acceptable and unacceptable variations, and accuracy on materials jumped. The second problem was citation drift: the agent would correctly identify a deviation but cite the wrong clause number. We fixed it by changing the fetch tool to return the contract text already segmented with stable clause IDs, so the agent cited an ID rather than counting paragraphs.

Neither fix was clever AI work; both were unglamorous interface design — making the tools and skills feed the model exactly the right shape of information. This is the pattern in nearly every real build: the model is rarely the limiting factor, the scaffolding around it is. We also added a hard guardrail: the agent's verdict tool refuses to write a "clean" bucket for any contract above a certain dollar value, forcing those into human review regardless of what the model thought. That's a policy gate doing risk containment, and it cost the team almost nothing.

Shipping and the outcome

We didn't flip it on for all traffic. The rollout was a shadow phase first: the agent triaged every incoming contract in parallel with the humans, but its verdict was logged, not acted on. For two weeks we compared agent buckets against human buckets on live traffic, which surfaced a few playbook gaps the historical eval set had missed. Once the live agreement rate held steady, we promoted the agent to handle the routing for real, with the humans now spending their time on the material-deviation bucket the agent surfaced — exactly the work they were good at and short on time for.

The shipped outcome wasn't "AI reviews contracts." It was a triage layer that let a constant-size team absorb a growing contract volume by spending their attention where it mattered, with every agent decision logged, citable, and reversible. The reviewers trusted it because they could see why each contract was bucketed the way it was. And the project succeeded not because the model was magic but because the problem was narrowed honestly, the evals came first, the tools fed the model clean information, and the rollout was staged so nobody had to bet the business on day one.

Frequently asked questions

Why narrow the problem to triage instead of full review?

Full review inherits unbounded liability and a fuzzy success criterion, which makes it nearly impossible to ship safely. Triage — sorting and explaining without approving — is bounded, measurable, and keeps humans in control of the consequential decision. Narrowing the scope is usually the difference between a shipped agent and a stalled demo.

Why build the eval set before writing the prompt?

Because without a held-out eval set you cannot tell whether a change to your skill or prompt actually helped. Capturing historical human decisions as ground truth first lets engineers iterate against an objective score instead of arguing from intuition, which is what makes confident, fast iteration possible.

When should this project have gone multi-agent?

Only if the evals showed a single agent couldn't handle the task — for instance if contracts grew so long or complex that one context couldn't hold them. Multi-agent orchestration multiplies token cost several times over, so it should be adopted to solve a proven limitation, not as a default architecture.

What was the most valuable engineering decision in the build?

Reshaping the tools and skills so the model received information in exactly the right form — segmented text with stable clause IDs, a playbook with explicit acceptable-and-unacceptable examples. The model was rarely the bottleneck; the scaffolding around it was, and fixing the interfaces fixed the accuracy.

Bringing agentic AI to your phone lines

CallSphere applies this same end-to-end discipline to voice and chat — narrowly scoped agents that answer every call, use tools mid-conversation, and route the cases that need a human, all in production. See a live build at callsphere.ai.

Source & attribution: This is an independent, original explainer inspired by Anthropic's coverage on the Claude blog. Claude, Claude Code, Claude Cowork, Claude Opus, and the Model Context Protocol are products and trademarks of Anthropic. CallSphere is not affiliated with or endorsed by Anthropic.