Your AI Feature Took Gross Margin From 82% to 74%. Draw the Line Between a Routine Request and a Hard One.

You already tried this. Some time in 2025 someone suggested moving the AI feature in your product to a cheaper model, you tried it for a fortnight, the summaries got noticeably worse, two customers complained, and you moved everything back to the expensive one and stopped talking about it. That was the correct decision with the information you had. Here is what is actually different now.

What was missing then was not a cheaper model. It was a defensible way to decide, request by request, which model each one deserves — and a way to see afterwards whether the decision was right. Cisco is rolling a personal AI agent out to roughly 90,000 employees with exactly this built in: routine requests to a fast, cheap model, and only the genuinely hard ones escalated. The mechanism is not exotic. What is new is that it is now normal practice with tooling behind it, rather than something you had to hand-build.

Where the margin actually went

Look at your gross margin line over the last six quarters. If you shipped an AI feature in 2025 there is a good chance it went from something in the low 80s to something in the mid 70s, and that the drop is sitting in cost of revenue next to hosting. Nobody in the company panicked, because the feature drove upgrades and the absolute dollars looked small. Then you started preparing a raise, or a renewal conversation with your largest customer, and someone asked why your Rule of 40 got worse while growth stayed flat.

The uncomfortable part is that the cost scales with usage while the price does not. You sell a seat. The customer's operations manager discovers your account summariser and runs it forty times a day. That is a good problem in every respect except the one that shows up in the finance review.

Model routing means sending each request to the cheapest model that can handle it, and escalating only the ones that genuinely need more — decided by a written rule you can inspect, not by whoever set it up last.

Routing in plain terms

Every request coming into your feature gets sorted before it is answered. A short, well-defined job — summarise one record with five fields, classify an incoming support message, extract the renewal date from an order form — goes to a fast cheap model and comes back in under a second. A long, ambiguous job — read forty records and a contract and tell me whether this account is at risk — goes to the strong model, takes longer, and costs more, and that is fine because it is rare.

The 2026 additions that make this manageable rather than a science project are unglamorous. Anthropic's enterprise governance update on 2 July 2026 added a cost and usage dashboard, spend limits at the organisation and user level, alerts at 75% and 90% of budget, and the ability to set which model a given team gets by default. That is what turns "we should route" into something your controller can actually watch. Meanwhile frontier AI costs have come down roughly tenfold since 2025, which means the cheap end is now good enough to carry the volume that used to force you back to the expensive model.

flowchart TD
  A["Customer clicks Summarise this account"] --> B{"Which bucket?"}
  B -->|"Routine: one record, five fields"| C["Fast cheap model answers in under a second"]
  B -->|"Hard: 40 records plus the contract"| D["Strong model, longer run"]
  B -->|"Restricted: health or card data"| E["Runs on servers we control"]
  C --> F{"Did it hedge or come back thin?"}
  F -->|Yes| D
  F -->|No| G["Answer returned, cost logged"]
  D --> G
  E --> G

What counts as routine in B2B software

This is the part only you can answer, and it is a business question rather than a technical one. In practice, four tests separate routine from hard in this trade, and they hold up across most products.

• How many records does the answer depend on? One record is routine. One account's entire history — tickets, usage, invoices, the order form — is hard.
• Is there a right answer? "Which plan is this customer on" has one. "Is this account at risk of churning" does not, and a cheap model will produce something confident and shallow.
• Who reads it? A summary that a user glances at inside your product is routine. Anything that leaves your product — a report emailed to the customer's board, a renewal brief a salesperson reads out loud — is hard.
• What does being wrong cost? A slightly clumsy ticket classification costs nothing. A wrong number in a usage report a customer forwards to their finance team costs a conversation you do not want.

Write those four lines down, apply them to the ten things your feature actually does, and you will usually find that seventy to eighty percent of requests are unambiguously routine.

Who draws the line, and who checks it monthly

Not the engineer who set it up. The line is a commercial decision with an engineering implementation, and it should be owned by three people meeting for thirty minutes a month: whoever owns the product area, your head of support, and your controller.

The product owner brings quality — which requests came back thin and had to be rerun. Support brings the complaints, because if routing is too aggressive the evidence arrives as tickets saying "the summary used to be better". The controller brings the cost dashboard and the spend alerts. Thirty minutes, one decision: does the line move, and in which direction. Write the current rule in a shared document with a date on it. When a customer eventually asks how your product decides what to use — and enterprise security reviewers do ask this in 2026 — you hand them a page rather than improvising.

The arithmetic on 180,000 requests a month

Assumptions, all illustrative and easy to substitute: your feature serves 180,000 requests a month; a strong-model answer costs you $0.061 all in; a cheap-model answer costs $0.004; 74% of requests test as routine; and 6% of the routine ones come back thin and get rerun on the strong model.

On $9.1M of recurring revenue that is a little under one point of gross margin — 0.94 to be precise about an illustration. One point of margin does not save a company, but it is a point you get back without touching price, headcount or the product, and it compounds as usage grows. Note also the honest line in the table: the reruns are a real cost, and any vendor's routing pitch that leaves them out is selling you a number that will not survive contact with your invoice.

The requests that never get routed down

Three categories should be pinned to the strong model permanently, regardless of what the cost dashboard says.

Anything a customer will forward. Executive summaries, quarterly business review material, anything with a number in it that lands in front of the customer's own finance team. The saving on those is trivial because the volume is low, and the downside is a correction email.

Anything touching regulated data or a compliance answer. If your product handles protected health information under a business associate agreement, or card data, the routing decision has to respect where the data may be processed — and for some customers that is a contractual commitment, not a preference. Cisco's rollout leans on running things on their own premises for exactly this reason, and a smaller company can achieve the same by simply refusing to route certain categories at all.

And anything during a paid trial or an onboarding. A prospect's first two weeks with your product decides the deal. Spend the money there. You can be economical with the 200th summary a happy customer runs; be generous with their first.

Frequently asked questions

Will customers notice the difference?

On genuinely routine work, no — and the cheap end of 2026 is markedly better than the cheap end of 2025, which is why the 2025 experiment failed and this one usually does not. You will find out either way from your support queue, which is why the head of support sits in the monthly review.

How do I know what percentage of my requests are routine before I build anything?

Take a week of logged requests, sample 200 at random, and have a product manager sort them by hand against the four tests. Half a day of work gives you the split, and it is a far better basis for the decision than a vendor's estimate.

Should I just raise the price of the AI feature instead?

Often yes, and the two are not alternatives. But usage-based cost against seat-based price is a structural mismatch that pricing alone does not fix, and repricing an existing customer base takes a renewal cycle. Routing takes a sprint.

Does routing hurt if we ever want to change providers?

It helps. Once you have a written rule for what counts as routine, swapping which model serves the routine bucket is a contained change you can test on your own past requests, rather than a whole-product migration decided on a hunch.

The half-day version

Before you change any code, sample 200 requests from last week and sort them into routine and hard by hand. If the split is 70/30 or better, the arithmetic above is worth running on your real numbers, and the meeting to schedule is the monthly one with your product owner, support lead and controller in it.

Worth noting where the same logic already applies outside your product: your inbound phone and web chat get the same mix, where most contacts are simple and a few are the deal. CallSphere builds AI voice and chat agents that answer business lines and website chat, book demos, and capture leads at any hour, handing the ones that need a person straight to your team — the same routine-versus-hard line, drawn at the front door.