By Sagar Shankaran, Founder of CallSphere
Direct OpenAI Realtime + Agents SDK = thinner stack vs Vapi's vendor middleware layer. CallSphere ships voice agents in days, not sprints.
Key takeaways
CallSphere targets the OpenAI Realtime API directly and orchestrates agents with the OpenAI Agents SDK. There is no third-party voice middleware in the critical path. Vapi.ai is itself a middleware layer — a useful one, but a layer nonetheless that sits between your code and the underlying STT, LLM, and TTS providers. When OpenAI ships a new Realtime feature (interruption handling, async tool execution, native multi-language support), CallSphere can adopt it the day it lands. Vapi customers wait for Vapi to integrate it. This time-to-feature gap matters when the underlying APIs are evolving every month.
Every voice AI stack has the same five layers:
Until late 2024, you wired up each layer separately. STT from Deepgram, LLM from OpenAI, TTS from ElevenLabs. Each round trip cost 100-300ms. Total latency hovered around 2 seconds.
The OpenAI Realtime API collapsed STT + LLM + TTS into one streaming endpoint. Latency floor dropped to <1 second. The thin pipeline became viable.
Vapi sits as middleware over the same providers. You configure your STT (Deepgram, AssemblyAI), your LLM (OpenAI, Anthropic, custom), and your TTS (ElevenLabs, PlayHT, Cartesia). Vapi orchestrates the round trip and provides the telephony, VAD, barge-in, and webhook glue.
This works well when you want flexibility. It strains when:
CallSphere's stack is intentionally thinner:
There is no commercial voice-AI middleware between our code and OpenAI. When OpenAI ships an update, we read the changelog and have it in production within a sprint.
Hear it before you finish reading
Talk to a live CallSphere AI voice agent in your browser — 60 seconds, no signup.
graph TD
subgraph Vapi
A1[Caller] --> B1[Vapi Telephony]
B1 --> C1[Vapi VAD + Orchestrator]
C1 --> D1[STT Provider]
D1 --> E1[LLM Provider]
E1 --> F1[TTS Provider]
F1 --> B1
C1 --> G1[Vapi Webhook]
G1 --> H1[Your Backend]
end
subgraph CallSphere
A2[Caller] --> B2[Twilio or WebRTC Gateway]
B2 --> C2[CallSphere Voice Server]
C2 --> D2[OpenAI Realtime API<br/>STT + LLM + TTS combined]
D2 --> E2[Agents SDK Orchestrator]
E2 --> F2[Function-Calling Tools]
F2 --> G2[Your Backend / Postgres]
D2 --> C2
end
The Vapi path has six external boundaries between the caller and the backend. CallSphere has three. Fewer boundaries means fewer places where versions can drift, fewer round trips, and faster feature adoption.
When OpenAI shipped gpt-4o-realtime-preview-2025-06-03 with improved interruption handling, CallSphere updated the model string in one config file and redeployed. Total time: 30 minutes. Customers saw the improvement the same day.
When OpenAI added async tool execution support, CallSphere migrated the Healthcare 14-tool flow to async pattern in one week. The flow's 95th percentile latency dropped 280ms.
A Vapi customer waiting for the same async tool support waits for Vapi engineering to roll it out, document it, and surface the config knobs. Sometimes that's one sprint. Sometimes it's three.
| Capability | CallSphere (Direct Realtime) | Vapi (Middleware) |
|---|---|---|
| Layers between your code and the model | 1 (Agents SDK) | 3+ (Vapi orchestrator, STT/LLM/TTS providers) |
| Time to adopt new OpenAI Realtime feature | Days | Weeks-to-months |
| Raw frame inspection | Yes | Limited |
| Token-level latency telemetry | Yes | Limited |
| Multi-provider LLM flexibility | OpenAI-centered, Anthropic via Agents SDK | Yes, native |
| Vendor lock-in | OpenAI Realtime + Agents SDK | Vapi platform |
| Pricing model | OpenAI usage + infra | Platform fee + per-minute markup |
The flexibility tradeoff is real: Vapi makes it easier to swap LLM providers because that's its core value proposition. CallSphere bets on OpenAI's Realtime API as the reference platform and ships against it directly.
The Agents SDK shines because it has direct access to the Realtime session. Handoffs, tool calls, and session state live inside one process. When Vapi orchestrates handoffs (via Squads), the orchestrator sits outside the model session, so handoff latency and state-sharing complexity grow.
Concrete example: CallSphere Real Estate runs 10 specialist agents behind a triage agent. A buyer call routes Triage → Property Search → Buyer Lead → Mortgage Pre-Qual → Tour Scheduling → Triage in one session. The Agents SDK manages session state across handoffs natively. There is no out-of-process orchestrator to coordinate.
Doing the same on Vapi requires Squads or external state coordination. Possible — but more code, more failure modes.
from openai import AsyncOpenAI
from agents import Agent, Runner
triage = Agent(
name="Triage",
model="gpt-4o-realtime-preview-2025-06-03",
instructions="Route caller to the right specialist.",
handoffs=[buyer_lead, seller_lead, property_search],
)
async def handle_call(call_id: str):
runner = Runner(agent=triage)
async for event in runner.stream(call_id):
...
That is roughly the shape of a CallSphere triage entry point. The Agents SDK takes care of handoffs, state passing, and tool dispatch. We add CallSphere-specific guardrails and analytics on top.
Vapi wins on speed in one important case: when you're a small team that wants a working voice agent in a weekend without learning any OpenAI internals. Their developer experience is excellent. Their docs are crisp. The first call literally takes 10 minutes.
CallSphere is a deeper investment up front for verticals (Real Estate, Healthcare, Salon, IT, Sales, After-Hours) where the agent topology matters more than the time-to-first-call.
Still reading? Stop comparing — try CallSphere live.
CallSphere ships complete AI voice agents per industry — 14 tools for healthcare, 10 agents for real estate, 4 specialists for salons. See how it actually handles a call before you book a demo.
Primarily yes for voice. The Agents SDK can route to Anthropic Claude for non-voice text reasoning and analytics flows, but the Realtime voice path is OpenAI-native. This is a deliberate bet on the leading realtime API.
We monitor the price/quality curve and would re-evaluate against Anthropic, Google, and any open-source realtime model that matches latency. Our architecture is portable because we own the orchestration layer.
We fall back to a cascaded pipeline (Whisper STT + GPT-4o + ElevenLabs TTS) for graceful degradation. The Agents SDK abstraction lets us swap the underlying model without changing agent logic.
Yes. It has been stable through several OpenAI releases. The CallSphere team contributes patterns and fixes upstream where appropriate.
If you need a voice agent for a single workflow in a weekend, Vapi. If you need a vertical-grade stack (Healthcare with 14 tools, Real Estate with 10 agents and vision, multi-language at scale), CallSphere.
A thin stack is not a free lunch. There are real costs to the choice CallSphere made.
Vendor concentration. Betting on the OpenAI Realtime API as the voice backbone means our voice quality and capability are tied to OpenAI's roadmap. If they slow shipping, we slow shipping. We mitigate by maintaining a fallback cascaded pipeline (Whisper STT, GPT-4o, ElevenLabs TTS) that can absorb a Realtime API outage with degraded but functional service.
Less LLM diversity. Vapi's strength is BYO LLM — Anthropic, OpenAI, custom — for non-voice paths. Our voice path is OpenAI-first. Non-voice agent reasoning can route through Anthropic via the Agents SDK, but the realtime conversational layer is OpenAI-centered today.
More platform engineering. Customers don't see this, but the team does. We maintain our own voice servers, our own gateways, our own observability. Vapi customers don't. This is fine for us because we're an agentic AI company, not a voice platform consumer; for a non-AI product team, the calculus could go the other way.
The velocity difference between a thin stack and a middleware layer doesn't matter if you ship one voice agent and never touch it again. It matters enormously if you are iterating weekly on agent quality, exploring new tool patterns, or shipping vertical-specific features that require new Realtime API capabilities the day they land.
CallSphere's bet is that voice AI in 2026 looks more like a continuously evolving product than a configured workflow. Verticals get richer, tools multiply, agent topologies grow. Teams that own their stack iterate faster. Teams that depend on a middleware vendor wait for the vendor's roadmap. Both can ship customer value; only one compounds.
See the thin stack in action. Book a demo or browse Healthcare and Real Estate.
Written by
Sagar Shankaran· Founder, CallSphere
LinkedInSagar Shankaran is the founder of CallSphere, where he builds production AI voice and chat agents deployed across healthcare, hospitality, real estate, and home services. He writes about agentic AI, LLM engineering, and shipping voice agents that handle real calls in production.
See how AI voice agents work for your industry. Live demo available -- no signup required.
How CallSphere's 14 healthcare tools are designed: clear naming, idempotency, auth-bound, error contract. Practical patterns for voice AI tool schemas.
OpenAI Realtime dominates production voice AI in 2026. Claude wins on analytics. Here's a task-by-task decision framework from a real voice agent stack.
k3s + hostPath volumes give CallSphere agent hot-reload without redeploys. Vapi customers ship through their pipeline. Engineering velocity matters.
Triage to specialist to return-to-orchestrator pattern explained with code. CallSphere's OpenAI Agents SDK handoffs vs Vapi Squads' linear chain.
Real per-minute cost breakdown for ElevenLabs Conversational AI vs OpenAI Realtime in 2026, with the hidden costs most teams miss.
The three production-grade native speech-to-speech LLMs of 2026, side by side. Latency, prosody quality, function calling, and where each one breaks.
© 2026 CallSphere Inc. All rights reserved.
Made within San Francisco
Watch how CallSphere handles real customer calls, schedules appointments, and processes payments — live.
Try Live DemoBook a DemoCalculate Your ROI