Build an AI Voice Agent with SvelteKit + WebRTC + OpenAI Realtime (2026)

TL;DR — Svelte 5 runes ($state, $derived) plus SvelteKit 2 form actions make WebRTC voice agents 30-40% smaller than the React equivalent. OpenAI ephemeral keys make browser-direct WebRTC safe to expose.

What you'll build

A SvelteKit page where a button mounts a WebRTC peer connection to OpenAI Realtime, the audio plays through a hidden <audio> element, and live transcripts stream into a runes-driven UI.

Prerequisites

1. @sveltejs/kit@^2.5, svelte@^5, Vite 5+.
2. OPENAI_API_KEY server-side.
3. Node 20+ or Bun 1.3.

Architecture

flowchart LR
  UI[Svelte 5 page] -->|form action| SK[SvelteKit /api/key]
  SK -->|POST /v1/realtime/sessions| OA1[OpenAI sessions]
  OA1 --> SK --> UI
  UI -- WebRTC SDP --> OA2[OpenAI Realtime]

Step 1 — Server: mint ephemeral key

```ts // src/routes/api/key/+server.ts import { OPENAI_API_KEY } from "$env/static/private"; import { json } from "@sveltejs/kit";

export async function POST() { const r = await fetch("https://api.openai.com/v1/realtime/sessions", { method: "POST", headers: { Authorization: Bearer ${OPENAI_API_KEY}, "Content-Type": "application/json" }, body: JSON.stringify({ model: "gpt-realtime", voice: "verse" }), }); return json(await r.json()); } ```

Step 2 — Svelte 5 page with runes

```svelte

{live ? "Live" : "Talk"}

{transcript}

```

Step 3 — Add tool calls

Send tool definitions on session.update via dc.send(...). When the model emits response.function_call_arguments.done, run your tool and reply with conversation.item.create.

Step 4 — Deploy

@sveltejs/adapter-vercel deploys both the SSR routes and /api/key to Vercel functions.

Step 5 — Bundle size

The voice page weighs ~32kb gzipped (vs ~52kb for the React equivalent), thanks to runes' compiled-out reactivity.

Pitfalls

• Svelte 4 $: reactivity doesn't carry over to runes — pick one and stay consistent.
• bind:this timing: Audio element is null in onMount if you reference it too early.
• Ephemeral key TTL: Default 60s — mint right before createOffer.

How CallSphere does this in production

Explore a live demo and compare current plans to find the right fit for your business.

FAQ

Svelte 5 stable? Yes — GA October 2024, ~5M weekly downloads by 2026.

SvelteKit 2 + Vite 5? Vite 5+ is required; Vite 6 supported in 2.5+.

SSR for the voice page? Use +page.svelte with ssr = false — WebRTC needs the browser.

Tool streaming? Yes — Realtime data channel emits tool deltas just like JSON events.

Sources

• SvelteKit docs - https://kit.svelte.dev/
• Svelte 5 runes - https://svelte.dev/docs/svelte/what-are-runes
• OpenAI Realtime WebRTC - https://developers.openai.com/api/docs/guides/realtime-webrtc
• ForaSoft Voice Agents 2026 - https://www.forasoft.com/blog/article/openai-realtime-api-voice-agent-production-guide-2026

How this plays out in production

One layer below what Build an AI Voice Agent with SvelteKit + WebRTC + OpenAI Realtime (2026) covers, the practical question every team hits is multi-turn handoffs between specialist agents without losing slot state, sentiment, or escalation context. Treat this as a voice-first system from the first prompt: the agent's persona, its tool surface, and its escalation rules all flow from that single decision. Teams that ship fast tend to instrument the loop end-to-end before they tune any single component, because the bottleneck is rarely where intuition puts it.

Voice agent architecture, end to end

A production-grade voice stack at CallSphere stitches Twilio Programmable Voice (PSTN ingress, TwiML, bidirectional Media Streams) to a realtime reasoning layer — typically OpenAI Realtime or ElevenLabs Conversational AI — with sub-second response as a hard SLO. Anything north of one second of perceived silence and callers either repeat themselves or hang up; that single number drives the whole architecture. Server-side VAD with proper barge-in support is non-negotiable, otherwise the agent talks over the caller and the conversation collapses. Streaming TTS with phoneme-aligned interruption keeps the cadence natural even when the user changes their mind mid-sentence. Post-call, every transcript is run through a structured pipeline: sentiment, intent classification, lead score, escalation flag, and a normalized slot extraction (name, callback number, reason, urgency). For healthcare workloads, the BAA-covered storage path, audit logs, encryption-at-rest, and PHI-safe transcript redaction are wired in from day one, not bolted on at compliance review. The end state is a system where every call produces a row of structured data, not just a recording.

FAQ

What is the fastest path to a voice agent the way Build an AI Voice Agent with SvelteKit + WebRTC + OpenAI Realtime (2026) describes?

Treat the architecture in this post as a starting point and instrument it before you tune it. The metrics that matter most early on are end-to-end latency (target < 1s for voice, < 3s for chat), barge-in correctness, tool-call success rate, and post-conversation lead score distribution. Optimize whatever the data flags as the bottleneck, not whatever feels slowest in your head.

What are the gotchas around voice agent deployments at scale?

The two failure modes that bite hardest are silent context loss across multi-turn handoffs and tool calls that succeed in dev but get rate-limited in production. Both are solvable with a proper agent backplane that pins state to a session ID, retries with backoff, and writes every tool invocation to an audit log you can replay.

What does the CallSphere outbound sales calling product do that a regular dialer does not?

It uses the ElevenLabs "Sarah" voice, runs up to 5 concurrent outbound calls per operator, and ships with a browser-based dialer that transfers warm calls back to a human in one click. Dispositions, transcripts, and lead scores write back to the CRM automatically.

See it live

Book a 30-minute working session at calendly.com/sagar-callsphere/callsphere-llc-meeting and bring a real call flow — we will walk it through the live outbound sales dialer at callsphere.ai/demo and show you exactly where the production wiring sits.