SIPSorcery for .NET AI Voice Agents in 2026: SIP, WebRTC, and OpenAI

SIPSorcery has been the.NET telephony library for over a decade, but the AGPL classes scared off commercial teams. As of January 16 2026 those classes are gone, and the companion SIPSorcery.OpenAI.WebRTC package gives you a 50-line console app that connects a microphone to OpenAI Realtime over WebRTC. For.NET shops it is the cleanest AI voice path in 2026.

Background

SIPSorcery is a.NET Standard 2.0 library covering SIP, RTP, SDP, STUN, ICE, DTLS, SRTP, and WebRTC. It is platform-agnostic and runs anywhere.NET runs (Windows, Linux, macOS, embedded ARM). The latest NuGet release is 10.0.6 (April 24 2026) with 1.33 million total downloads.

The SIPSorcery.OpenAI.WebRTC package (8.0.x preview as of mid-2026) provides helper classes specifically for OpenAI's Realtime WebRTC endpoint: WebRTCEndPoint negotiates the peer connection, DataChannelMessenger sends session updates and tool call results, and bundled samples demonstrate microphone-to-OpenAI, two-bot dialog, and SIP-to-OpenAI gateway scenarios.

The two paired implementations (SIPSorceryMedia.Windows for MediaCapture/NAudio and SIPSorceryMedia.FFmpeg for cross-platform) handle codec encode/decode and audio device IO so you do not have to.

Architecture

graph LR
 A[SIP Trunk] --> B[SIPSorcery SIPUserAgent]
 B --> C[RTPSession]
 C --> D[SIPSorceryMedia Audio]
 D --> E[WebRTCEndPoint]
 E -->|Opus over WebRTC| F[OpenAI Realtime]
 F -->|data channel| E
 F -->|Opus back| E
 E --> D
 D --> C
 C --> B
 B --> A

// SIPSorcery.OpenAI.WebRTC starter
using SIPSorcery.OpenAI.WebRTC;
var endpoint = new WebRTCEndPoint(openAiKey, logger);
endpoint.OnDataChannelMessage += (msg) => {
 Console.WriteLine($"AI says: {msg}");
};
await endpoint.InitWindowsAudioDevicesAsync();
await endpoint.StartAsync();
// session is live; speak into the default mic, listen on default speaker

CallSphere implementation

Explore a live demo and compare current plans to find the right fit for your business.

Build steps

1. dotnet new console -n MyAIVoice && cd MyAIVoice.
2. dotnet add package SIPSorcery && dotnet add package SIPSorcery.OpenAI.WebRTC --prerelease.
3. Add Microsoft.Extensions.Logging and a console logger.
4. Read OpenAI API key from environment; never hardcode.
5. Instantiate WebRTCEndPoint, hook OnDataChannelMessage and OnPeerConnectionStateChange.
6. Call InitWindowsAudioDevicesAsync on Windows, or ConfigureFFmpegAudioAsync on Linux/macOS.
7. For SIP integration, use the GetStartedSIP sample as a base; create a SIPUserAgent, register to your SIP server, on incoming call pass the RTP audio into WebRTCEndPoint.

Pitfalls

• The pre-2026 NuGet versions had AGPL classes; pin to >=10.0.0 to stay BSD-clean.
• SIPSorceryMedia.Windows uses MediaCapture which has UWP heritage; on plain Win32 some methods need MTA threading.
• FFmpeg variant requires the FFmpeg shared libs in PATH; mismatched architectures (x86 vs x64) silently fail to load.
• Opus codec negotiation on.NET ARM64 was buggy until 9.0.5; upgrade if you target Apple Silicon or Linux ARM.
• Data channel messages from OpenAI are JSON; deserialize into the typed event records exposed by the helper, not raw strings.

FAQ

Why is the AGPL removal a big deal? Closed-source commercial products could not legally use the previous SIPSorcery without releasing source. Post-January 2026 it is BSD-style and safe.

Does SIPSorcery support Opus? Yes through the SIPSorceryMedia.FFmpeg or the Opus.NET wrapper. Required for OpenAI Realtime WebRTC.

Can I run this on Linux? Yes with.NET 8 and the FFmpeg media stack. SIPSorcery itself is platform-agnostic.

SIPSorcery vs LiveKit Agents.NET? LiveKit's.NET SDK targets WebRTC rooms; SIPSorcery is a SIP-first library that also speaks WebRTC. Pick based on whether you start from a phone number or from a browser.

Does it handle DTMF? Yes via RFC 2833 RTP events and SIP INFO. The library decodes both and exposes events on the user agent.

Sources

• SIPSorcery on GitHub
• SIPSorcery.OpenAI.WebRTC repository
• SIPSorcery NuGet package 10.0.6
• SIPSorcery documentation

Explore a live demo and compare current plans to find the right fit for your business.

SIPSorcery for.NET AI Voice Agents in 2026: SIP, WebRTC, and OpenAI: production view

SIPSorcery for.NET AI Voice Agents in 2026: SIP, WebRTC, and OpenAI usually starts as an architecture diagram, then collides with reality the first week of pilot. You discover that vector store choice (ChromaDB vs. Postgres pgvector vs. managed) is not really a vector store choice — it's a latency, freshness, and ops choice. Picking wrong forces a re-platform six months in, exactly when you have customers depending on it.

Shipping the agent to production

Production AI agents live or die on three loops: evals, retries, and handoff state. CallSphere runs 37 agents across 6 verticals, each with its own eval suite — synthetic call transcripts replayed nightly with assertion checks on extracted entities (date, time, party size, insurance, address). Without that loop, prompt regressions ship silently and you only find out when bookings drop.

Structured tools beat free-form text every time. Our 90+ function tools all enforce JSON schemas validated server-side; if the model hallucinates an integer where a string is required, we retry with a corrective system message before falling back to a deterministic path. For long-running flows, we treat agent handoffs as a state machine — booking → confirmation → SMS — so context survives turn boundaries.

The Realtime API vs. async decision usually comes down to "is the user holding the phone right now?" If yes, Realtime; if no (callback queue, after-hours voicemail), async wins on cost-per-conversation, which we track per agent in 115+ database tables spanning all 6 verticals.

FAQ

Is this realistic for a small business, or is it enterprise-only? The healthcare stack is a concrete example: FastAPI + OpenAI Realtime API + NestJS + Prisma + Postgres healthcare_voice schema + Twilio voice + AWS SES + JWT auth, all HIPAA aligned. For a topic like "SIPSorcery for.NET AI Voice Agents in 2026: SIP, WebRTC, and OpenAI", that means you're not starting from scratch — you're configuring an agent template that's already been hardened across thousands of conversations.

Which integrations have to be in place before launch? Day one is integration mapping (scheduler, CRM, messaging) and prompt tuning against your top 20 real call transcripts. Day two through five is shadow-mode running, where the agent transcribes and recommends but a human still answers, so you can compare side-by-side. Go-live is the moment your eval pass-rate clears your internal bar.

How do we measure whether it's actually working? The honest answer: it scales until your tool catalog gets stale. The agent is only as good as the integrations it can actually call, so the operational discipline is keeping schemas, webhooks, and fallback paths green. The platform handles the rest — observability, retries, multi-region routing — without your team owning the GPU layer.

Talk to us

Explore a live demo and compare current plans to find the right fit for your business.