Twilio Conversations for Omnichannel AI: Voice, SMS, WhatsApp, RCS (2026)

TL;DR — Twilio Conversations + Conversation Orchestrator + Conversation Memory (GA May 2026) let one AI agent track a customer across Voice, SMS, WhatsApp, RCS, and Chat. Use ConversationSid as your AI's session key and stop building bespoke channel glue.

Background

At SIGNAL 2026 Twilio made Conversation Memory, Conversation Orchestrator, Conversation Intelligence, and Agent Connect generally available. Together they solve the "every channel feels like a cold start" problem. Conversation Orchestrator becomes the single routing brain across Voice, SMS, WA, RCS, Chat.

Architecture / config

flowchart TD
 V[Voice] --> ORC[Conversation Orchestrator]
 S[SMS] --> ORC
 W[WhatsApp] --> ORC
 R[RCS] --> ORC
 C[Chat] --> ORC
 ORC --> MEM[Conversation Memory]
 ORC --> AI[Your AI runtime via Agent Connect]
 AI --> ORC
 ORC -->|escalate| FLEX[Flex / human queue]

CallSphere implementation

CallSphere uses Twilio across all products to power omnichannel for Healthcare, Salon, Sales, Hospitality, Real Estate, and After-hours:

• Inbound voice → <Connect><Stream/></Connect> → FastAPI :8084 → OpenAI Realtime.
• AI tool send_followup_sms posts a Message into the same Conversation; Memory carries the transcript.
• WhatsApp inbound joins the same Conversation when phone numbers match.
• After-hours simul-call+SMS in 120 s — both legs share ConversationSid.

Explore a live demo and compare current plans to find the right fit for your business.

Build steps with code

// Find or create a Conversation keyed on customer phone
const conv = await twilio.conversations.v1.conversations.create({
 friendlyName: `cust:${phone}`,
 uniqueName: `cust:${phone}`,
});

// Add channel-specific addresses as participants
await twilio.conversations.v1.conversations(conv.sid).participants.create({
 "messagingBinding.address": phone,
 "messagingBinding.proxyAddress": twilioNumber,
});

// On voice answer, attach a custom attribute so memory knows the leg
await twilio.conversations.v1.conversations(conv.sid).update({
 attributes: JSON.stringify({ active_voice_call: callSid }),
});

// AI tool — send omnichannel follow-up
async function sendFollowup({ conversationSid, body, channel }) {
 return twilio.conversations.v1.conversations(conversationSid).messages.create({
 body,
 author: "ai-agent",
 attributes: JSON.stringify({ channel })
 });
}

Pitfalls

• Conversations (classic) vs Orchestrator — new builds use Orchestrator; classic API stays for back-compat.
• WhatsApp 24-hour window — outside it, you must use templates. Memory survives; sending does not.
• RCS sender registration — required, takes 2–5 days; do not promise launch dates.
• Memory PII drift — set retention TTL (e.g., 30 d for Healthcare) and field-level redaction.
• Cross-channel attribution — you need a single customer_id linked to phone + WhatsApp ID + chat user, or Memory creates duplicates.

FAQ

Q: Does Memory persist across Twilio accounts? No — scoped to one account. Use Segment as the cross-account identity layer.

Q: How do I expose Memory to my LLM? Pull last N turns from Conversations.Messages and inject into system prompt; or use Conversation Intelligence custom operators.

Q: Pricing? Memory + Orchestrator are usage-billed (per active session). Add to per-channel cost.

Q: Can I run a non-Twilio AI? Yes — Agent Connect is model-agnostic; we use OpenAI in CallSphere.

Q: Best channel mix for Healthcare? Voice + SMS for confirmations + WhatsApp for international patients.

Sources

• Twilio — SIGNAL 2026 Announcements
• Twilio — Conversation Orchestrator GA
• Twilio Conversations docs
• Twilio Blog — Conversation Intelligence launch

Twilio Conversations for Omnichannel AI: Voice, SMS, WhatsApp, RCS (2026): production view

Twilio Conversations for Omnichannel AI: Voice, SMS, WhatsApp, RCS (2026) sounds like a single decision, but in production it splits into eval design, prompt cost, and observability. The deeper you push toward live traffic, the more those three pull against each other — better evals catch silent failures, prompt cost limits how often you can re-run them, and weak observability hides which retries are actually saving conversations versus burning latency budget.

Serving stack tradeoffs

The big fork is managed (OpenAI Realtime, ElevenLabs Conversational AI) versus self-hosted on GPUs you operate. Managed wins on cold-start, model freshness, and zero-ops; self-hosted wins on unit economics past a certain conversation volume and on data residency for regulated verticals. CallSphere runs hybrid: Realtime for live calls, self-hosted Whisper + a hosted LLM for async, both routed through a Go gateway that enforces per-tenant rate limits.

Latency budgets are non-negotiable on voice. End-to-end target is sub-800ms ASR-to-first-token and sub-1.4s first-audio-out; anything beyond that and turn-taking feels stilted. GPU residency in the same region as your TURN servers matters more than choosing a slightly bigger model.

Observability is the unglamorous backbone — every conversation produces logs, traces, sentiment scoring, and cost attribution piped to a per-tenant dashboard. HIPAA aligned isolation keeps healthcare traffic separated from salon traffic at the storage layer, not just the API.

FAQ

What's the right way to scope the proof-of-concept? CallSphere runs 37 production agents and 90+ function tools across 115+ database tables in 6 verticals, so most workflows you'd want already have a template. For a topic like "Twilio Conversations for Omnichannel AI: Voice, SMS, WhatsApp, RCS (2026)", that means you're not starting from scratch — you're configuring an agent template that's already been hardened across thousands of conversations.

How do you handle compliance and data isolation? Day one is integration mapping (scheduler, CRM, messaging) and prompt tuning against your top 20 real call transcripts. Day two through five is shadow-mode running, where the agent transcribes and recommends but a human still answers, so you can compare side-by-side. Go-live is the moment your eval pass-rate clears your internal bar.

When does it make sense to switch from a managed model to a self-hosted one? The honest answer: it scales until your tool catalog gets stale. The agent is only as good as the integrations it can actually call, so the operational discipline is keeping schemas, webhooks, and fallback paths green. The platform handles the rest — observability, retries, multi-region routing — without your team owning the GPU layer.

Talk to us

Explore a live demo and compare current plans to find the right fit for your business.