By Sagar Shankaran, Founder of CallSphere
Flink 2.2 adds ML_PREDICT and VECTOR_SEARCH SQL functions, baking remote LLM inference into stream processing. We show how to enrich a call event stream with sentiment, topic, and embeddings — all in one Flink job.
Key takeaways
TL;DR — Flink 2.2 (Dec 2025) ushered in the AI era for stream processing:
ML_PREDICTcalls remote LLMs as a SQL function,VECTOR_SEARCHdoes similarity search inline, and Confluent Cloud Managed Flink scales it without ops. CallSphere uses a single Flink job to enrich call events with sentiment + intent + embedding before they hit ClickHouse.
Stream-then-enrich-then-store is a classic pattern. Pre-2026 you needed a Python service in the middle calling OpenAI. Flink 2.2 collapses that into SQL: SELECT call_id, ML_PREDICT('gpt-4o-mini', transcript) AS sentiment FROM stream. One job, one operator graph, one set of metrics.
The other 2026 win is VECTOR_SEARCH — embedding lookup against a vector store directly inside the Flink job, so you can do "did this caller ask the same question last week?" inline.
flowchart LR
Kafka[(Kafka<br/>call.transcript)] --> Flink[Flink 2.2 job]
Flink -->|ML_PREDICT gpt-4o-mini| LLM[(OpenAI / Bedrock)]
Flink -->|VECTOR_SEARCH| VS[(Vector store)]
Flink -->|enriched| Out[(Kafka<br/>call.enriched)]
Out --> CH[(ClickHouse)]
Out --> Ice[(Iceberg lake)]
A single Flink SQL job pulls from call.transcript, calls the LLM and vector store, and writes call.enriched for downstream sinks.
CallSphere — 37 agents · 90+ tools · 115+ DB tables · 6 verticals. $149 / $499 / $1499 at /pricing. 14-day trial, 22% affiliate. On Healthcare (/industries/healthcare) the Flink enrichment job calls GPT-4o-mini for sentiment (-1.0..1.0) and lead score (0..100) and runs VECTOR_SEARCH against historical transcripts to flag repeat callers. Output flows to ClickHouse + Iceberg. Demo at /demo.
Hear it before you finish reading
Talk to a live CallSphere AI voice agent in your browser — 60 seconds, no signup.
async i/o operators with a 100-call queue.CREATE MODEL gpt_4o_mini
INPUT (transcript STRING)
OUTPUT (sentiment FLOAT, intent STRING, lead_score INT)
WITH (
'provider' = 'openai',
'task' = 'classification',
'model' = 'gpt-4o-mini',
'system_prompt' = 'Score sentiment in [-1,1], detect intent enum, lead score 0-100. JSON.'
);
INSERT INTO call_enriched
SELECT
c.call_id,
c.vertical,
p.sentiment,
p.intent,
p.lead_score
FROM call_transcript AS c,
LATERAL TABLE(ML_PREDICT(MODEL gpt_4o_mini, c.transcript)) AS p;
Flink vs. RisingWave? Flink is more general (Java/Python UDFs, complex event processing); RisingWave is SQL-first with simpler ops. For AI enrichment specifically, Flink 2.2's ML_PREDICT is the cleanest API.
Latency? With async i/o and a 200-row buffer, end-to-end is ~600 ms — fine for analytics, too slow for in-call decisions.
Cost? Confluent Cloud Managed Flink charges per CFU; ML_PREDICT cost is the LLM API call.
Exactly-once? Flink's exactly-once guarantees apply to sink writes; LLM side effects are at-least-once.
Self-host? Yes — Flink Helm chart works; budget for a 3-JM cluster.
Apache Flink 2.2 for AI Conversation Analytics: ML_PREDICT, VECTOR_SEARCH, and Real Use Cases (2026) forces a tension most teams underestimate: agent handoff state. A single LLM call is easy. A booking agent that hands a confirmed slot to a billing agent that hands a follow-up to an escalation agent — that's where context loss, hallucinated IDs, and double-bookings live. Solving it well means treating the conversation as a stateful workflow, not a chat.
Still reading? Stop comparing — try CallSphere live.
CallSphere ships complete AI voice agents per industry — 14 tools for healthcare, 10 agents for real estate, 4 specialists for salons. See how it actually handles a call before you book a demo.
The big fork is managed (OpenAI Realtime, ElevenLabs Conversational AI) versus self-hosted on GPUs you operate. Managed wins on cold-start, model freshness, and zero-ops; self-hosted wins on unit economics past a certain conversation volume and on data residency for regulated verticals. CallSphere runs hybrid: Realtime for live calls, self-hosted Whisper + a hosted LLM for async, both routed through a Go gateway that enforces per-tenant rate limits.
Latency budgets are non-negotiable on voice. End-to-end target is sub-800ms ASR-to-first-token and sub-1.4s first-audio-out; anything beyond that and turn-taking feels stilted. GPU residency in the same region as your TURN servers matters more than choosing a slightly bigger model.
Observability is the unglamorous backbone — every conversation produces logs, traces, sentiment scoring, and cost attribution piped to a per-tenant dashboard. HIPAA + SOC 2 aligned isolation keeps healthcare traffic separated from salon traffic at the storage layer, not just the API.
How does this apply to a CallSphere pilot specifically?
Real Estate runs as a 6-container pod (frontend, gateway, ai-worker, voice-server, NATS event bus, Redis) backed by Postgres realestate_voice with row-level security so multi-tenant data never crosses tenants. For a topic like "Apache Flink 2.2 for AI Conversation Analytics: ML_PREDICT, VECTOR_SEARCH, and Real Use Cases (2026)", that means you're not starting from scratch — you're configuring an agent template that's already been hardened across thousands of conversations.
What does the typical first-week implementation look like? Day one is integration mapping (scheduler, CRM, messaging) and prompt tuning against your top 20 real call transcripts. Day two through five is shadow-mode running, where the agent transcribes and recommends but a human still answers, so you can compare side-by-side. Go-live is the moment your eval pass-rate clears your internal bar.
Where does this break down at scale? The honest answer: it scales until your tool catalog gets stale. The agent is only as good as the integrations it can actually call, so the operational discipline is keeping schemas, webhooks, and fallback paths green. The platform handles the rest — observability, retries, multi-region routing — without your team owning the GPU layer.
Want to see how this maps to your stack? Book a live walkthrough at calendly.com/sagar-callsphere/new-meeting, or try the vertical-specific demo at salon.callsphere.tech. 14-day trial, no credit card, pilot live in 3–5 business days.
Written by
Sagar Shankaran· Founder, CallSphere
Sagar Shankaran is the founder of CallSphere, where he builds production AI voice and chat agents deployed across healthcare, hospitality, real estate, and home services. He writes about agentic AI, LLM engineering, and shipping voice agents that handle real calls in production.
See how AI voice agents work for your industry. Live demo available -- no signup required.
Beyond single-shot RAG — agentic RAG with LangGraph that re-retrieves, self-grades, and rewrites queries. With evals that catch silent retrieval drift.
Build a production RAG agent with LangChain, then measure faithfulness, answer relevance, and context precision with RAGAS. The four metrics that matter and how to wire them up.
Live news studios in 2026 deploy an AI fact-checker behind every anchor, validating claims against trusted sources and offering on-air corrections within 30 seconds. Here is the production stack.
The four major vector index algorithms in 2026 — HNSW, IVF, ScaNN, DiskANN — and which one fits your scale, recall, and latency budget.
When and how to fine-tune embeddings for your domain. The 2026 patterns, the cost-quality tradeoffs, and the open-source tooling.
Embedding models are not interchangeable. The 2026 comparison of OpenAI, BGE, Voyage, Cohere, and the dimensions that matter for production RAG.
© 2026 CallSphere LLC. All rights reserved.