Server-Sent Events vs WebSockets for AI Streaming: Choosing the Right Protocol

Two Protocols, Different Design Philosophies

When building AI applications that stream responses — token-by-token LLM output, live agent status updates, or progressive search results — you have two main protocol choices: Server-Sent Events (SSE) and WebSockets. Both enable real-time data delivery, but they solve fundamentally different problems.

SSE is a unidirectional protocol built on top of plain HTTP. The server pushes events to the client over a long-lived HTTP response. The client cannot send data back over the same connection — it uses regular HTTP requests for that. WebSockets, by contrast, provide full-duplex communication over a single TCP connection. Either side can send messages at any time.

This distinction shapes everything: infrastructure compatibility, scaling behavior, error handling, and implementation complexity.

Server-Sent Events: The Simpler Choice

SSE works with standard HTTP infrastructure. Load balancers, CDNs, proxies, and API gateways all understand HTTP, so SSE connections pass through without special configuration. The browser handles reconnection automatically — if the connection drops, the EventSource API reconnects and sends a Last-Event-ID header so the server can resume from where it left off.

sequenceDiagram
    autonumber
    participant Client
    participant Edge as Edge Worker
    participant LLM as LLM Provider
    participant DB as Logs and Trace
    Client->>Edge: POST /chat (stream=true)
    Edge->>LLM: messages.create(stream=true)
    loop Each token
        LLM-->>Edge: SSE chunk delta
        Edge-->>Client: SSE chunk delta
        Edge->>DB: append token to span
    end
    LLM-->>Edge: stop_reason=end_turn
    Edge-->>Client: event: done
    Edge->>DB: finalize trace

from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse
import asyncio
import json

app = FastAPI()

async def stream_agent_response(prompt: str):
    """Generator that yields SSE-formatted events."""
    # Simulate streaming LLM tokens
    tokens = await run_agent_streaming(prompt)

    event_id = 0
    async for token in tokens:
        event_id += 1
        data = json.dumps({"token": token, "done": False})
        yield f"id: {event_id}\ndata: {data}\n\n"

    final = json.dumps({"token": "", "done": True})
    yield f"id: {event_id + 1}\ndata: {final}\n\n"

@app.get("/api/agent/stream")
async def agent_stream(prompt: str, request: Request):
    return StreamingResponse(
        stream_agent_response(prompt),
        media_type="text/event-stream",
        headers={
            "Cache-Control": "no-cache",
            "Connection": "keep-alive",
            "X-Accel-Buffering": "no",  # Disable nginx buffering
        },
    )

The client side is equally straightforward:

function streamAgentResponse(prompt: string): void {
  const url = \`/api/agent/stream?prompt=\${encodeURIComponent(prompt)}\`;
  const source = new EventSource(url);

  source.onmessage = (event) => {
    const data = JSON.parse(event.data);
    if (data.done) {
      source.close();
      return;
    }
    appendToken(data.token);
  };

  source.onerror = () => {
    // EventSource automatically reconnects
    console.log("Connection lost, reconnecting...");
  };
}

The EventSource API handles reconnection, event ID tracking, and connection management. You get built-in resilience with zero custom code.

WebSockets: When You Need Bidirectional Communication

WebSockets become necessary when the client needs to send data to the server during an active stream. Consider an AI coding assistant where the user types while the agent is still generating — the client needs to send keystrokes, cursor positions, and cancellation signals on the same connection that receives streaming tokens.

from fastapi import FastAPI, WebSocket

app = FastAPI()

@app.websocket("/ws/agent")
async def agent_ws(ws: WebSocket):
    await ws.accept()
    agent_task = None

    try:
        while True:
            msg = await ws.receive_json()

            if msg["type"] == "query":
                if agent_task and not agent_task.done():
                    agent_task.cancel()
                agent_task = asyncio.create_task(
                    run_and_stream(ws, msg["prompt"], msg["request_id"])
                )

            elif msg["type"] == "cancel":
                if agent_task and not agent_task.done():
                    agent_task.cancel()
                    await ws.send_json({
                        "type": "cancelled",
                        "request_id": msg["request_id"],
                    })

            elif msg["type"] == "context_update":
                update_agent_context(msg["payload"])

    except Exception:
        pass

async def run_and_stream(ws: WebSocket, prompt: str, request_id: str):
    try:
        async for token in run_agent_streaming(prompt):
            await ws.send_json({
                "type": "token",
                "data": token,
                "request_id": request_id,
            })
        await ws.send_json({"type": "done", "request_id": request_id})
    except asyncio.CancelledError:
        pass

The key advantage here is that context_update and cancel messages arrive on the same connection, interleaved with the streaming response. With SSE, you would need a separate HTTP POST endpoint for cancellation, introducing coordination complexity.

Decision Framework

Criteria	SSE	WebSocket
Direction	Server to client only	Bidirectional
Protocol	HTTP	Custom (upgrade from HTTP)
Auto-reconnect	Built-in (EventSource)	Must implement yourself
Proxy/CDN support	Excellent	Requires configuration
Browser support	All modern browsers	All modern browsers
Max connections	6 per domain (HTTP/1.1)	No HTTP limit
Binary data	Text only	Text and binary frames

Choose SSE when your AI application follows a request-then-stream pattern: the user submits a prompt via POST, and the server streams back the response. This covers most chatbots, summarization tools, and content generators.

Choose WebSockets when the client and server need ongoing, interleaved communication: collaborative editing with AI suggestions, real-time agent dashboards with user controls, or multi-agent coordination where the client routes messages between agents.

HTTP/2 Changes the SSE Equation

Under HTTP/1.1, browsers limit SSE to six connections per domain. This becomes a problem if a single page opens multiple streams. HTTP/2 multiplexes all streams over one TCP connection, eliminating the limit. If your infrastructure supports HTTP/2 (most modern setups do), SSE becomes viable even for applications with many concurrent streams.

# With HTTP/2, multiple SSE streams multiplex efficiently
# No special server code needed — just ensure your reverse proxy
# supports HTTP/2:
# nginx: listen 443 ssl http2;
# This removes the 6-connection-per-domain limitation

FAQ

Can I use SSE with POST requests to send large prompts?

The native EventSource API only supports GET requests, which limits prompt size due to URL length restrictions. However, you can use the Fetch API with ReadableStream to make POST requests and process SSE-formatted responses. Libraries like eventsource-parser handle the event parsing. This gives you POST semantics (large request bodies) with SSE streaming — a pattern the OpenAI API itself uses for chat completions.

How does authentication work differently between SSE and WebSockets?

SSE rides on HTTP, so standard authentication mechanisms work directly — cookies, Bearer tokens in headers, and API keys in query parameters all function normally. WebSockets perform authentication during the initial HTTP upgrade handshake, but the WebSocket browser API does not allow custom headers. You typically authenticate by sending a token as the first message after connection, or by passing it as a query parameter in the WebSocket URL.

What about gRPC streaming as a third option for AI applications?

gRPC server streaming is an excellent choice when both client and server are services you control — for example, inter-service communication between an API gateway and an AI inference backend. gRPC offers strong typing via Protocol Buffers, efficient binary serialization, and built-in streaming support. However, browser support requires gRPC-Web (a proxy layer), making it less practical for browser-to-server AI streaming compared to SSE or WebSockets.

---

#SSE #WebSocket #Streaming #RealTimeAI #APIDesign #AgenticAI #LearnAI #AIEngineering