Realtime: polling, SSE and WebSockets

The problem: HTTP only answers questions

Snip adds a live counter to the dashboard: during a product launch, an owner wants to watch clicks tick up as they happen. But plain HTTP is request and response. The server can only speak when the browser asks. How does the server tell the browser "another click just happened"?

There are four standard answers, from simplest to most capable.

1. Polling

The browser asks every few seconds: "any new clicks?"

GET /links/x7Kp/stats   → { clicks: 18420 }
… 5 seconds later …
GET /links/x7Kp/stats   → { clicks: 18420 }      nothing new
… 5 seconds later …
GET /links/x7Kp/stats   → { clicks: 18431 }

Dead simple, works everywhere, and uses ordinary stateless servers and caches. But updates are up to one interval late, and most requests come back with nothing new. A thousand dashboards polling every 2 seconds is 500 requests a second, mostly wasted.

2. Long polling

The browser asks, and the server holds the request open until there's something to say (or a timeout of, say, 30 seconds passes), then answers. The browser immediately asks again. Updates arrive almost instantly with far fewer empty answers. The cost: every waiting browser holds an open request on a server.

3. Server-sent events (SSE)

The browser opens one ordinary HTTP request and the server keeps the response open, writing events into it whenever it likes:

GET /links/x7Kp/live
Accept: text/event-stream

← id: 1042
← data: {"clicks": 18431}

← id: 1043
← data: {"clicks": 18440}

It's one-way, server to browser, and plain HTTP, so it passes through proxies and load balancers easily. Browsers reconnect automatically and send the last event ID they saw, so the server can fill in anything missed.

4. WebSockets

A WebSocket starts as an HTTP request that asks to upgrade the connection. After that, it's a long-lived, two-way channel: either side can send messages at any time, with very little overhead per message. It's the right tool when the browser also sends a stream of messages: chat, multiplayer editing, games.

The hard part: holding connections at scale

Every SSE or WebSocket client is a connection that stays open for minutes or hours. That changes the design:

• Servers hold state again. Each connection lives on one particular server. When a click for x7Kp is counted, the system has to find the server(s) holding connections that are watching x7Kp.
• So you add a fan-out layer. Connection servers subscribe to the topics their clients care about, usually through a pub/sub system, and the code that counts clicks publishes to the topic.
• Memory per connection adds up. Each open connection costs some memory and a file descriptor; a server can hold many thousands, but you size for connections, not just requests per second.
• Deploys and failures cause reconnect storms. Restart a server holding 50,000 connections and 50,000 clients reconnect at once. Clients should wait a random short delay before reconnecting (03.09 Timeouts, retries, backoff and jitter).

Snip uses SSE: updates only flow one way, it's plain HTTP, and reconnection is built in. Polling every 10 seconds would also have been reasonable for a first version; the click counts already lag by a few seconds.

Recap

Interview questions