sehaj/learn
System design/03 Networking and APIs

03.02 · concept · 8 min

TCP vs UDP

TCP delivers every byte in order and pays for it in waiting. UDP delivers what it can, now. When each fits, and why HTTP/3 moved to UDP.

Two ways to send data

Underneath every request, data travels as small packets. The network makes no promises about them: a packet can be lost, arrive twice, or arrive after one sent later. The two main transport protocols make different choices about what to do with that mess.

TCPUDP
ConnectionA handshake first (one round trip)None: just send
DeliveryEvery byte arrives, or the connection failsPackets may be lost; nobody resends them
OrderBytes arrive in the order sentPackets may arrive in any order
Speed controlSlows down when the network is congestedNone built in; the app decides
OverheadMore: acknowledgements, retransmissionsLess: a small header and the data
Used byWeb pages, APIs, databases, email, file transferDNS lookups, video calls, games, live streams, QUIC

TCP: reliable, ordered, and sometimes waiting

TCP numbers every byte. The receiver acknowledges what it gets; anything not acknowledged in time is sent again. The receiving side hands data to the application strictly in order.

That last part has a cost called head-of-line blocking: if one packet is lost, everything after it waits until the lost one is resent and arrives, even if the later packets are already sitting there. For a file download that's fine; you need every byte anyway. For a video call it's not: a frame that arrives 300 ms late is worse than a frame that never arrives.

UDP: whatever arrives, right now

UDP sends each packet on its own, with no handshake, no acknowledgements and no resending. The application gets whatever arrives, as it arrives. That suits two kinds of traffic:

  • Small, one-shot questions where a retry is simpler than a connection. A DNS lookup is a single small packet each way; if the answer doesn't come back, the resolver just asks again.
  • Real-time data where late is useless: voice, video, game state. The app would rather skip a lost packet and play the next one, or send a fresh update, than wait.

If an app on UDP does need reliability or ordering for some of its data, it builds just that part itself.

QUIC: building a better TCP on top of UDP

HTTP/2 sends many requests over one TCP connection at once. That cut the number of handshakes, but brought back head-of-line blocking at a new level: one lost packet stalls every request on the connection, because TCP only knows about one stream of bytes.

Changing TCP itself is very slow: it lives in operating systems and in network equipment along the way. So Google built QUIC, which runs on UDP and does the reliability work in the application instead:

  • Many independent streams in one connection, so a lost packet only delays the stream it belongs to.
  • Encryption built in, with the transport and TLS handshakes combined into one round trip, or none when reconnecting to a server it has seen before.
  • Connections that survive the client's network changing, say from Wi-Fi to mobile data.

QUIC became the transport for HTTP/3. The lesson for designers: UDP isn't only for "unreliable" traffic. It's also the base you build on when you want reliability with different trade-offs than TCP's.

Seen in production

Google

2017

QUIC

Google built QUIC in user space on UDP and deployed it in Chrome, the YouTube app and the Google Search app. The paper reports that, on average, QUIC cut Google Search response latency by 8.0% for desktop users and 3.6% for mobile users, and cut YouTube rebuffer rates by 18.0% on desktop and 15.3% on mobile. It carried over 30% of Google's egress traffic, an estimated 7% of all Internet traffic.

What we foundMoving the transport onto UDP, in the application, let Google change it at app-update speed and measurably cut latency.

The QUIC Transport Protocol: Design and Internet-Scale Deployment (SIGCOMM 2017)research.google ↗
Predict first

You're designing a multiplayer game that sends each player's position 30 times a second. A packet is lost. What should happen?

Commit to an answer before you read on.

Recap

Interview questions

JuniorfreqMust-knowConcept

What's the difference between TCP and UDP? Give an example where you'd use each.

Report a mistakeEdit this page