The Request Path

"The network is the computer." — John Gage, Sun Microsystems

In Volume 0 you learned the pieces in isolation: DNS turns a name into an address, TCP carries bytes reliably, HTTP frames a request and a response. That was the mechanic's view — each part on its own bench. System design asks a different, scarier question: what actually happens in the ~200 milliseconds between a user's click and the pixels changing on their screen — and why does that path have seven boxes in it instead of one? Every one of those boxes was added by an engineer who was losing sleep over latency, cost, or a 3am page. Understanding the request path is understanding the skeleton every large system hangs off. It is also, not by accident, the diagram interviewers ask you to draw first — because if you can trace a request end to end, you can reason about where anything scales, breaks, or slows down.

Core Concepts

You already know a single request-response cycle: client opens a connection, sends GET /timeline, server replies. At scale, that straight line grows hops — each one a specialised layer that exists to solve a problem the layer before it couldn't. Here is the whole path, top to bottom, for a modern web request:

Walk it hop by hop, and notice what each box buys you:

• DNS resolution turns api.example.com into an IP. The result is cached at many levels (browser, OS, resolver) with a TTL, so most requests skip the lookup entirely. Increasingly the IP you get back belongs to a CDN edge, not your origin — geography-aware DNS steers each user to their nearest point of presence.
• CDN (edge network). For static, cacheable bytes — images, JS/CSS bundles, video segments — the request ends here, at a server physically near the user, never touching your infrastructure. This is the single biggest latency win available, and it also shields your origin from load. A dynamic, personalised request falls through to the origin.
• Load balancer. The front door to your system. It spreads incoming requests across a pool of identical app servers and, crucially, routes around any server that fails a health check. It is a reverse proxy (more on that below).
• API gateway. A policy layer: it authenticates the caller, enforces rate limits, terminates TLS, routes /timeline to the timeline service and /media to the media service, and stamps a request ID for tracing. In small systems the LB and gateway collapse into one box.
• App server. Where your business logic runs. The critical property, discussed below, is that it holds no per-user state — any server can handle any request.
• Cache. An in-memory store (Redis, Memcached) checked before the database. A cache hit answers in well under a millisecond; a miss falls through to the DB and usually populates the cache on the way back.
• Database. The source of truth — durable, consistent, and by far the most expensive hop to touch. The entire architecture above it exists, in large part, to avoid hitting it more than necessary.

Where the latency actually goes

You cannot cut time you cannot see. Here is a realistic budget for a cold, cross-region request — the numbers are typical mid-2020s figures, not laws of physics, but the ratios are what matter:

Read the pattern: the expensive hops are the ones that cross a network or hit a disk. A same-datacenter cache read is ~1000× faster than an intercontinental round trip. That is why the great latency wins are all about distance and avoidance: move data close to the user (CDN), reuse connections instead of re-handshaking (keep-alive), answer from memory instead of disk (caching), and keep a warm pool of DB connections ready (pooling) instead of paying the handshake per query.

Forward vs reverse proxy

A proxy is just a middleman that relays requests. The direction it faces is the whole distinction. A forward proxy sits in front of clients and speaks to the wider internet on their behalf — think a corporate proxy that filters employee traffic, or a VPN. A reverse proxy sits in front of servers and receives traffic on their behalf — the client thinks it is talking to one machine, while the proxy quietly fans requests out to a pool behind it.

Your load balancer and API gateway are reverse proxies — that is precisely their job. Naming this in an interview signals you understand the shape of the architecture, not just the box names.

Why the app tier must be stateless

Here is the load-bearing principle of the whole chapter. The load balancer sends each request to whichever app server is free. If server A remembered that "user 42 is logged in" in its own memory, then a follow-up request routed to server B would find a stranger — the user would appear logged out at random. Worse, you could never add or remove servers safely.

The fix is to make the app tier stateless: each app server keeps no per-user memory between requests. Any request can land on any server and be handled identically. Session state lives outside the app tier — either in a shared store (a session row in Redis, keyed by a cookie) or carried by the client itself in a signed token (a JWT the server verifies without looking anything up).

Statelessness is what makes horizontal scaling possible: to handle more load you just add more identical app servers behind the LB, and to survive a failure you just remove a sick one. Nothing is lost because nothing important lived on the box. This single idea — push state to the edges (client token or shared store), keep the middle interchangeable — is the difference between a system that scales to millions and one that falls over the moment one server reboots.

A concrete walk: loading a Twitter/X timeline

Trace it. You open the app and it requests your home timeline:

1. DNS resolves the API host — cached, so ~0 ms — and points at a nearby edge.
2. Your avatar, the app's JS bundle, and images load straight from the CDN at the edge. They never reach the origin.
3. The dynamic request GET /timeline crosses to the load balancer, which picks a healthy app server.
4. The API gateway verifies your auth token (stateless — no DB lookup needed), checks you are under your rate limit, and stamps a request ID.
5. A stateless app server handles it. It asks: is this user's timeline already computed in the cache? For most users, yes — timelines are pre-computed and "fanned out" into a per-user cache when people they follow post. That is a cache hit, answered from Redis in under a millisecond.
6. Only on a miss (or for accounts following millions) does it fall through to the database and assemble the timeline live — the slow path.
7. The response retraces the hops back to your screen.

The lesson: at scale, the common case is engineered to never touch the database at all. The whole path is a series of increasingly expensive fallbacks, and good design keeps you in the cheap tiers.

Common Mistakes

• Drawing one box for "the server." A senior answer distinguishes the edge (DNS/CDN/LB), the application tier, and the data tier — because they scale, fail, and cost differently.
• Making the app tier stateful. Storing sessions or uploads in a single server's memory or local disk quietly kills horizontal scaling and breaks the moment the LB routes elsewhere. Push state to a shared store or a token.
• Ignoring where latency actually lives. Optimising 2 ms of app code while a cross-region DB round trip costs 150 ms is polishing the doorknob while the house is on fire. Find the dominant hop first.
• Forgetting the CDN for static assets. Serving images and JS bundles from your origin wastes your most expensive capacity on your cheapest, most cacheable content.
• Re-handshaking per request. No keep-alive / no connection pooling means paying the TCP+TLS or DB-connection setup cost every single time — often more than the useful work.
• Assuming retries are free and safe. A retried non-idempotent write double-charges, double-posts, double-sends. The path needs idempotency, not just optimism.

Interview Corner

Chapter Summary

• A request at scale is a series of hops — DNS → CDN → load balancer → API gateway → app server → cache → database — and each box was added to solve a specific latency, scale, or reliability problem.
• Latency concentrates in the hops that cross a network or hit disk: intercontinental RTTs (~150 ms), TLS handshakes (multiple RTTs), and DB queries dwarf in-memory cache reads (~1 ms).
• Cut latency by distance and avoidance: CDN/edge, keep-alive and TLS resumption, caching in front of the DB, connection pooling, and co-locating services.
• The load balancer and API gateway are reverse proxies — one front door hiding a pool of servers behind it.
• The app tier must be stateless — session state lives in a shared store or a signed token, never in one server's memory — because that is what makes horizontal scaling and failover possible.
• Distinguish north-south (client↔system) from east-west (service↔service) traffic; at scale, east-west dominates. Make writes idempotent for safe retries, and stamp a request ID for tracing.

Editorial reflection — the request path is the first diagram you should be able to draw in your sleep, because every other system-design question is a variation on "which box do we change, and what does that cost the others?" Add a CDN and you cut latency but complicate cache invalidation. Add a replica and you scale reads but invite stale data. Make the tier stateless and you scale freely but push state somewhere you now have to manage. There is no box you can add for free — only trade-offs you choose on purpose. Learn the path not as trivia, but as the map you'll navigate for the rest of your career.