Lesson 1 of 5
What a Request Actually Is
"If a message is received with both a Transfer-Encoding and a Content-Length header field, the Transfer-Encoding overrides the Content-Length." — RFC 7230, HTTP/1.1 Message Syntax and Routing (2014), §3.3.3
Part III built the machinery for moving bytes: streams, backpressure, framing, encodings. This Part spends it. Over five chapters it builds an HTTP server from the wire upward — parsing, routing, bodies, responses — so that when Express appears in Part V it has nothing left to hide.
The starting point is a claim that sounds obvious and is routinely forgotten: an HTTP request is not an object. It is a sequence of bytes arriving on a TCP socket, in chunks whose boundaries mean nothing, and every convenient req.headers or req.body you have ever used is the result of somebody parsing those bytes. This chapter is that somebody.
HTTP/1.1 is a text protocol, which is unusual and was a deliberate choice: a request line, some header lines, a blank line, and then a body whose length is determined by the headers you just read. You can type it by hand into a socket, and for twenty-five years people have. That readability is why the protocol won and why so much tooling exists for it.
The interesting part — and the reason this chapter is not just a format description — is the last clause. Where does the body end? HTTP/1.1 offers two answers, they can contradict each other, and the epigraph exists because that contradiction is a security vulnerability rather than an inconvenience.
A request arrives with both Content-Length: 6 and Transfer-Encoding: chunked. Where does its body end?
Two parsers can legitimately disagree, and that is the whole problem. A parser trusting Content-Length reads six bytes. A parser honouring Transfer-Encoding reads until the zero-length chunk — which in the example below is immediately, so it sees an empty body and treats the remaining bytes as the start of the next request.
Now put those two parsers in sequence — a proxy in front of an application server, which is the shape of essentially every production deployment — and an attacker who can make them disagree can inject a request that the front end never saw. That is HTTP request smuggling, and it lets an attacker prepend content to another user's request. The RFC's answer is that Transfer-Encoding wins, and the modern answer, in RFC 9112, is stronger: a message with both should be rejected. The lesson generalises past HTTP — when two components can disagree about where a message ends, that ambiguity is an attack surface.
What is actually on the wire
A request in full, as bytes.
Four details in that structure are worth fixing in your mind, because each has consequences later.
The line separator is CRLF, \r\n, not \n. Splitting on \n leaves a trailing \r on every header value — a bug that produces header values with an invisible character, which then fails an exact comparison for no visible reason.
The request target is a path, not a URL. /api/items?tag=widget has no scheme and no host. The host lives in the Host header, which is why that header became mandatory in HTTP/1.1 — one IP address had to serve many sites. Reconstructing a full URL means combining the two, and §5 shows why that needs care.
The blank line ends the head. Two CRLFs in a row. Everything after is body, and the head is where all the information about the body lives.
The body's length is not in the body. Nothing in the byte stream says "the body is seventeen bytes" except a header you already parsed. That is the framing problem from Part III, and §3 is about the two incompatible answers HTTP gives it.
A parcel with a paper docket taped to the outside. The docket says who it is for, what is inside, and — crucially — how heavy the contents are. The contents themselves are just stuff in a box; nothing inside tells you where the box ends.
That works until the docket says "6 kg" and a second docket underneath says "weigh it yourself, I'll tell you when to stop". Now two people in the depot handling the same parcel can reach different conclusions about which items belong to it, and the ones they disagree about get attributed to the next parcel on the belt. Nobody notices, because both dockets look official and both readings are internally consistent. That is request smuggling, and it is why the modern advice is not "pick a rule" but "refuse the parcel".
Parsing the head without assuming chunk boundaries
The parser above works because the whole request was one string. On a real socket it is not.
This is the third time this Part has produced the same shape, which is the point: accumulate, look for a complete unit, emit it, keep the rest. A line splitter, a length-prefixed frame decoder and an HTTP head parser are the same algorithm with different terminators.
Two things a real parser must add, both of which are security requirements rather than niceties.
A maximum head size. A client that sends headers forever and never sends the blank line will make an accumulating parser grow without bound — a trivial denial of service. Node caps this at 16KB by default and responds 431 Request Header Fields Too Large.
A limit on header count. The same attack with many small headers rather than one enormous one, and historically also a hash-collision vector.
The general rule for anything parsing untrusted input: every accumulator needs a bound, and the bound belongs in the parser rather than in a comment.
HTTP/0.9, in 1991, was one line: GET /path, followed by the document. No headers, no status code, no version. HTTP/1.0 added headers and status codes in 1996, and HTTP/1.1 in 1997 added the things that made the modern web possible — persistent connections, chunked transfer encoding, and a mandatory Host header so one address could serve many sites.
The decision to make it a text protocol was contested and turned out to be enormously consequential. It made the protocol debuggable with telnet, implementable in an afternoon, and inspectable by every intermediary on the path — which is a large part of why the web's tooling ecosystem is so rich. It also made parsing slow and, as this chapter's big question shows, ambiguous in ways a binary format would not have been.
HTTP/2, in 2015, reversed that choice: it is binary, with a proper framing layer, header compression via HPACK, and multiplexed streams over one connection. Request smuggling is much harder because frame lengths are explicit and unambiguous. The trade is that you can no longer read it off the wire — and notably, HTTP/2's semantics are identical, which is why everything in this Part still applies. What changed is the transport, not the meaning.
Where the body ends
Two mechanisms, and the disagreement between them.
The two mechanisms exist for genuinely different situations.
Content-Length requires knowing the size before you send. Fine for a file or a serialised object, impossible for a response you are generating as you go — which is exactly the streaming case from Part III.
Chunked encoding sends the size of each piece as it goes, terminated by a zero-length chunk. It is what makes streaming responses possible over HTTP/1.1, and it is why a streamed response has no Content-Length header.
A request with neither has no body — for requests. Responses have a third option: read until the connection closes, which is what HTTP/1.0 did and why persistent connections needed one of the other two.
The disagreement is the security problem, and the defences are worth stating plainly. Reject any message with both headers — that is RFC 9112's guidance and what modern servers do. Reject duplicate or conflicting Content-Length headers. And do not write your own HTTP parser for production; the failure modes are subtle, adversarial, and thoroughly explored by people who are not on your side.
One more piece explains why any of this is exploitable rather than merely untidy: HTTP/1.1 connections are persistent by default. In HTTP/1.0 each request got its own TCP connection, which was closed afterwards — so bytes a parser did not consume were simply discarded with the socket. HTTP/1.1 reuses one connection for many requests, which is essential for performance because a TCP handshake costs a round trip and a TLS handshake costs more.
That reuse is what turns a framing disagreement into an attack. If a front end thinks a request ended at byte six and the back end thinks it ended at byte five, the leftover byte is not thrown away — it stays in the connection's buffer and becomes the first byte of the next request on that connection, which usually belongs to a different user. Without connection reuse, smuggling would be a parsing curiosity; with it, an attacker can prepend arbitrary content to a stranger's request.
Which gives a rule worth carrying: a parser handling a reused connection must be exact about where a message ends, because any leftover byte is not lost, it is attributed to somebody else.
Three questions when handling raw HTTP.
How does this parser decide where the body ends? If the answer is "Content-Length" without also handling Transfer-Encoding, it will disagree with whatever sits in front of it.
Is every accumulator bounded? Head size, header count, body size, chunk size. Untrusted input plus an unbounded buffer is a denial of service that needs no cleverness.
Am I splitting on \n or \r\n? Splitting on \n leaves a trailing carriage return on every value, which compares unequal to the string you expect and is invisible in a log.
Headers are not a plain object
Header semantics are stranger than the { key: value } shape suggests, and the platform's Headers class encodes the real rules.
Four rules that a plain object cannot express.
Names are case-insensitive. Content-Type and content-type are the same header. Node lowercases them in req.headers, and so does Headers. Code that reads req.headers["Content-Type"] works by accident on some clients and fails on others.
A header can legitimately appear several times, and the RFC says repeated fields are equivalent to one field with the values comma-joined. Headers.get does that joining for you.
Set-Cookie is the exception, because cookie values can contain commas — joining them corrupts them. This one header needs its own accessor, getSetCookie(), which is why Node's res.setHeader accepts an array for it and why req.headers["set-cookie"] is an array while every other header is a string.
Order is not preserved. Headers iterates in sorted order, so any logic depending on which header arrived first is depending on something the abstraction has thrown away.
Request smuggling is not theoretical — it has been used against major CDNs and produced serious breaches, and the reason is architectural rather than incidental. Almost every production deployment chains parsers: a load balancer, a CDN, a reverse proxy, then the application server. Each one parses the request independently, and they are frequently different implementations by different vendors. All it takes is one pair that disagrees about framing, and an attacker can prepend bytes to the next user's request — stealing session cookies, or poisoning a cache so that a poisoned response is served to everyone. The defence is uniformity: reject ambiguous messages at the edge rather than normalising them, and prefer HTTP/2 to the origin, where framing is explicit.
The everyday version of the same class is header case. Code reads req.headers["X-Request-Id"], which is undefined because Node lowercases every incoming header name. It works in a unit test where the header was set by hand with that exact casing, and returns undefined in production for every request. Tracing then silently stops correlating, which nobody notices until an incident when the traces are needed.
The third is Set-Cookie flattening. Middleware copies headers between requests with Object.entries, joins the two Set-Cookie values with a comma, and produces one malformed cookie instead of two valid ones. Users are logged out intermittently, and the cause is a header-copying helper that is correct for every header except the one that matters.
The request target and the URL
The last piece of the request line is the one with the most traps.
Five practical points.
A request target is not a URL. It has no scheme or host, so new URL(target) throws. Combine it with the Host header — and treat that header as untrusted input, because a client controls it. Using it to build absolute URLs in emails or redirects is how host-header injection works.
Query parameters are multi-valued. ?tag=a&tag=b is legitimate, and get returns only the first. Code that assumes a single value gets one silently.
+ means space in a query string, but not in a path. URLSearchParams handles this; decodeURIComponent does not, which is why hand-rolled query parsing turns hello+world into hello+world.
Percent-encoded slashes are not slashes. /files/a%2Fb is a two-segment path where the second segment contains a slash. Decoding before routing turns it into three segments — a classic path-traversal vector.
URL does relative resolution correctly, which is genuinely hard to get right by hand and is the main reason to use it rather than string manipulation.
Common Mistakes
Splitting on
\ninstead of\r\n. Every header value keeps an invisible trailing carriage return.Treating the request target as a URL. It has no scheme or host;
new URLthrows without a base.Reading headers with the wrong case. Incoming names are lowercased, so
req.headers["X-Request-Id"]isundefined.Assuming a header appears once. Repeated headers are legal and comma-joined — except
Set-Cookie.Comma-joining
Set-Cookie. Cookie values contain commas; joining produces one malformed cookie.Relying on header order.
Headersiterates sorted, not in the order they arrived.Honouring
Content-Lengthwithout handlingTransfer-Encoding. That is one half of a smuggling pair.Accepting a message with both framing headers. Reject it; normalising it is what lets two parsers disagree.
An unbounded accumulator on untrusted input. Head size, header count and body size all need limits.
Decoding the path before routing.
%2Fis not a path separator, and treating it as one is a traversal vector.
Interview Corner
POST /api/items HTTP/1.1. Then zero or more header lines of the form Name: value. Then a blank line, which is what marks the end of the head. Then the body, if there is one. Every line ends with CRLF rather than just a newline, which matters because splitting on \n leaves a stray carriage return on every value. Two things surprised me when I first looked at the raw bytes. The request target is a path, not a full URL — there is no scheme or host in it, which is why the Host header became mandatory in 1.1 so one server could serve several sites. And nothing in the body says how long the body is; that comes from the headers you have already read, either Content-Length or Transfer-Encoding: chunked. So parsing is: read until the blank line, parse the head, then use the head to work out how many body bytes to read.Content-Length and Transfer-Encoding: chunked. What should a server do?Transfer-Encoding wins and the Content-Length should be ignored, but the current guidance in RFC 9112 is to treat the message as invalid, and that is the safer position. The reason is that the two headers give different answers about where the request ends, so two parsers handling the same bytes can disagree — and real deployments always have at least two parsers in a chain, typically a proxy or CDN in front of an application server, often different implementations from different vendors. If the front end uses Content-Length and the back end uses Transfer-Encoding, then bytes the front end thinks are part of this request are seen by the back end as the beginning of the next one, so an attacker can prepend content to another user's request. That is HTTP request smuggling, and it has been used to steal session cookies and poison caches. So the rule is not to pick a winner but to refuse the ambiguity, and to make sure the whole chain refuses it consistently — normalising the message at the edge instead of rejecting it is what creates the disagreement.Content-Length values, and reject anything with obsolete line folding, since all of those are places two parsers can differ. I would also be careful not to normalise on the way through — rewriting a request to make it valid is exactly how a front end and a back end come to see different messages. And I would not decode the path before routing, because a percent-encoded slash is not a separator and decoding early merges paths that should stay distinct, which is a traversal vector. The general principle is that a parser handling untrusted input has two jobs, and understanding the input is only the first one.undefined in production but not in tests.Hands-on
Four exercises building a request parser. Exercise 3 is the security-relevant one — deciding how a body is framed, and refusing the ambiguous case.
Summary
- A request is bytes on a socket, not an object — every
req.headersyou have used is the output of a parser. - HTTP/1.1 is text: a request line, header lines, a blank line, then a body. CRLF, not
\n. - The request target is a path, with the host in a header — which is why
Hostbecame mandatory. - The body's length is never in the body. It comes from
Content-LengthorTransfer-Encoding: chunked. - Chunked encoding exists so you can stream a body whose total size you do not yet know.
- A message with both framing headers is ambiguous, and that ambiguity is request smuggling — reject it.
- Every accumulator needs a bound: head size, header count, body size. Untrusted input plus an unbounded buffer is a denial of service.
- Header names are case-insensitive and may repeat, comma-joined — except
Set-Cookie, which needs its own accessor. Headersiteration is sorted, so wire order is not recoverable.- A percent-encoded slash is not a separator. Decoding the path before routing merges distinct paths and enables traversal.
The request is now parsed, and everything so far has been done by hand. The next chapter introduces Node's own http module and asks a precise question of it: what does it actually do for you? It handles the parsing in this chapter, connection management, and the framing rules — and it deliberately stops short of routing, body parsing and anything resembling a framework. Understanding exactly where that line falls is what makes the rest of this Part, and Express after it, comprehensible rather than magical.
HTTP being a text protocol is the design decision I find most interesting to argue about. It made the web debuggable by anyone with a terminal, implementable by a student in an afternoon, and inspectable by every box on the path — and I think that accessibility is a large part of why it won, in the way that most winning formats win. It also produced request smuggling, because a text protocol with two framing mechanisms and lenient parsers is ambiguous in ways a binary format simply is not. That is a real cost, paid in real breaches, for a real benefit.
What I would push back on is the culture of leniency that grew around it. Postel's robustness principle — be conservative in what you send, liberal in what you accept — reads as good manners and turns out to be a security hazard when applied to parsers, because "liberal in what you accept" means two implementations accept different things, and every difference is exploitable. The modern direction, visible in RFC 9112's instruction to reject rather than reconcile a doubly-framed message, is the correct correction: for anything adversarial, being strict is the interoperable choice, because strictness is the only property two independent implementations can actually agree on.
The detail I would most want a reader to keep is the smallest. Nothing in an HTTP body says how long the body is. That single fact is why chunked encoding exists, why smuggling is possible, why every parser needs a bound, and why the framing patterns from Part III kept reappearing in this chapter. Framing is not an HTTP topic — it is what every protocol has to solve, and HTTP's answer being negotiable in the headers is what makes it interesting and dangerous.
Q&A · 0
Enrol to ask questions and join the discussion.
No questions yet — be the first to ask.