How HTTP works

Every page on the web, every image on it and most of the data that apps exchange travel by the Hypertext Transfer Protocol (HTTP). Its idea is simple: a client sends a request for a resource, and a server sends back a response. Around this exchange, HTTP defines a precise vocabulary of methods, status codes and header fields that lets browsers, servers, caches and proxies made by different people work together.

This article is the last step of a journey. Before an HTTP request can be sent, the client looks up the server’s name with DNS, reaches its IP address, opens a TCP connection and secures it with TLS. Then HTTP begins.

A short history

HTTP has carried the World Wide Web since 1990 (RFC 1945). Its first version, later called HTTP/0.9, knew a single command: GET, followed by a path, answered with the document and nothing else. HTTP/1.0, described in RFC 1945 in 1996 by Tim Berners-Lee, Roy Fielding and Henrik Frystyk, added headers, status codes and other methods. HTTP/1.1 followed in 1997 and was revised several times; it is still in daily use.

The next two versions changed how messages travel, not what they mean. HTTP/2 (2015), which drew on an earlier experimental protocol called SPDY, replaced the text format with binary frames and carried many requests over one connection at once. HTTP/3 (2022) moved HTTP onto QUIC, a new transport protocol built on UDP. In June 2022, the IETF republished the whole family in a new structure: HTTP Semantics (RFC 9110), which is the same for every version, plus one document each for caching (RFC 9111), HTTP/1.1 (RFC 9112), HTTP/2 (RFC 9113) and HTTP/3 (RFC 9114).

Resources and URLs

HTTP transfers representations of resources: anything that can be named, such as a page, an image, or a record in a database. Resources are named by URLs (RFC 3986):

https://www.example.com:443/articles/?page=2#history

The default port is usually left out, so the same URL is normally written https://www.example.com/articles/?page=2#history. The fragment, after #, is never sent to the server; the browser uses it to scroll to a part of the page.

Requests and responses

In HTTP/1.1, messages are plain text, which makes them easy to read. A request for a page and its response look like this (shortened):

Every message has the same parts:

1. A start line. In a request, it holds the method, the target and the protocol version. In a response, it holds the version and a status code with a short reason phrase.
2. Header fields, one per line: a case-insensitive name, a colon and a value. They describe the message, the client’s preferences and the content.
3. An empty line, which ends the header.
4. The content, if there is any, such as the HTML of a page or the data of a form.

The Host field names the site the request is for. It is required in HTTP/1.1, because a single server, at a single IP address, often hosts many sites. HTTP/2 and HTTP/3 carry the same information in a field called :authority.

Methods

The method says what the client wants to do with the resource (RFC 9110):

Two properties matter for everything that sits between client and server:

• A safe method only reads; it does not change anything on the server. Browsers, search engines and prefetching tools may send safe requests freely.
• An idempotent method has the same effect whether it is sent once or several times. A client may repeat an idempotent request automatically when a connection fails before the answer arrives. A client should not repeat a non-idempotent request such as POST automatically (RFC 9110): a repeated order form could place the order twice.

Status codes

Every response carries a three-digit status code. The first digit gives the class:

Two pairs are easy to confuse:

• Redirects. 301 and 302 were historically allowed to turn a POST into a GET when the browser follows them, and browsers do. 307 (temporary) and 308 (permanent) forbid changing the method, so a redirected form submission stays a submission.
• 401 and 403. 401 Unauthorized means the request lacks valid credentials, so logging in may help. 403 Forbidden means the server understood who is asking and refuses anyway.

Header fields at work

Header fields carry most of HTTP’s intelligence. A few groups are worth knowing:

• Describing the content. Content-Type gives the media type, such as text/html; charset=utf-8 or application/json. Content-Length gives its size in bytes.
• Content negotiation. The client states its preferences with Accept (media types), Accept-Language and Accept-Encoding, and the server picks the best representation it has. The response’s Vary field names the request fields that influenced the choice, so that caches do not hand an English page to someone who asked for German.
• Compression. With Accept-Encoding: gzip, br, zstd, a client says it understands gzip, Brotli (RFC 7932) and Zstandard (RFC 8878). The server compresses text before sending it and names the method in Content-Encoding. Text such as HTML, CSS and JavaScript typically shrinks to a fraction of its size.
• Conditions and ranges. If-None-Match and If-Modified-Since make a request conditional, which caches use to revalidate a copy. Range asks for part of a resource, which lets a download resume where it stopped (answered with 206 Partial Content).

Cookies: state on a stateless protocol

HTTP is stateless: every request stands on its own, and the server does not remember earlier ones. To build sessions, such as a login or a shopping cart, servers use cookies (RFC 6265). A response sets a cookie:

Set-Cookie: session=7f3a9c; Path=/; Secure; HttpOnly; SameSite=Lax; Max-Age=86400

From then on, the browser sends it back with every matching request, in a Cookie field. The attributes limit where it goes and who can read it:

• Secure sends the cookie only over HTTPS.
• HttpOnly hides it from JavaScript in the page, which limits the damage of injected scripts.
• SameSite controls whether the cookie is sent with requests that other sites trigger, which helps against cross-site request forgery. It is defined in the revised cookie specification, which the IETF has approved for publication as the successor of RFC 6265.
• Max-Age or Expires sets its lifetime; without them, it disappears when the browser closes.

Caching

The fastest request is the one that is never sent. HTTP has a detailed caching model (RFC 9111) that lets browsers, proxies and content delivery networks keep copies of responses and reuse them safely.

A cache first asks whether its copy is fresh. The server sets the lifetime with Cache-Control:

When a copy is stale, the cache does not have to download it again. It revalidates: it asks the server whether the resource has changed, using a validator from the stored response. The ETag field holds an identifier of a version, such as a hash of its content, which the cache sends back in If-None-Match; the Last-Modified date is sent back in If-Modified-Since. If nothing has changed, the server answers 304 Not Modified with no content, and the cache uses its copy for another lifetime.

A common pattern for websites combines both ideas. Files whose names contain a hash of their content, such as app.3f9a2c.js, get a long lifetime and immutable, because a new version gets a new name. HTML pages, which link to those names, get a short lifetime or no-cache, so that visitors see changes quickly.

Connections in HTTP/1.1

HTTP/1.0 opened a new TCP connection for every request, paying for a TCP handshake (and later a TLS handshake) every time. HTTP/1.1 keeps connections open by default, so that several requests can use one connection, one after another. The end of each response is marked by its Content-Length, or by chunked transfer coding when the length is not known in advance.

The limit is that a connection carries only one exchange at a time. A slow response blocks the requests behind it, a problem called head-of-line blocking. HTTP/1.1 allowed pipelining, sending several requests without waiting, but responses still had to come back in the order of the requests, so a slow response still held up the ones behind it. Browsers worked around the limit by opening several connections to each site, and websites by combining files and spreading them across domains.

HTTP/2: many streams, one connection

HTTP/2 keeps the meaning of every message and changes how messages travel.

• Binary frames. Each message is split into frames with a 9-byte header that gives the frame’s length, type and stream. HEADERS frames carry the header fields; DATA frames carry the content.
• Streams. Every request and its response form a stream with its own number. Frames of different streams are interleaved on a single connection, so dozens of requests can be in flight at once without waiting for each other. This is called multiplexing.
• Header compression. Requests repeat many of the same header fields. HPACK (RFC 7541) replaces common fields by references to a static table of 61 entries and to a table of fields already sent on the connection, and compresses the rest with a Huffman code.
• Flow control per stream, so that one large download cannot crowd out everything else.

HTTP/2 also defined server push, which let a server send resources before the client asked for them. A client can turn it off with a setting (SETTINGS_ENABLE_PUSH). A simpler way to speed up a page is 103 Early Hints (RFC 8297), a preliminary response that tells the browser which resources it will need while the real response is still being prepared.

Over TLS, client and server agree on HTTP/2 during the TLS handshake through the ALPN extension, where HTTP/2 is identified as h2.

HTTP/3: HTTP over QUIC

HTTP/2 solved head-of-line blocking between requests, but not underneath them. All its streams share one TCP connection, and TCP delivers bytes strictly in order. When a single packet is lost, TCP holds back everything that arrived after it until the lost packet is retransmitted, even data that belongs to other streams.

QUIC (RFC 9000) fixes this by moving the transport’s work into a new protocol that runs over UDP:

• Independent streams. QUIC knows about streams, so a lost packet delays only the streams whose data it carried.
• TLS built in. QUIC uses TLS 1.3 for its handshake and encrypts almost all of its own headers. Setting up a connection takes one round trip instead of two, and resumed connections can send their first request with no round trip at all (0-RTT).
• Connection migration. A connection is identified by connection IDs rather than by addresses and ports, so it can survive a change of network, such as a phone moving from Wi-Fi to mobile data.

HTTP/3 (RFC 9114) is HTTP mapped onto QUIC streams, with QPACK (RFC 9204) as its header compression, a variant of HPACK designed for streams that arrive out of order. A client learns that a server supports HTTP/3 from an Alt-Svc header field in an earlier response (RFC 7838) or from the server’s HTTPS record in DNS (RFC 9460), then connects over UDP, to port 443 unless another port is announced. Because some networks block UDP, clients should fall back to HTTP over TCP when a QUIC connection fails (RFC 9114).

HTTPS everywhere

An http:// URL sends everything in the clear: anyone on the path can read and change it. An https:// URL sends the same messages through TLS. One weakness remains even for sites that use HTTPS: if a user types a name without https://, the first request may still go out unencrypted and be intercepted before a redirect to HTTPS happens.

HTTP Strict Transport Security (HSTS, RFC 6797) closes this gap. A site sends:

Strict-Transport-Security: max-age=31536000; includeSubDomains

and the browser remembers, for the given number of seconds (here, one year), to use only HTTPS for that site and to refuse connections with invalid certificates. Browsers also ship a preload list of sites that are HTTPS-only from the very first visit.

Everything together: loading a page

Loading https://www.example.com/articles/ for the first time brings every layer of this learning path into play:

1. DNS. The browser looks up www.example.com and receives its IPv4 and IPv6 addresses, and possibly an HTTPS record announcing HTTP/3.
2. Connection. It opens a QUIC connection, or a TCP connection followed by a TLS handshake. TLS verifies the certificate for www.example.com and, through ALPN, settles the HTTP version.
3. Request. It sends GET /articles/ with its headers: accepted formats, languages and compression methods, and any cookies for the site.
4. Response. The server answers 200 OK with compressed HTML and caching instructions.
5. More requests. While parsing the HTML, the browser finds stylesheets, scripts, fonts and images, and requests them over the same connection, many at once. Anything already fresh in its cache is not requested at all.

On the second visit, most of this is gone: the DNS answer and many files come from caches, and the connection may be resumed with 0-RTT.

To watch HTTP at work, open the network panel of a browser’s developer tools, which lists every request with its method, status, protocol version and timing. On the command line, curl -v https://www.example.com/ prints the request and response headers.

Key points

• HTTP is a request–response protocol: a method and a target in, a status code and content out, with header fields describing both.
• Safe and idempotent methods tell clients and intermediaries what they may repeat or prefetch; status codes tell them what happened.
• Caching with Cache-Control, ETag and 304 Not Modified avoids sending data that has not changed. Cookies add state to a stateless protocol.
• HTTP/1.1 handles one exchange per connection at a time, HTTP/2 multiplexes streams over one TCP connection, and HTTP/3 runs over QUIC, where a lost packet delays only its own stream.
• HTTPS and HSTS make sure all of it travels encrypted and authenticated.