Networking in C · intermediate · ~25 min

HTTP/1.1 chunked transfer encoding

Decode chunked-encoded bodies; understand the smuggling vector.

Overview

Chunked encoding lets a server send a body of unknown length. Each chunk is HEXLEN CRLF DATA CRLF. A 0 CRLF CRLF terminates the body.

Why it matters

Front-end and back-end parsers disagreeing about how a body ends → HTTP request smuggling. Two of the most-exploited 2019-2021 web vulns were exactly this.

Core concepts

Chunk format. Hex size, optional ; chunk-extension, CRLF, data, CRLF.

Terminator. 0\r\n\r\n. The empty chunk + final CRLF.

Trailers. Headers can follow the terminator chunk — rare; allow-list which you accept.

Pentester mindset. Smuggling = front-end honours Content-Length, back-end honours Transfer-Encoding, or vice versa. Defence: refuse any request with BOTH headers; refuse chunk sizes with leading-zero/whitespace/sign/non-hex.

Defensive coding habit. Cap total decoded body length. Refuse chunk size > 256 KiB. Refuse trailing garbage on the size line.

Syntax notes

See RFC 7230 §4.1.

Lesson

When the server can't predict Content-Length up front (streaming), HTTP/1.1 lets it use Transfer-Encoding: chunked. Each chunk is a hex length, CRLF, that many bytes of body, CRLF. A zero-length chunk ends the body. Where the parser disagrees with another parser, you have a smuggling vulnerability.

Code examples

4\r\n
Wiki\r\n
5\r\n
pedia\r\n
0\r\n
\r\n

Line by line

/* Pseudo-loop */
for (;;) {
    char line[64];
    read_line(fd, line, sizeof line);             /* one CRLF-terminated line */
    long size = strict_hex(line);                  /* refuse garbage */
    if (size == 0) break;                          /* terminator */
    if (decoded_so_far + size > MAX) return -1;
    read_n_bytes(fd, dst + decoded_so_far, size);
    decoded_so_far += size;
    read_crlf(fd);                                 /* trailing CRLF */
}
read_optional_trailers(fd);
read_crlf(fd);                                     /* final CRLF */

Common mistakes

  • Accepting chunk-size with leading 0s and getting confused by 'integer overflow'.

Debugging tips

curl with -T - and --header 'Transfer-Encoding: chunked' lets you craft a chunked request from the shell. Or use the nc -l + hand-typed chunks.

Memory safety

Always cap the running total. A malicious 0xFFFFFFFFFFFFFFFF size at the parser level becomes a near-instant OOM.

Real-world uses

Every HTTP/1.1 server. Standard for streaming responses (Server-Sent Events, log tails, etc.).

Practice tasks

  1. Decode a 3-chunk fixture. 2. Refuse a chunk size with leading 0x. 3. Refuse a request that has both CL and TE.

Summary

Chunk = hex size + CRLF + body + CRLF. 0-chunk terminates. Refuse permissive parsing; smuggling lives there.

Practice with these exercises