Reading HTTP requests

Overview

Security objective. The asset you protect here is your server's request parser and everything behind it. The threat is a malformed or hostile HTTP request — a client that sends an oversized line, a path full of ../, a bogus method, or a body that lies about its own length. In this lesson you learn to read an HTTP request field by field so you can later detect probing and reject bad input before it reaches application logic.

An HTTP/1.x request is just plain text sent over a socket. Because it is human-readable, you can parse it directly instead of using a black-box library — which is exactly what you need when reviewing captured traffic or writing a defensive proxy in a lab.

This builds directly on Sockets introduction (your prerequisite): there you learned to read() bytes from a connected socket into a buffer. Here you take that same buffer and give the bytes meaning — turning a flat array of characters into a structured request you can inspect. A socket hands you bytes; HTTP parsing tells you what those bytes are asking for.

Everything in this lesson runs against sample text fixtures or a local, isolated lab you control — never live third-party traffic.

Why it matters

In authorized professional work, reading raw HTTP is a daily skill:

• Detection engineering / SOC work. When you triage a suspected web attack, you read the raw request to see what the attacker actually sent — the method, the path, the headers — not a pre-parsed summary that may have hidden the interesting bytes.
• Building defensive tooling. WAF rules, reverse proxies, and log parsers all start by splitting a request into fields. If you cannot parse it, you cannot filter it.
• Vulnerability triage. Many classes of bug (path traversal, request smuggling, header injection) live in the gap between how two programs parse the same bytes. Understanding the exact byte layout is what lets you reason about those gaps.
• Writing robust servers. A parser that trusts the wire is a parser that gets overflowed. Careful, bounded HTTP parsing is a foundational secure-coding habit.

Employers value engineers who can look at a captured request and immediately say "that path has a traversal sequence" or "that Content-Length disagrees with the body" — and then write code that rejects it.

Core concepts

1. The request message layout

Definition. An HTTP/1.x request is a text message with four regions, in order: a request line, zero or more header lines, a blank line, and an optional body.

How it works. Every line ends with the two bytes \r\n (carriage return 0x0D, then line feed 0x0A). The headers end when the parser sees an empty line — i.e. two \r\n in a row (\r\n\r\n). Everything after that is body.

GET /index.html HTTP/1.1\r\n      <- request line
Host: localhost\r\n                 <- header
User-Agent: curl/8.0\r\n            <- header
\r\n                                <- blank line: headers end here
(optional body bytes...)           <- body

When/when not. This exact layout is HTTP/1.0 and 1.1. HTTP/2 and HTTP/3 use binary framing, so this text parsing does not apply to them — do not assume \r\n structure on an h2 connection.

Pitfall. People split on \n alone and forget the \r. A stray \r left on the end of the path (/index.html\r) then breaks every later comparison. Always account for the full \r\n.

2. The request line: METHOD PATH VERSION

Definition. The first line holds three space-separated tokens: the method (GET, POST, HEAD, ...), the request target / path (/search?q=hi), and the HTTP version (HTTP/1.1).

Plain explanation. This one line says what the client wants to do (method) and to which resource (path). It is the highest-value line for security review.

How it works. You find the first space, then the second space; the bytes between them are the path. The method is everything before the first space; the version is everything after the second.

When/when not. Do not assume exactly one space between tokens or that the line fits your buffer. A hostile client can send megabytes with no space at all. Bound every copy.

Pitfall. Copying the path into a fixed buffer with strcpy or an unbounded loop is a classic buffer overflow. Always track the buffer size and stop early.

3. Headers: Name: value

Definition. Each header is a line of the form Name: value. Names are case-insensitive; a single colon separates name from value; leading spaces in the value are trimmed.

How it works. You read lines until you hit the blank line. For each, split on the first colon: left is the name, right (after optional spaces) is the value.

Pitfall. Splitting on every colon breaks values that contain colons (e.g. Host: localhost:8080). Split on the first colon only.

4. Body and Content-Length (never trust the wire)

Definition. If a body is present, a Content-Length header states how many bytes follow the blank line (or Transfer-Encoding: chunked is used instead).

Insecure assumption to avoid. "The body is exactly Content-Length bytes, so I can read that many into a buffer." A malicious client can send Content-Length: 999999999. Treat the header as a claim, cap it against a hard maximum, and never size an allocation directly from an attacker-controlled number without a bound.

5. Suspicious signals in captured traffic

Once parsed, certain field values are worth flagging when you review authorized captures:

Flagging is detection, not blocking — it produces an alert a human reviews.

Threat model

         UNTRUSTED                         TRUSTED (you control)
  +------------------+   TCP/socket   +--------------------------+
  |  HTTP client     |  ----bytes---> |  Your parser (this code) |
  |  (browser, curl, |   entry point  |  - split request line    |
  |   scanner, fuzz) |                |  - parse headers         |
  +------------------+                |  - bound every buffer    |
         |                            |  - flag suspicious paths |
     may be hostile                   +------------+-------------+
                                                   |
   ============ TRUST BOUNDARY ============        | only clean, bounded
   (all bytes above the parser are UNTRUSTED)      | fields cross onward
                                                   v
                                      +--------------------------+
                                      |  App logic / file access |
                                      |  (must never see raw ../)|
                                      +--------------------------+

Assets: the parser's memory and the files/app behind it. Trust boundary: the socket read — every byte from the client is untrusted. Entry point: the raw request buffer.

Knowledge check

1. What asset is protected by bounding the path copy, and which insecure assumption (about the path length) would cause an overflow if you ignored it?
2. Where is the trust boundary in the diagram, and why is Content-Length on the untrusted side of it?
3. Which single log field best lets a reviewer detect path-traversal probing, and why must this parsing only run against fixtures or an authorized lab rather than random internet hosts?

Syntax notes

Key building blocks for parsing HTTP text in C safely.

#include <string.h>
#include <ctype.h>

/* Find the CRLF that ends a line. memchr is safe on binary-ish buffers. */
char *line_end = memchr(buf, '\n', buf_len);   /* then check line_end[-1] == '\r' */

/* Find the header/body separator: the empty line \r\n\r\n */
char *sep = memmem(buf, buf_len, "\r\n\r\n", 4); /* GNU/BSD; body starts at sep+4 */

/* Bounded token copy: never write past out[outsz-1], always NUL-terminate. */
size_t n = (size_t)(tok_end - tok_start);
if (n > (size_t)outsz - 1) n = (size_t)outsz - 1;  /* clamp BEFORE copying */
memcpy(out, tok_start, n);
out[n] = '\0';

Notes:

• Prefer memchr/memmem over strchr/strstr when the buffer may contain embedded NUL bytes from the network. If you must use string functions, first confirm the region is NUL-terminated.
• Always clamp a length to outsz - 1 before the copy, then write the terminator.
• memmem is a POSIX-ish extension (Linux/BSD/macOS). If unavailable, write a small bounded search loop.

Lesson

HTTP/1.x is plain text

An HTTP/1.x request is just text sent over the network. Because it is human-readable, you can read it directly.

The structure looks like this:

METHOD path HTTP/version\r\n
Header-Name: value\r\n
Header2: value2\r\n
\r\n
optional body

Reading the parts

• Request line: the first line. It holds the method (such as GET or POST), the path being requested, and the HTTP version.
• Headers: one per line, in the form Name: value.
• Blank line: an empty line (\r\n by itself) marks the end of the headers.
• Body: optional data that follows the blank line.

\r\n is the carriage-return + line-feed pair that HTTP uses to end each line.

Why the format matters

Once you know the format, parsing logs becomes straightforward. It also lets you spot anomalies, such as:

• empty user-agent values
• unusual or unexpected methods
• paths containing ../

A safety note

The exercises parse sample requests from text fixtures. They never touch live traffic.

Code examples

The example parses one request from a fixture, extracts the path with a bounded buffer, and flags traversal — the exact shape you would use when reviewing captured, authorized traffic.

1) WARNING: intentionally vulnerable — use only in a local, isolated, authorized lab. Do not deploy.

/* insecure_path.c — DO NOT DEPLOY. Demonstrates the overflow this lesson prevents. */
#include <stdio.h>
#include <string.h>

/* BUG: copies the path with no bound. A long path smashes the stack. */
void bad_extract_path(const char *line, char *out) {
    const char *p1 = strchr(line, ' ');        /* after METHOD */
    const char *p2 = strchr(p1 + 1, ' ');      /* before VERSION */
    /* unbounded copy: attacker controls the distance p1+1 .. p2 */
    memcpy(out, p1 + 1, (size_t)(p2 - (p1 + 1)));
    out[p2 - (p1 + 1)] = '\0';                  /* writes wherever, no bound */
}

int main(void) {
    char out[16];                              /* tiny on purpose */
    /* A hostile request line with a 40-char path overflows out[16]. */
    const char *evil = "GET /AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA HTTP/1.1";
    bad_extract_path(evil, out);               /* undefined behaviour */
    printf("path=%s\n", out);
    return 0;
}

Why it is wrong: the copy length comes from the attacker-controlled request line, but out is only 16 bytes. This is a classic stack buffer overflow. Compile with -fsanitize=address and it will abort — proof of the flaw.

2) SECURE fix — bounded parse + traversal flag

/* safe_http.c — bounded HTTP request-line parsing and traversal detection.
 * Build: cc -std=c11 -Wall -Wextra -fsanitize=address -o safe_http safe_http.c
 */
#include <stdio.h>
#include <string.h>
#include <stddef.h>

/* Copy the PATH (2nd space-separated token) of an HTTP request line into out.
 * Returns 0 on success, -1 on malformed input. Never writes past out[outsz-1]. */
int request_path(const char *line, char *out, size_t outsz) {
    if (line == NULL || out == NULL || outsz == 0) return -1;

    const char *sp1 = strchr(line, ' ');           /* end of METHOD */
    if (sp1 == NULL) return -1;
    const char *start = sp1 + 1;                    /* first byte of PATH */
    const char *sp2 = strchr(start, ' ');           /* start of VERSION */
    if (sp2 == NULL) return -1;

    size_t n = (size_t)(sp2 - start);               /* raw path length */
    if (n > outsz - 1) n = outsz - 1;               /* CLAMP before copy */
    memcpy(out, start, n);
    out[n] = '\0';                                  /* always terminate */
    return 0;
}

/* Detect path-traversal indicators. Returns 1 if suspicious, else 0. */
int is_suspicious_path(const char *path) {
    if (path == NULL) return 0;
    if (strstr(path, "../") != NULL) return 1;      /* unix traversal */
    if (strstr(path, "..\\") != NULL) return 1;     /* windows traversal */
    if (strstr(path, "%2e%2e") != NULL) return 1;   /* url-encoded ..    */
    return 0;
}

int main(void) {
    /* Fixtures: one clean request, one traversal probe. No live traffic. */
    const char *good = "GET /reports/summary.html HTTP/1.1";
    const char *bad  = "GET /../../etc/passwd HTTP/1.1";

    char path[64];

    if (request_path(good, path, sizeof path) == 0)
        printf("GOOD path=%s suspicious=%d\n", path, is_suspicious_path(path));
    else
        printf("GOOD malformed\n");

    if (request_path(bad, path, sizeof path) == 0)
        printf("BAD  path=%s suspicious=%d\n", path, is_suspicious_path(path));
    else
        printf("BAD  malformed\n");

    return 0;
}

Expected output:

GOOD path=/reports/summary.html suspicious=0
BAD  path=/../../etc/passwd suspicious=1

3) VERIFY — the fix rejects bad input and accepts good input

/* verify.c — minimal checks. Build with the same flags, run ./verify. */
#include <assert.h>
#include <string.h>
#include <stddef.h>

int request_path(const char *line, char *out, size_t outsz);
int is_suspicious_path(const char *path);

int main(void) {
    char out[8];   /* deliberately small to test clamping */

    /* ACCEPTS good input, and NEVER overflows even when path > buffer. */
    assert(request_path("GET /aaaaaaaaaaaaaaaa HTTP/1.1", out, sizeof out) == 0);
    assert(strlen(out) == sizeof out - 1);       /* clamped, terminated */

    /* REJECTS malformed lines (no second space). */
    assert(request_path("GARBAGE", out, sizeof out) == -1);

    /* FLAGS traversal, does not flag clean paths. */
    assert(is_suspicious_path("/../../etc/passwd") == 1);
    assert(is_suspicious_path("/index.html") == 0);

    return 0;   /* all assertions hold => fix works */
}

Run both files under AddressSanitizer. The secure version never triggers a sanitizer abort; the assertions in verify.c all pass, proving the clamp both accepts good input and safely truncates oversized input instead of overflowing.

Line by line

Walkthrough of request_path on the fixture "GET /../../etc/passwd HTTP/1.1".

1. if (line == NULL || out == NULL || outsz == 0) return -1; — reject null/zero-size arguments up front so later pointer math is safe.
2. strchr(line, ' ') finds the first space, right after GET. sp1 now points at that space.
3. start = sp1 + 1; — start points at /, the first byte of the path.
4. strchr(start, ' ') finds the next space, right before HTTP/1.1. sp2 points there. If either space is missing, the line is malformed and we return -1 — we never read past the end.
5. n = sp2 - start; computes the raw path length (here 15 for /../../etc/passwd... counted between the spaces).
6. if (n > outsz - 1) n = outsz - 1; — the clamp. With path[64], 15 fits, so n stays 15. With the tiny out[8] in verify.c, n is forced down to 7, so the copy can never exceed the buffer.
7. memcpy(out, start, n); out[n] = '\0'; — copy exactly n bytes, then terminate. out now holds the path as a proper C string.
8. Back in main, is_suspicious_path runs strstr(path, "../"), finds a match, and returns 1.

Trace table for the clamp decision:

The key insight: the copy length is min(raw_len, outsz-1), so it is impossible to write past the buffer regardless of what the client sends.

Common mistakes

Debugging tips

Common failures and how to chase them down:

• Garbled path with a trailing box character. You forgot to strip \r. Print the token length and hex-dump the last byte (printf("%02x\n", (unsigned char)tok[len-1]);). If it's 0d, that's the \r.
• Segfault or ASan stack-buffer-overflow. A copy exceeded its buffer. Rebuild with -fsanitize=address -g; ASan prints the exact line and the overflowing size. Add the missing clamp.
• Parser reads past the buffer on a request with no spaces. A strchr returned NULL and you dereferenced it. Check every strchr/memchr result against NULL before using it.
• Headers and body run together. Your separator search used \n\n instead of \r\n\r\n. Search for the 4-byte sequence and start the body at sep + 4.
• Works on one fixture, hangs on another. You assumed a full request arrived in one read. In the lab, printf the number of bytes read each loop; loop until you see the blank line or reach your size cap.

Questions to ask when parsing fails: Did every strchr/memchr find what I expected (or return NULL)? Is the buffer NUL-terminated before I call string functions? Did I clamp before copying? Am I matching \r\n, not just \n? Is the input a real HTTP/1.x message, or binary HTTP/2 framing?

Memory safety

Memory safety (C). Every field you extract comes from an untrusted buffer, so: check each strchr/memchr for NULL before dereferencing; clamp every copy to buffer_size - 1 and write the '\0' yourself; prefer memchr/memmem when embedded NUL bytes are possible; never size an allocation or read count directly from a wire value like Content-Length without a hard cap. Build and test with -std=c11 -Wall -Wextra -fsanitize=address,undefined — ASan/UBSan turn silent overflows into loud, locatable aborts.

Security & safety — detection & logging. When you parse authorized traffic, log the security-relevant facts of each request so a reviewer can reconstruct events:

• Log: timestamp (UTC), source IP/port, method, requested path, HTTP version, response/decision (accepted, flagged, rejected), a correlation/request id, and which rule fired (e.g. flag=path_traversal).
• Never log: passwords, Authorization header values, session cookies, API tokens, private keys, full payment card numbers, or PII you don't need. If you must record that a token was present, log a fact like auth=present, never the token itself.
• Events that signal abuse: bursts of ..//%2e%2e paths from one source, unusual methods, duplicate Content-Length headers, or non-printable bytes in the request line.
• False positives: legitimate URLs can contain .. in query strings or filenames; a security scanner run by your own team will trip traversal rules. That's why flagging feeds a human review queue rather than auto-banning — and why this work is confined to systems you own or are authorized to test.

Real-world uses

Authorized use case. A blue-team analyst exports a day of requests from a company web server (a system the company owns) and runs a small parser like the one above to flag paths containing traversal sequences, then reviews the hits to decide whether they were probes, scanner noise, or benign. The parsing never touches third-party systems.

Professional habits

Always confirm scope before any lab that involves a running server: you may test only systems you own or are explicitly authorized to test. Labs belong on localhost, containers, intentionally-vulnerable VMs, or CTF environments.

Practice tasks

Beginner 1 — Split the request line. Objective: extract all three tokens of METHOD PATH VERSION into three bounded buffers. Requirements: int split_request_line(const char *line, char *m, size_t ms, char *p, size_t ps, char *v, size_t vs); return -1 if either space is missing. Input: "POST /login HTTP/1.1". Output: m=POST, p=/login, v=HTTP/1.1. Constraints: no writes past any buffer; always terminate. Hints: two strchr calls; clamp each copy. Concepts: bounded copy, strchr null-checks.

Beginner 2 — Find the header/body boundary. Objective: given a full request buffer, return the offset where the body starts. Requirements: long body_offset(const char *buf, size_t len) returns the index just after \r\n\r\n, or -1 if not found. Input: a fixture with two headers and a one-line body. Output: the integer offset. Constraints: use memmem or a bounded loop; do not assume NUL-termination. Hints: search the 4-byte sequence. Concepts: separator detection, binary-safe search.

Intermediate 1 — Parse one header safely. Objective: int header_value(const char *line, const char *name, char *out, size_t outsz) copies the value for a case-insensitive name, splitting on the first colon and trimming leading spaces; return -1 if the name doesn't match. Input: "Host: localhost:8080", name host. Output: localhost:8080. Constraints: bound the copy; split on first colon only. Hints: strncasecmp. Concepts: first-colon split, case-insensitive compare.

Intermediate 2 — Bound the body by a capped Content-Length. Objective: parse Content-Length, reject values above a hard cap (e.g. 1<<20), and report how many body bytes you would safely accept. Requirements: reject negative/non-numeric/duplicate length headers. Input: fixtures with valid, huge, and duplicate lengths. Output: accepted byte count or a rejection reason. Constraints: never allocate directly from the wire value. Hints: strtol with error checks, compare to cap. Concepts: never-trust-the-wire, hard caps. Defensive conclusion: verify with a fixture whose length claims 1 GB and confirm your code rejects it — that rejection is the mitigation, and log the decision.

Challenge — Anomaly-flagging request reviewer (lab only). Objective: read a fixture file of many requests (one per record) and print a report flagging traversal paths, unusual methods (anything outside a small allowlist), missing Host, and duplicate Content-Length. Requirements: bounded parsing throughout; a per-request line of id, method, path, flags; a summary count per flag. Constraints: fixtures only, no sockets to third parties; run under ASan. Hints: reuse your earlier functions; build the method allowlist as an array. Concepts: detection vs prevention, structured logging. Authorization checklist before any live-lab variant: (1) I own or am authorized in writing to test this target; (2) it runs on localhost/container/VM/CTF; (3) I have a rollback/reset plan; (4) I log decisions, not secrets. Defensive conclusion: your report is a detection aid for a human reviewer — remediate by canonicalizing and denying traversal at the file layer, then re-run the fixture to verify the flags now correspond to blocked requests. Cleanup/reset: delete generated logs and restore fixture files from a pristine copy after the exercise.

Summary

• An HTTP/1.x request is plain text in four regions: request line, headers, a blank line (\r\n\r\n), then an optional body.
• The request line is METHOD PATH VERSION; split it with two spaces and copy each token into a bounded buffer.
• Headers are Name: value — split on the first colon, and treat Content-Length as an untrusted claim to be capped, never as a safe allocation size.
• Key syntax: strchr/memchr for delimiters (null-check them), memmem for the \r\n\r\n separator, and clamp every copy to outsz-1 before writing plus a '\0'.
• Common mistakes: unbounded copies (overflow), splitting on \n only, splitting headers on every colon, and trusting the wire's length.
• Detection is not prevention: flag suspicious paths for human review, log the facts (source, method, path, decision, correlation id) but never secrets, and only ever parse fixtures or systems you are authorized to test.
• Remember: decoding or parsing a request tells you what it claims; safety comes from bounding, capping, and failing closed.

Practice with these exercises

• Extract the request path
• Suspicious request path?