HTTP, HTTPS, and TLS

Overview

The web runs on a beautifully simple idea: a client sends a request made of text, and a server sends back a response made of text. That protocol is called HTTP (HyperText Transfer Protocol). When you type a URL, click a link, or an app calls an API, an HTTP request goes out and an HTTP response comes back.

The catch is that plain HTTP travels in the clear. Anyone on the path — a coffee-shop Wi-Fi router, an ISP, a compromised switch — can read it and change it. HTTPS fixes this. HTTPS is not a different protocol; it is the same HTTP, but sent through an encrypted tunnel built by TLS (Transport Layer Security). Plain HTTP normally uses port 80; HTTPS uses port 443.

This lesson builds directly on TCP vs UDP, ports, and the ones to know. There you learned that TCP gives you a reliable, ordered byte stream and that ports (like 80 and 443) tell the operating system which program should handle an incoming connection. HTTP is just an agreement about what bytes to send over that TCP stream. TLS sits between TCP and HTTP: TCP moves the bytes, TLS encrypts them, and HTTP gives them meaning.

In plain language: HTTP is the language two computers speak; TLS is the sealed envelope that language travels inside. In the terminology you will meet below, HTTP is an application-layer protocol, TLS is a security layer, and HTTPS is simply "HTTP over TLS."

Why it matters

Almost everything you touch online is HTTP or HTTPS. Web pages, mobile-app backends, REST APIs, cloud dashboards, webhooks, software updates, and internal microservices all speak it. If you can read a raw HTTP exchange, you can debug half the problems in modern software without any special tools.

• Debugging and health checks. Load balancers, uptime monitors, and orchestration systems decide whether a service is alive by reading its status line. A 200 OK means "send traffic here"; a 503 means "take it out of rotation." Knowing exactly where that status line lives in the bytes is the difference between a working health-checker and a broken one.
• Headers reveal behavior. Response headers expose caching rules, cookies, content types, and security policies. Most web-security findings surface in headers, status codes, and cookies.
• TLS both protects and reveals. TLS keeps traffic private, but the certificate it presents is public. Its Subject Alternative Names (SANs), issuer, and expiry dates are visible to anyone who connects — useful for mapping what hostnames a server serves, and for catching an expired or mismatched certificate before it breaks users.
• Correctness matters at scale. A parser that grabs the status code from the wrong byte, or that trusts a Content-Length it never validated, becomes a reliability or security bug when it runs a billion times a day.

Core concepts

1. The request: request line, headers, blank line, body

An HTTP request is text with a strict shape. The very first line is the request line: a method, a path, and a protocol version. After it come headers, one per line, each Name: Value. Then a single blank line marks the end of the headers. Anything after that blank line is the body.

GET /index.html HTTP/1.1        <- request line: METHOD PATH VERSION
Host: example.com               <- header
User-Agent: curl/8.0            <- header
Accept: */*                     <- header
                                <- BLANK LINE ends the headers
(body goes here, if any)

Each line ends with the two bytes \r\n (carriage return + line feed), and the blank line is an extra \r\n. That \r\n\r\n sequence is how a program knows the headers are finished. Common methods: GET (fetch), POST (send data), PUT, DELETE, HEAD.

When to use which method: GET for reading (no body, safe to repeat), POST for creating or submitting (has a body). Do not put sensitive data in a GET query string — it lands in logs and browser history.

Pitfall: forgetting the Host header. In HTTP/1.1 it is mandatory, because one server (one IP) hosts many sites; the Host header is how the server knows which one you want. Leave it out and you often get a 400 Bad Request.

Knowledge check: In the request above, what exactly separates the headers from the body — a keyword, a byte count, or a blank line?

2. The response: status line, headers, body

The response mirrors the request. Its first line is the status line: version, a three-digit status code, and a human reason phrase.

HTTP/1.1 200 OK                 <- status line: VERSION CODE REASON
Content-Type: text/html         <- header
Content-Length: 1256            <- header (body length in bytes)
                                <- BLANK LINE
<html>...</html>                <- body

The Content-Length header tells the client how many body bytes to read. The reason phrase (OK, Not Found) is for humans only — programs act on the numeric code.

Knowledge check (predict the output): A health-checker reads only the first line of a response and splits it on spaces. Given HTTP/1.1 404 Not Found, what are the three pieces, and which one is the status code?

3. Status code classes

The first digit tells you the category. Memorize the classes; you can look up individual codes.

Pitfall: treating 401 and 403 as the same. 401 Unauthorized means "you are not authenticated — log in." 403 Forbidden means "you are authenticated, but you are not allowed." The distinction changes what the client should do next.

4. TLS: the sealed envelope under HTTPS

Before any HTTP bytes flow over an HTTPS connection, TLS runs a handshake to build a secure tunnel. TLS provides three guarantees:

TLS does not guarantee availability — it cannot stop a server from crashing or being flooded offline. (That is exactly the point of this lesson's quiz.)

Here is the handshake at a high level:

Client                                   Server
  |  --- ClientHello (versions, ciphers, SNI) --->  |
  |  <-- ServerHello (chosen cipher) ------------   |
  |  <-- Certificate (server's public cert chain)   |
  |  --- key exchange messages -----------------> <-- |
  |  === both sides derive the same session key === |
  |  <===== encrypted HTTP now flows =============> |

The SNI (Server Name Indication) field in the ClientHello is sent in plaintext so the server knows which site's certificate to present — one IP can serve many domains. The server proves its identity by sending a certificate whose chain leads up to a CA your system already trusts. Your client checks: is the signature valid, is it within the validity dates, and does the hostname match a name in the certificate?

When NOT to disable verification: in real code, never turn off certificate checking to "make it work." That silently removes the authentication guarantee and invites a man-in-the-middle. Fix the trust store instead.

Knowledge check (explain in your own words): TLS gives confidentiality, integrity, and authentication. Why does none of those stop an attacker from taking a server offline?

5. The certificate as data you can read

A certificate is public — anyone who connects sees it. Useful fields:

• Subject / SAN (Subject Alternative Name): the hostnames this certificate is valid for, e.g. example.com, www.example.com, api.example.com.
• Issuer: which CA signed it.
• Validity (notBefore / notAfter): the date window in which it is valid.

For an operator, these fields catch real bugs: a certificate that expired last night, or one whose SAN list does not include the hostname users actually type (which makes browsers show a scary warning).

Pitfall: assuming a valid certificate means the site is safe. A certificate only proves you are talking to the owner of that name over an encrypted channel. A phishing site can hold a perfectly valid certificate for its own look-alike domain.

Syntax notes

HTTP has no C-style syntax — it is a text format. The structure to internalize:

<start-line>\r\n            request line OR status line
<Header-Name>: <value>\r\n  zero or more headers
<Header-Name>: <value>\r\n
\r\n                        blank line: END OF HEADERS
<optional body bytes>      length given by Content-Length

Key rules:

• Every line ends with \r\n (CRLF), not just \n.
• The header section always ends with \r\n\r\n.
• Header names are case-insensitive; Host: and host: are the same.
• A request line is METHOD SP PATH SP VERSION; a status line is VERSION SP CODE SP REASON (SP = a single space).

URLs also have structure: https://example.com:443/path?query#fragment — scheme, host, optional port, path, optional query, optional fragment.

Lesson

HTTP is the request/response protocol of the web (port 80). HTTPS is the same protocol wrapped in TLS for encryption (port 443). TLS (Transport Layer Security) is the standard that encrypts traffic between client and server.

An HTTP request

A request starts with a request line, followed by headers:

GET /index.html HTTP/1.1
Host: example.com
User-Agent: curl/8.0
Accept: */*

A blank line ends the headers. A body, if there is one, follows after it.

The response has the same shape:

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 1256

<html>...</html>

Status codes

Status codes are grouped by their first digit:

• 2xx success (200 OK)
• 3xx redirect (301, 302)
• 4xx client error (401 Unauthorized, 403 Forbidden, 404 Not Found)
• 5xx server error (500, 503)

What TLS adds

TLS provides three guarantees:

• Confidentiality — the traffic is encrypted.
• Integrity — tampering is detected.
• Authentication — the server's certificate proves its identity.

Before any HTTP flows, a TLS handshake runs. It negotiates a cipher, validates the certificate chain, and derives the session keys.

The certificate also leaks useful recon: hostnames in the SAN field, the issuer, and the validity dates.

Why it matters

Web application testing is largely about reading and manipulating these requests and responses.

Headers, status codes, and cookies are where most web vulnerabilities surface. TLS misconfigurations are findings in their own right — for example, weak ciphers, or certificates that are expired or do not match the hostname.

Code examples

#include <stdio.h> #include <string.h> #include <ctype.h>

/*

• Parse the status LINE of an HTTP response: "HTTP/1.1 404 Not Found".

• Writes the numeric code to *out_code and the reason phrase into

• out_reason (a buffer of size cap). Returns 0 on success, -1 on a

• malformed line. This is the first thing a health-checker reads. */ int http_parse_status(const char *line, int *out_code, char out_reason, size_t cap) { if (line == NULL || out_code == NULL || out_reason == NULL || cap == 0) return -1; / defensive: no NULLs */

  /* 1) The version token must start with "HTTP/" */ if (strncmp(line, "HTTP/", 5) != 0) return -1;

  /* 2) Skip the version token up to the first space */ const char p = strchr(line, ' '); if (p == NULL) return -1; p++; / now at the code */

  /* 3) Read exactly a 3-digit code */ int code = 0, digits = 0; while (isdigit((unsigned char)p)) { / cast: isdigit wants unsigned */ code = code * 10 + (p - '0'); p++; digits++; } if (digits != 3) return -1; / HTTP codes are 3 digits */ *out_code = code;

  /* 4) One space, then the reason phrase runs to end of line (\r or \n) */ if (*p != ' ') return -1; p++;

  size_t i = 0; while (*p != '\0' && *p != '\r' && *p != '\n' && i + 1 < cap) { out_reason[i++] = p++; / i+1<cap leaves room for '\0' / } out_reason[i] = '\0'; / always NUL-terminate */ return 0; }

int main(void) { const char responses[] = { "HTTP/1.1 200 OK\r\n", "HTTP/1.1 404 Not Found\r\n", "HTTP/1.1 503 Service Unavailable\r\n", "GARBAGE not a status line\r\n" / should fail */ }; size_t n = sizeof(responses) / sizeof(responses[0]);

for (size_t k = 0; k < n; k++) {
    int code = 0;
    char reason[64];
    if (http_parse_status(responses[k], &code, reason, sizeof reason) == 0) {
        const char *class_name =
            code / 100 == 2 ? "success"  :
            code / 100 == 3 ? "redirect" :
            code / 100 == 4 ? "client error" :
            code / 100 == 5 ? "server error" : "unknown";
        printf("code=%d reason=\"%s\" class=%s\n", code, reason, class_name);
    } else {
        printf("malformed status line: %.20s...\n", responses[k]);
    }
}
return 0;

}

/*

• Expected output:
• code=200 reason="OK" class=success
• code=404 reason="Not Found" class=client error
• code=503 reason="Service Unavailable" class=server error
• malformed status line: GARBAGE not a status...
• Edge cases: a truncated line with no reason phrase still parses the
• code and yields an empty reason; a reason longer than the buffer is
• safely truncated because the loop stops at i+1 < cap. */

Line by line

The program parses four status lines and classifies each code. Walk through http_parse_status:

1. Guard clause. We reject NULL pointers and a zero-size buffer immediately. This is the cheapest bug prevention there is.
2. Version check. strncmp(line, "HTTP/", 5) compares the first five bytes. If they are not HTTP/, this is not a status line, so we return -1. That is why the "GARBAGE..." input fails.
3. Find the first space. strchr(line, ' ') returns a pointer to the space after the version token (HTTP/1.1). p++ steps past the space so p now points at the first digit of the code.
4. Read three digits. The while (isdigit(...)) loop builds the number: code = code*10 + (*p - '0'). For 404 it computes 0 -> 4 -> 40 -> 404. We count digits and insist on exactly 3.
5. Reason phrase. After one required space, we copy characters into out_reason until we hit end-of-line or run out of buffer. The condition i + 1 < cap guarantees room for the terminating '\0'.

Here is the pointer walking across HTTP/1.1 404 Not Found:

Back in main, code / 100 maps the code to a class: 404 / 100 == 4, so it prints class=client error. Integer division is the whole trick behind the status-class table.

Common mistakes

Mistake 1 — splitting the response on \n alone.

/* WRONG: assumes lines end in \n and forgets the \r */
char *nl = strchr(response, '\n');
*nl = '\0';   /* leaves a stray '\r' at the end of the line */

HTTP lines end in \r\n. If you split on \n only, a trailing \r clings to your token and later comparisons like strcmp(reason, "OK") fail mysteriously. Fix: stop at either \r or \n (as the lesson code does), or trim the \r explicitly. Recognize it: values that "look right" in a debugger but never compare equal.

Mistake 2 — trusting Content-Length blindly.

/* WRONG: read exactly Content-Length bytes with no sanity check */
char *body = malloc(content_length);          /* what if it's huge or negative? */
fread(body, 1, content_length, sock);

A hostile or buggy server can send a Content-Length that is enormous or does not match the real body. Allocating or reading on that number unchecked leads to memory exhaustion or a hang. Fix: validate the header is a non-negative integer and cap it against a sane maximum before allocating.

Mistake 3 — disabling certificate verification to "fix" TLS.

curl --insecure https://api.example.com   # or SSL_VERIFY_NONE in code

This makes the error go away and silently removes the authentication guarantee — now any machine on the path can impersonate the server. Fix: install or point to the correct CA bundle; only ever disable verification against a throwaway test server you fully control. Recognize it: "it works after I turned off SSL checking" is a red flag, not a solution.

Mistake 4 — confusing 401 and 403 in a client retry loop. Retrying a 403 with the same credentials will never succeed (you lack permission), while retrying a 401 after logging in might. Treating them the same causes infinite retry storms.

Debugging tips

Compiler errors (for the C example):

• implicit declaration of function 'isdigit' → you forgot #include <ctype.h>.
• Passing a plain char to isdigit can trigger -Wchar-subscripts or undefined behavior on negative values; cast to unsigned char as the code does.

Runtime errors:

• Buffer overrun in the reason copy → double-check the loop bound is i + 1 < cap, not i < cap. Run under AddressSanitizer (gcc -fsanitize=address) and it will pinpoint the write.
• Parser returns -1 on a real response → print the first 20 bytes in hex. You will often find a leading byte-order mark, a proxy banner, or that you passed the whole response instead of just the first line.

Logic errors:

• Wrong status code extracted → you probably started reading before skipping the version token. Verify p points at a digit right before the digit loop.
• Headers "never end" → you are searching for \n\n but the real delimiter is \r\n\r\n.

Debugging a live HTTPS problem — questions to ask:

1. Is the failure in TLS (handshake) or HTTP (after the tunnel is up)? A connection that dies before any HTTP response is usually TLS.
2. Does the certificate's SAN include the exact hostname you connected to?
3. Is the certificate within its validity dates? Check notAfter.
4. Does the server support the TLS version and cipher your client offers? Tools like openssl s_client -connect host:443 print the negotiated version, the certificate chain, and the SANs — the fastest way to see what the server actually presents.

Memory safety

Parsing untrusted network text in C is a classic source of memory-safety bugs. For this topic specifically:

• Bounds on every copy. The reason-phrase loop uses i + 1 < cap so it can never write past out_reason, and it always writes a terminating '\0'. Any time you copy header or body bytes into a fixed buffer, the copy length must be bounded by the destination size, not by the input.
• Never trust length fields from the network. Content-Length, chunk sizes, and TLS record lengths all come from the other side. Validate them (non-negative, within a maximum, consistent with bytes actually received) before using them to allocate or index. An unchecked length is how buffer overflows and out-of-memory crashes happen.
• NUL-terminate before using string functions. Network data is not automatically NUL-terminated. Calling strchr/strcmp on a buffer that is not terminated reads out of bounds. Terminate deliberately, or use length-bounded functions.
• Initialize outputs. out_code and out_reason are written only on the success path; the caller must check the return value before reading them. Returning -1 without touching the outputs is fine as long as the contract ("check the return first") is documented and followed.
• isdigit/isspace want unsigned char. Passing a possibly-negative char is undefined behavior. Cast, as the example does.

Defensive/security practices for HTTP and TLS:

• Validate all input. Reject malformed status lines, absurd header counts, and oversized bodies early. A parser is an attack surface.
• Keep certificate verification ON. Confidentiality without authentication is nearly worthless — an encrypted channel to an impostor is still an encrypted channel to an impostor. If you must show a TLS weakness for learning, label it clearly and pair it with the fix: use only certificates and servers you control in a lab, never a real target, and never ship verification-disabled code.
• Prefer safe, well-reviewed libraries (OpenSSL, BoringSSL, mbedTLS) over hand-rolled crypto or hand-rolled parsers in production.

Real-world uses

Where this shows up in real systems:

• Load balancers and uptime monitors read the status line to decide whether a backend is healthy — exactly what the lesson's parser does. A wrong parse routes traffic to a dead server.
• REST APIs and mobile backends are HTTP end to end; status codes and JSON bodies are the contract between app and server.
• CDNs and reverse proxies (nginx, Cloudflare, Envoy) rewrite headers, terminate TLS, and forward requests based on the Host header and path.
• Certificate monitoring tools connect, read the SAN and notAfter, and alert on certificates about to expire — a very common outage cause.
• Security tooling inspects headers, cookies, and TLS configuration (weak ciphers, hostname mismatches) as part of routine assessments — always against systems the tester is authorized to touch.

Professional habits:

• Beginner: end every line with \r\n, always send the Host header, check the return value of your parser before using its outputs, and read status codes by class rather than memorizing every number.
• Advanced: stream large bodies instead of buffering them, enforce maximum header and body sizes, handle chunked transfer encoding, keep certificate verification on with an up-to-date CA bundle, prefer modern TLS (1.2+, ideally 1.3), pin timeouts on every network call, and log the status class of every response for observability. Name variables for what they hold (status_code, reason_phrase), and clean up sockets and buffers on every path, including errors.

Practice tasks

Beginner 1 — Classify a code. Write a function const char *status_class(int code) that returns "success", "redirect", "client error", "server error", or "unknown". Requirements: use integer division by 100; handle codes outside 100–599 as "unknown". Example: status_class(204) -> "success", status_class(302) -> "redirect". Concepts: status classes, integer division.

Beginner 2 — Find the blank line. Given a full HTTP response in a char *, write a function that returns a pointer to the first body byte (the byte right after \r\n\r\n), or NULL if the header terminator is not present. Requirement: search for the exact 4-byte sequence. Hint: strstr(response, "\r\n\r\n"), then add 4. Concepts: header/body boundary, CRLF.

Intermediate 1 — Build a GET request. Implement int build_http_get(const char *host, const char *path, char *out, size_t cap) that writes a minimal, valid HTTP/1.1 GET request into out. It must include the request line, a Host header, and end with a blank line. Requirements: never write past cap; return the number of bytes written or -1 on truncation. Example output for host example.com, path /: GET / HTTP/1.1\r\nHost: example.com\r\n\r\n. Hint: snprintf returns the length it would have written — compare it to cap. Concepts: request structure, bounded formatting.

Intermediate 2 — Extract a header value. Write int get_header(const char *response, const char *name, char *out, size_t cap) that finds a header (case-insensitive) and copies its value (trimmed) into out. Requirements: search only within the header section (before \r\n\r\n); return 0 on success, -1 if the header is absent. Input/output: for header Content-Type, extract text/html. Hint: match name followed by :, then skip spaces. Concepts: headers, case-insensitive matching, bounds.

Challenge — Certificate summary reader (concept + safety). Without doing any live networking, write a small program that takes a pre-parsed structure describing a certificate (its SAN list, issuer string, and notAfter date as YYYY-MM-DD) and a hostname + today's date, and prints a verdict: VALID, EXPIRED, or HOSTNAME MISMATCH. Requirements: hostname must exactly match one SAN entry; the date comparison must treat an equal date as still valid; handle an empty SAN list as a mismatch. Hint: compare dates as strings since YYYY-MM-DD sorts lexicographically. Concepts: certificate fields (SAN, validity), authentication logic, defensive checks. Do NOT connect to any real server — operate only on the supplied data.

Summary

• HTTP is the web's text request/response protocol (port 80). A request is a request line + headers + a blank line + optional body; a response swaps the request line for a status line. Every line ends in \r\n, and \r\n\r\n ends the headers.
• Status codes are grouped by first digit: 2xx success, 3xx redirect, 4xx client error, 5xx server error. Read by class; code / 100 gives you the class.
• HTTPS is HTTP carried over TLS (port 443). TLS provides confidentiality, integrity, and server authentication via a certificate chain — but not availability.
• The TLS handshake negotiates a cipher, validates the certificate (signature, dates, hostname/SAN), and derives a session key before any HTTP flows. SNI is the plaintext hostname in the ClientHello.
• A certificate is public data: SANs, issuer, and validity dates are readable and useful for both robustness (catch expiry/mismatch) and reconnaissance.
• Most important habits: end lines with \r\n, always send Host, never trust length fields from the network, bound every copy and NUL-terminate, and keep certificate verification ON.

Practice with these exercises

• Final Project: HTTP status-line parser
• Build an HTTP GET request
• Extract the SNI host from a TLS extension