Checksums — integrity over the wire

Overview

When bytes travel across a network cable, a radio link, a serial bus, or even just from disk into memory, they can get corrupted. A single voltage glitch flips a bit; a noisy Wi-Fi channel garbles a frame; a flaky USB cable drops a byte. The receiver needs a cheap way to ask: "did I get exactly what was sent?"

A checksum answers that question. It mixes every input byte into a small, fixed-size number (often 8, 16, or 32 bits). The sender computes the checksum over the data and ships it alongside. The receiver recomputes it over the bytes it actually received and compares. If the two numbers differ, the data was corrupted in transit and is discarded or re-requested.

This lesson builds directly on Bitwise operations: every checksum here is built from shifts (<<, >>), masks (&), and XOR (^). If you are comfortable reading c = (c << 1) ^ 0x07, you already have the tools. We are just arranging those operations into algorithms with useful mathematical properties.

A word on terminology before we start. People loosely say "checksum" for three different families of functions, and conflating them causes real security bugs:

• A checksum (Internet checksum, Fletcher, Adler) — a simple running sum, fast, catches most accidental corruption.
• A CRC (Cyclic Redundancy Check) — polynomial division over binary fields, much stronger at catching burst errors, used in Ethernet and storage.
• A cryptographic hash / MAC (SHA-256, BLAKE3, HMAC) — designed so that an intelligent attacker cannot forge a matching value.

The first two protect against accidents. Only the third protects against adversaries. Keeping that line sharp is the single most important idea in this lesson, and it leads straight into the next lessons on Sockets — where you will actually see these checksums on real packets.

Why it matters

Almost every wire protocol and binary file format on your machine carries a checksum. The Ethernet frame leaving your network card ends in a 32-bit CRC. Every IPv4, TCP, and UDP header has a 16-bit Internet checksum. ZIP, PNG, gzip, and ELF files embed CRC-32 values so a corrupt download is caught before it crashes a program. A temperature sensor on a 1-Wire bus appends a CRC-8 so the microcontroller never acts on a garbled reading.

Getting integrity right is also a safety and reliability concern. A storage system that silently accepts a corrupted block returns wrong data to applications — "silent data corruption," one of the hardest classes of bug to diagnose. A firmware updater that flashes a corrupted image can brick a device. Checksums are the cheap first line of defense against all of this.

There is a security angle too, and it cuts the other way. A common, serious mistake is treating a checksum or CRC as if it proves authenticity. It does not. An attacker who can modify the data can trivially recompute the checksum, so the receiver sees a perfectly valid-looking message. Understanding exactly where checksums stop — and where you must escalate to a MAC — is a core defensive-engineering skill that the quiz at the end of this lesson zeroes in on.

Finally, when you reverse-engineer an unknown binary blob or debug a custom protocol, one of the first practical steps is "find the checksum." Recognizing a one's-complement sum versus a CRC, and knowing the standard polynomials, lets you validate or regenerate frames you didn't design.

Core concepts

Each major idea below is taught on its own: what it is, how it works internally, when to use it, and a pitfall to avoid.

1. What a checksum actually computes

Definition. A checksum is a function that maps an arbitrary-length byte buffer to a small fixed-size integer, designed so that common corruption changes the output.

Plain language. Think of it as a fingerprint that is fast to take and good enough to catch smudges, but not unique enough to stop a forger.

How it works internally. The sender computes c = f(data) and transmits data || c (the data followed by the checksum). The receiver splits the message, recomputes f(data) over the received bytes, and checks it equals the received c. Many protocols use a clever twist: the checksum field is chosen so that running the function over the whole message (data + checksum) yields a fixed constant (often 0). That makes verification a single comparison.

Sender                          Receiver
------                          --------
data ──► f() ──► c              data' ──► f() ──► c'
send: [ data | c ] ───────────► split into data', c
        (link may flip bits)    compare c' vs received c
                                equal   → accept (probably intact)
                                differ  → reject (definitely corrupt)

When to use / not use. Use a checksum whenever you need to detect accidental corruption cheaply. Do not use it when an attacker controls or can modify the channel — see concept 5.

Pitfall. "The checksum matched, so the data is correct." A match only means no detected error. Weak checksums miss some corruptions, and any checksum can be deliberately recomputed by a tamperer.

Knowledge check: In your own words, what is the difference between "the checksum matched" and "the data is authentic"?

2. Parity and the XOR checksum (BCC)

Definition. The simplest checksum: XOR all bytes together into one byte (also called a Block Check Character).

How it works. bcc = d[0] ^ d[1] ^ ... ^ d[n-1]. Each bit position is the parity (even/odd count of 1s) of that column across all bytes.

When to use. Extremely cheap serial protocols where you only need to catch single-bit glitches. When not to: anything important — it misses any error that flips an even number of bits in the same column (e.g., two bytes both changing the same bit cancel out).

Pitfall. Reordering bytes does not change an XOR checksum at all, because XOR is commutative. A frame whose bytes are shuffled passes the check.

3. The Internet checksum (16-bit one's-complement sum)

Definition. Treat the data as a sequence of big-endian 16-bit words, add them with one's-complement arithmetic, then take the bitwise complement of the result.

One's-complement addition, plainly. Add normally in a wider accumulator. Whenever the sum overflows past 16 bits, take the carry bits that spilled out and add them back into the low 16 bits. Repeat until no carry remains. This "end-around carry" is what makes the sum independent of where you split the data.

Internal structure / data flow.

bytes:  B0 B1 B2 B3 B4 B5 ...
words:  [B0 B1] [B2 B3] [B4 B5]   (each = (Bk<<8) | Bk+1, big-endian)
          w0      w1      w2
sum = w0 + w1 + w2 + ...          (in a 32-bit accumulator)
fold:   while (sum >> 16) sum = (sum & 0xFFFF) + (sum >> 16);
result: ~sum (low 16 bits)

Why one's complement? It is endianness-friendly in a useful way and lets the receiver verify by summing data plus the stored checksum and expecting all-ones (0xFFFF), which complements to 0.

When to use / not use. Use it to match IP/TCP/UDP/ICMP, or for a cheap 16-bit integrity field. Don't rely on it for burst errors — it is weaker than a CRC and, famously, cannot detect reordering of 16-bit words or certain compensating errors.

Pitfall — odd length. If the buffer has an odd number of bytes, the last lone byte must be padded with a zero byte on the right (treated as the high byte of a final word). Forgetting this is the most common Internet-checksum bug.

Knowledge check (predict the output): For the two bytes 0xFF, 0xFF, the single 16-bit word is 0xFFFF. There is no carry to fold. What is the final checksum after taking the complement?

4. CRC (Cyclic Redundancy Check)

Definition. A CRC treats the message as the coefficients of a big polynomial over GF(2) — binary arithmetic where addition and subtraction are both XOR and there are no carries. It divides that polynomial by a fixed generator polynomial and uses the remainder as the check value.

Plain language. It is long division, but with XOR instead of subtraction. The remainder is extremely sensitive to changes: flip any bit, or corrupt any short burst of bits, and the remainder almost always changes.

How the bitwise loop works (CRC-8, poly 0x07). Start the register at 0. For each byte, XOR it into the register, then for each of its 8 bits: if the top bit is set, shift left and XOR the polynomial; otherwise just shift left. The top-bit test is asking "does the divisor 'fit' here?" exactly like a division step.

register c (8 bits)
for each byte d:
    c ^= d
    repeat 8 times:
        msb = c & 0x80
        c <<= 1
        if msb: c ^= 0x07     <- subtract the generator (XOR) when it fits
final c = CRC-8

Strength. A well-chosen n-bit CRC detects: all single-bit errors, all double-bit errors (within a range), any odd number of bit errors, and all burst errors up to n bits long. This is why Ethernet uses CRC-32 — burst noise on a wire is exactly what it is tuned to catch.

Common polynomials.

When to use / not use. Use a CRC when you need strong detection of transmission/storage corruption, especially bursts. Do not use it as a security check — see concept 5. Also do not mix up parameters: CRCs vary by polynomial, initial value, bit-reflection, and final XOR. The string 123456789 is the standard test vector to confirm you matched a spec.

Pitfall. Using a real CRC-32 value but the wrong bit order (reflected vs. non-reflected) gives a number that looks plausible but never matches the standard. Always validate against a published test vector.

Knowledge check (find the bug): A teammate's CRC-8 loop reads c = c << 1 ^ 0x07; for every bit unconditionally — no top-bit test. Why is the output not a CRC, and what real operation did they accidentally write?

5. Checksums vs. cryptographic integrity (the security line)

Definition. A MAC (Message Authentication Code) such as HMAC-SHA-256 combines a secret key with the data so that only someone holding the key can produce a valid tag. A plain cryptographic hash (SHA-256, BLAKE3) is collision-resistant but, by itself, is keyless.

Why checksums fail against attackers. Every checksum and CRC is a public, keyless function. An attacker who edits the payload simply recomputes the checksum over their forged data and attaches it. The receiver's comparison passes. There is no secret involved, so there is nothing to stop them.

Threat model (why a CRC is not enough)
--------------------------------------
Asset:        the integrity/authenticity of a message
Entry point:  a network path an attacker can read AND modify (MITM)
Trust boundary: sender ──[ untrusted link ]── receiver
CRC defends:  accidental bit flips  ✔
CRC ignores:  attacker rewrites payload + recomputes CRC  ✘ (forgery passes)
MAC defends:  forgery — attacker lacks the secret key      ✔

Defensive habit — pick the right primitive:

Pitfall. "We hash the file with SHA-256, so it's tamper-proof." A bare hash is only tamper-evident if the hash itself is delivered over a channel the attacker can't touch (or is signed). If the attacker can replace both the file and its posted hash, you need a signature or a MAC, not just a hash. Never invent your own integrity scheme — use vetted primitives.

Knowledge check: An app downloads an update over plain HTTP and verifies a CRC-32 embedded in the file. Why does this provide zero protection against a network attacker?

Syntax notes

Typical signatures use uint*_t from <stdint.h> so widths are explicit and portable:

#include <stdint.h>
#include <stddef.h>

/* 16-bit one's-complement Internet checksum (IP/TCP/UDP) */
uint16_t inet_checksum(const uint8_t *buf, size_t len);

/* Bitwise CRC-8, generator polynomial 0x07, init 0, no reflection */
uint8_t  crc8(const uint8_t *data, size_t len);

Key building blocks, all from Bitwise operations:

uint16_t word = ((uint16_t)buf[i] << 8) | buf[i + 1]; /* big-endian 16-bit word */
sum = (sum & 0xFFFF) + (sum >> 16);                   /* fold end-around carry  */
uint16_t cks = (uint16_t)~sum;                        /* one's complement       */
c = (c & 0x80) ? (uint8_t)((c << 1) ^ 0x07) : (uint8_t)(c << 1); /* CRC step */

Note the casts back to uint8_t/uint16_t: in C, operands smaller than int are promoted to int before shifting, so you must mask or cast to keep the value in range.

Lesson

A checksum compresses a buffer into a small integer. The goal: any single byte change should produce a different output.

The Internet checksum (a one's complement sum) appears in every IP, TCP, and UDP header.

CRC-8, CRC-16, and CRC-32 are used in 1-Wire, Ethernet, ZIP, PNG, and USB.

Code examples

#include <stdio.h>
#include <stdint.h>
#include <stddef.h>

/* 16-bit one's-complement Internet checksum (IP/TCP/UDP/ICMP). */
uint16_t inet_checksum(const uint8_t *buf, size_t len) {
    uint32_t sum = 0;                 /* wide enough to hold carries */
    size_t i = 0;

    for (; i + 1 < len; i += 2)       /* add complete 16-bit words */
        sum += ((uint32_t)buf[i] << 8) | buf[i + 1];

    if (i < len)                      /* odd length: pad last byte on the right */
        sum += (uint32_t)buf[i] << 8;

    while (sum >> 16)                 /* fold all carry bits back in */
        sum = (sum & 0xFFFF) + (sum >> 16);

    return (uint16_t)~sum;            /* one's complement of the folded sum */
}

/* Bitwise CRC-8, polynomial 0x07, init 0, no reflection, no final XOR. */
uint8_t crc8(const uint8_t *data, size_t len) {
    uint8_t c = 0;
    for (size_t i = 0; i < len; i++) {
        c ^= data[i];
        for (int b = 0; b < 8; b++)
            c = (c & 0x80) ? (uint8_t)((c << 1) ^ 0x07)
                           : (uint8_t)(c << 1);
    }
    return c;
}

int main(void) {
    /* Standard CRC test vector: ASCII "123456789" */
    const uint8_t vec[] = { '1','2','3','4','5','6','7','8','9' };
    size_t n = sizeof vec;

    printf("CRC-8(123456789)  = 0x%02X\n", crc8(vec, n));

    /* A real 20-byte IPv4 header with its checksum field zeroed.
       The correct checksum for this header is 0xB861. */
    uint8_t iphdr[20] = {
        0x45, 0x00, 0x00, 0x3C, 0x1C, 0x46, 0x40, 0x00,
        0x40, 0x06, 0x00, 0x00,            /* checksum field = 0x0000 */
        0xAC, 0x10, 0x0A, 0x63,            /* src 172.16.10.99 */
        0xAC, 0x10, 0x0A, 0x0C             /* dst 172.16.10.12 */
    };
    uint16_t cks = inet_checksum(iphdr, sizeof iphdr);
    printf("IPv4 header cksum = 0x%04X\n", cks);

    /* Verify: store the checksum back, recompute, expect 0x0000. */
    iphdr[10] = (uint8_t)(cks >> 8);
    iphdr[11] = (uint8_t)(cks & 0xFF);
    printf("Re-check (want 0) = 0x%04X\n", inet_checksum(iphdr, sizeof iphdr));
    return 0;
}

What it does. It computes the CRC-8 of the canonical test string, computes the Internet checksum over a real IPv4 header (with the checksum field zeroed, as the algorithm requires), then stores that checksum back into the header and recomputes — the hallmark verification step.

Expected output.

CRC-8(123456789)  = 0xF4
IPv4 header cksum = 0xB861
Re-check (want 0) = 0x0000

Edge cases. Odd-length buffers hit the if (i < len) branch. A zero-length buffer returns ~0 = 0xFFFF for the Internet checksum and 0x00 for CRC-8. The uint32_t accumulator guarantees the fold loop never overflows even for large inputs.

Line by line

Walking the Internet checksum over the 20-byte IPv4 header, then the verification.

1. uint32_t sum = 0; — a 32-bit accumulator. We need the extra room above 16 bits to collect carries before folding.
2. The for loop reads two bytes at a time and forms a big-endian word (buf[i] << 8) | buf[i+1]. For the header above, the first word is (0x45 << 8) | 0x00 = 0x4500, the second 0x003C, and so on. Bytes 10–11 (the checksum field) are 0x0000, contributing nothing — that is exactly why the field must be zeroed first.
3. if (i < len) — the header length (20) is even, so this branch is skipped here. It exists for odd buffers, where the final lone byte becomes the high byte of a padded word.
4. The fold loop while (sum >> 16) repeatedly takes the bits above 16 and adds them into the low 16. The header's words sum to a value with carry; folding produces 0x479E.
5. return (uint16_t)~sum; complements 0x479E to 0xB861 — the printed checksum.

Trace of the key step (illustrative):

6. Verification. We write 0xB861 back into bytes 10–11 and recompute. Now those two bytes add 0xB861 to the running total. Adding a number to its own complement (with end-around carry) yields all-ones, 0xFFFF; the final ~0xFFFF is 0x0000. That single 0x0000 result is how a router cheaply says "header intact."

For CRC-8 on 123456789: the register starts at 0; XOR in '1' (0x31), then run 8 shift/conditional-XOR steps; carry the register into the next byte; after all nine bytes the register holds 0xF4, matching the published test vector.

Common mistakes

1. Reusing a checksum as a security MAC.

/* WRONG: trusting a CRC to prove the message wasn't tampered with */
if (crc32(msg, len) == received_crc) accept(msg); /* attacker recomputed it */

Why it's wrong: a CRC is keyless and public. An attacker who edits msg recomputes received_crc to match. Fix: use a keyed MAC.

/* RIGHT: authenticate with HMAC over a shared secret key */
if (hmac_sha256_verify(key, msg, len, received_tag)) accept(msg);

Recognize it by asking: "could an attacker who controls the data also fix the check value?" If yes, you need a MAC.

2. Forgetting to zero the checksum field before computing.

IP/TCP/UDP compute the checksum over a header whose checksum field is 0. Leaving the old value in place yields garbage. Fix: clear bytes 10–11 (for IPv4) first, compute, then store the result back.

3. Mishandling odd-length buffers.

for (size_t i = 0; i + 1 < len; i += 2) sum += (buf[i] << 8) | buf[i+1];
/* WRONG: the last byte of an odd buffer is silently dropped */

Fix: add the trailing byte as the high byte of a padded word — if (i < len) sum += (uint32_t)buf[i] << 8;.

4. Too-narrow accumulator.

uint16_t sum = 0; /* WRONG: carries are lost on every add */

Fix: use uint32_t and fold the carry afterward.

5. Shift without masking.

c = (c << 1) ^ 0x07; /* c is promoted to int; high bits linger */

Fix: cast back — (uint8_t)((c << 1) ^ 0x07). Catch it by checking against a known test vector; a missing mask usually breaks the very first one.

Debugging tips

Compiler warnings. Build with -Wall -Wextra -Wconversion. -Wconversion flags the implicit narrowing in shift expressions and reminds you to cast — exactly the bug in mistake #5.

Use test vectors first. Before integrating into a protocol, prove the function alone is correct:

• CRC-8 (poly 0x07) of 123456789 must be 0xF4.
• CRC-32 of 123456789 must be 0xCBF43926.
• The IPv4 header in the code example checksums to 0xB861.

If the test vector fails, the bug is in the algorithm, not in your protocol glue.

Common symptoms and what they mean:

Concrete steps when it doesn't work. (1) Print the intermediate sum/register in hex. (2) Run on a 1- or 2-byte input you can compute by hand. (3) Confirm endianness — are you forming big-endian words? (4) Diff against a reference implementation byte-by-byte. (5) Ask: am I feeding the exact bytes the spec covers (right length, right field zeroed)?

Memory safety

This is a non-security lesson, but checksums run over raw buffers, so the usual C hazards apply:

• Bounds. Never read past len. The loop condition i + 1 < len guarantees buf[i+1] is valid; the separate odd-length branch handles the final byte without over-reading. Writing i + 2 <= len instead is also fine, but i + 1 < len avoids any chance of size_t issues.
• Length type. Use size_t for lengths so very large buffers don't overflow a signed int index, which would be undefined behavior.
• Integer overflow / width. Accumulate the Internet checksum in uint32_t; a uint16_t accumulator overflows and silently drops carries. CRC registers must be exactly the CRC width (uint8_t for CRC-8), and you must cast shift results back to that width because C promotes small types to int.
• Initialization. Initialize the accumulator/register before the loop (sum = 0, c = 0 or the spec's init value). An uninitialized register is undefined behavior and produces nondeterministic checksums.
• Const-correctness. Take const uint8_t * so the compiler enforces that you only read the input. This both documents intent and prevents accidental writes into a caller's buffer.
• Robustness over trust. When checksumming attacker-influenced input (e.g., a length field from the wire), validate the length against the actual buffer size before looping — never let a wire-supplied length drive a read past your allocation.

Real-world uses

Where these run in production:

• Networking: every IPv4, TCP, UDP, and ICMP header carries the 16-bit Internet checksum; every Ethernet frame ends in CRC-32 (the FCS). When you write socket code in the next lesson, the kernel computes these for you — but raw-socket and packet-crafting tools compute them by hand.
• File formats: ZIP, gzip, and PNG store CRC-32 per chunk/entry; a corrupt download fails the check before it can crash a decoder.
• Embedded / buses: 1-Wire temperature sensors, CAN bus frames, and USB Power Delivery use CRC-8; XMODEM uses CRC-16.
• Storage: filesystems and SSD controllers checksum blocks to detect silent corruption.

Professional best practices.

Beginner rules: always validate your implementation against a published test vector before shipping; use size_t for lengths and uint*_t for fixed widths; zero the checksum field before computing protocol checksums; name functions for the exact variant (crc8_poly07, not just crc).

Advanced rules: document the full CRC parameter set (polynomial, init, reflect-in, reflect-out, final XOR) in a comment so future maintainers can match a spec; for hot paths, switch the bitwise CRC loop to a precomputed 256-entry table; benchmark before optimizing, since the Internet checksum is already nearly free; and above all, choose the primitive by threat model — checksum/CRC for accidents, HMAC or a signature for adversaries — and never roll your own crypto.

Practice tasks

Beginner 1 — XOR checksum (BCC). Write uint8_t xor_checksum(const uint8_t *data, size_t n) returning the XOR of all bytes. Print it for the string "HELLO". Concepts: XOR, iteration. Hint: start the accumulator at 0; XOR is its own identity. Constraint: handle n == 0 (return 0).

Beginner 2 — verify, don't just compute. Given a buffer and an expected 8-bit checksum, write a function returning 1 if they match and 0 otherwise. Feed it both a correct and a deliberately corrupted byte to confirm it rejects the bad one. Concepts: compute-then-compare. Hint: reuse Beginner 1.

Intermediate 1 — Internet checksum with odd length. Implement inet_checksum from this lesson and test it on an odd-length buffer (e.g., 5 bytes). Verify your padding of the final byte by also computing the same data with one trailing 0x00 appended — the results must match. Concepts: one's-complement fold, odd-length padding. Example: {0x01,0x02,0x03,0x04,0x05}.

Intermediate 2 — table-driven CRC-8. Precompute a 256-entry CRC-8 table for polynomial 0x07, then rewrite the CRC so each byte is one table lookup plus an XOR. Confirm it still returns 0xF4 for 123456789. Concepts: CRC as division, lookup-table optimization. Hint: table[b] = crc8 of the single byte b with init 0.

Challenge — frame builder and verifier. Define a tiny frame: [ len:1 ][ payload:len ][ crc8:1 ]. Write build_frame() that fills in the length and CRC, and check_frame() that recomputes the CRC over len + payload and reports OK/corrupt. Then write a short test that flips one bit in the payload and confirms check_frame catches it. Stretch: explain in a comment why this frame still offers no protection against a deliberate attacker, and what you would change to fix that. Concepts: framing, CRC verification, the checksum-vs-MAC distinction. Do not implement HMAC — just describe it.

Summary

• A checksum mixes every byte into a small number to detect accidental corruption; a match means "no detected error," not "data is authentic."
• The Internet checksum is a one's-complement sum of big-endian 16-bit words: add in a uint32_t, fold the carry (while (sum >> 16) sum = (sum & 0xFFFF) + (sum >> 16)), then complement. Zero the checksum field first; pad an odd trailing byte on the right.
• A CRC is XOR-based polynomial division; it is far stronger against burst errors (CRC-32 in Ethernet/ZIP/PNG, CRC-8 in 1-Wire). Always pin and validate the full parameter set against a test vector — 123456789 → 0xF4 for CRC-8.
• Most common mistakes: too-narrow accumulator, dropped odd byte, missing mask after a shift, and not zeroing the checksum field.
• The rule to remember: checksum/CRC for accidents, HMAC or a signature for adversaries. Any keyless check can be recomputed by an attacker. Never invent your own integrity primitive.

Practice with these exercises

• Internet checksum (16-bit ones-complement)
• XOR checksum
• Compute CRC-8 over a byte stream (poly 0x07)
• Compute the IPv4/TCP one's-complement checksum
• Build a 32-bit checksum tool