Write a USTAR tar header for an evidence bundle

Overview

Security objective. The asset you are protecting is the integrity and safe handling of engagement evidence — the logs, screen captures, and JSON findings you pack into a .tar file and move between machines. The threat is a poisoned bundle: a header whose name field contains ../ or a leading /. When such a bundle is later extracted, a naive extractor can write files outside the target directory — this is the classic tar-slip / tarbomb class of bug. In this lesson you build the header writer and learn to reject dangerous names at the source, plus how to spot them in any archive you receive.

A tarball is just files concatenated together, each preceded by a small header. The USTAR ("Unix Standard TAR") layout is old, small, and unambiguous:

• A 512-byte header describes one file (its name, size, mode, and a checksum).
• Then the file's raw bytes, padded up to a multiple of 512.

You will write the header only — one file at a time. The payload is whatever bytes were handed to you.

This builds directly on your prerequisites. From C strings you use strlen, NUL-termination, and the fact that a C string is a byte buffer you must never write past. From byte-layout you use the idea that a fixed-format record places each field at a known offset and width, and that numbers here are stored as text digits, not raw integers. The header is exactly that: a fixed-offset byte record.

Why it matters

In authorized professional work, evidence bundles are how findings travel — from a test host to your workstation, from your workstation to the client, and into long-term storage for the report's chain of custody. Two things make the format worth understanding deeply:

• You produce bundles. If your bundler trusts attacker-influenced filenames (say, names harvested from a target during a lab exercise), it can emit an archive that is itself dangerous to open. A tester who ships a tar-slip bundle to a client has created a vulnerability, not just reported one.
• You audit bundles. You constantly receive archives from tools and teammates. Being able to parse a 512-byte header by hand means you can inspect any .tar for suspicious names, wrong sizes, or a bad checksum before you trust it — instead of blindly tar xf-ing it into your home directory.

Knowing the format at the byte level is also the foundation for reasoning about why higher-level tools have had traversal CVEs, and for verifying that the safe extraction flags your team relies on actually do what you think.

Core concepts

1. The USTAR header is a fixed-offset byte record

Definition. A single 512-byte block where every field lives at a known offset and width. There is no length-prefix and no delimiter parsing — you index into the block.

How it works. You zero the 512 bytes, then write each field at its offset. Anything you do not set stays zero, which USTAR treats as an empty/default value.

Pitfall. Off-by-one on an offset silently corrupts the next field. A size written one byte too far lands in mtime and the archive still "looks" fine until a tool rejects it.

2. Numbers are octal ASCII, not binary

Definition. size, mode, uid, mtime, and the checksum are stored as base-8 digit characters ('0'–'7'), zero-padded, usually NUL-terminated.

Plain explanation. The number 128 is not stored as the byte 0x80. It is stored as the text "0000000200" (128 in octal is 200), padded to the field width.

When / when-not. This is USTAR base format. GNU and pax extensions add binary-encoded large sizes, but you are not writing those here.

Pitfall. Writing the size in decimal ("128") is the single most common bug. Every reader will misinterpret it and read the wrong number of payload bytes.

3. The self-referential checksum

Definition. An integrity check over the whole 512-byte header, computed with the checksum field treated as 8 spaces.

How it works. (a) Fill every other field. (b) Set the 8 checksum bytes to ' ' (0x20). (c) Sum all 512 unsigned bytes. (d) Overwrite the checksum field with the result as 6 octal digits, a NUL, then a space.

Why the spaces. The checksum cannot depend on itself, so during the sum the field is a fixed, agreed-upon value: eight spaces (8 * 32 = 256 added).

Pitfall. Summing bytes as signed char makes any byte >= 0x80 negative and corrupts the total. Always sum as unsigned char.

4. The name field is untrusted input (the security core)

Definition. The 100-byte name field is the file's path as it will be created on extraction. If it contains .. components or a leading /, an extractor may write outside the intended directory.

Plain explanation. tar historically trusted the name and recreated the path verbatim. A name like ../../etc/cron.d/x escapes the extraction directory — a tar-slip (a.k.a. Zip-Slip for zips). The insecure assumption is: "a filename in an archive is just a name." It is actually attacker-controllable path input.

When / when-not. Validate names on both ends: reject them when you write the header, and again when you extract. Defense in depth — never rely on the producer having been careful.

Pitfall (false positive). A blanket "reject any name containing .." also rejects the legitimate name report..old.json. The precise rule splits the path on / and rejects a component that equals exactly .., plus any leading /.

THREAT MODEL — evidence bundle in transit

  ASSET: safe handling + integrity of engagement evidence
         (a poisoned bundle can overwrite files on the analyst's box)

  [ Lab target / source ]        TRUST BOUNDARY        [ Analyst workstation ]
   files + FILENAMES  ------ .tar bundle (transfer) ------>  tar extractor
   (names may be attacker-                |                       |
    influenced in a lab)                  |                       v
                                   ENTRY POINT:              writes files to disk
                                   the 100-byte name         using the name field
                                   field in each header      ( "../" escapes cwd! )

  INSECURE ASSUMPTION: "a filename in an archive is just a name"
  DEFENSE: validate name on write AND on extract; verify checksum before trust

Knowledge check.

1. What asset is protected here, and what does a poisoned bundle threaten on the receiving machine?
2. Where is the trust boundary, and which single header field is the entry point for a tar-slip?
3. Which insecure assumption makes tar-slip possible, and why do we still validate on extraction even after validating on write?
4. Why should experiments with ../ names run only in an isolated, authorized lab (localhost / a container / a throwaway VM)?

Syntax notes

The header is a uint8_t[512] buffer. Three building blocks:

#include <stdint.h>   /* uint8_t          */
#include <string.h>   /* memset, memcpy, strlen, strstr */
#include <stdio.h>    /* snprintf         */

uint8_t out[512];
memset(out, 0, 512);                 /* start from all-zero: unset fields stay 0 */

/* octal ASCII into a fixed-width field: N-1 digits + a NUL terminator */
snprintf((char *)out + 124, 12, "%011lo", (unsigned long)size);   /* size field */
/*                        ^offset ^width  ^zero-padded 11 octal digits           */

/* raw literal bytes at an offset */
memcpy(out + 257, "ustar", 5);       /* magic; out[262] stays NUL from memset    */

Field map (offset, width, meaning):

Everything from offset 265 to 511 (uname, gname, dev fields, prefix) can stay zero for a simple regular file.

Lesson

Why this matters

Engagement deliverables travel as bundles: logs, screenshots, the report, and the JSON of findings, all packed into a single .tar file.

A tarball is just files concatenated together with a small header in front of each one. The USTAR format (the "Unix Standard TAR" layout) is small, old, and unambiguous:

• A 512-byte header for each file.
• Then the file's bytes, padded up to a multiple of 512.

In this lesson you write the header only. The payload is whatever bytes were passed in.

The header layout (USTAR)

Every field sits at a fixed offset inside the 512-byte block:

offset  size  field
0       100   name           (NUL-padded ASCII)
100     8     mode           (octal ASCII + NUL)
108     8     uid            (octal ASCII + NUL)
116     8     gid            (octal ASCII + NUL)
124     12    size           (octal ASCII + NUL)
136     12    mtime          (octal ASCII + NUL)
148     8     checksum       (6 octal digits + NUL + space)
156     1     typeflag       ('0' for regular file)
157     100   linkname       (NUL)
257     6     magic          ("ustar\0")
263     2     version        ("00")
...     ...   (rest zero)

Two things to notice:

• Numbers are stored as octal ASCII (base-8 text), not raw binary.
• The checksum is computed over the whole 512-byte header, but with the checksum field itself treated as 8 spaces during the sum.

Your job

Implement:

int write_ustar_header(const char *name, size_t size, uint8_t out[512]);

Fill the 512-byte buffer with a valid header.

Return 0 on success, or -1 on any of these:

• name is NULL, or out is NULL.
• strlen(name) >= 100 (the name field holds 100 bytes).
• size does not fit in 11 octal digits. That is the size field's capacity (12 bytes) minus the trailing NUL.

Common mistakes

• Using the real checksum during computation. The checksum field must be 8 spaces while you add up the bytes. You overwrite it with the result afterward.
• Writing size in decimal. It is octal.
• Dropping the NUL after the checksum. The format is 6 octal digits, then a NUL, then a space.

What this is NOT

• It is not a multi-file tarball builder. You handle one header at a time.
• It is not a pax or GNU extended header. This is base USTAR only.

Code examples

The security shape is INSECURE -> SECURE -> VERIFY. Read the insecure version to understand the bug, then use the secure one.

1. Insecure writer (study only)

/* WARNING: intentionally vulnerable — use only in a local, isolated, authorized lab. Do not deploy. */
#include <stdint.h>
#include <string.h>
#include <stdio.h>

int write_header_insecure(const char *name, size_t size, uint8_t out[512]) {
    memset(out, 0, 512);
    strcpy((char *)out, name);                              /* (1) no bound: overruns the 100-byte name field */
    sprintf((char *)out + 124, "%lo", (unsigned long)size); /* (2) no width: can run past the size field       */
    out[156] = '0';
    memcpy(out + 257, "ustar", 6);
    memcpy(out + 263, "00", 2);
    return 0;                                               /* (3) no name validation -> "../x" escapes on extract */
}

Three defects: (1) strcpy overflows the buffer if name is >= 100 bytes; (2) a size that needs 12+ octal digits corrupts mtime; (3) it will happily encode ../../etc/cron.d/x, producing a tar-slip bundle.

2. Secure writer

#include <stdint.h>
#include <string.h>
#include <stdio.h>

/* Returns 0 on success, -1 on any invalid/unsafe input. Fills out[512]. */
int write_ustar_header(const char *name, size_t size, uint8_t out[512]) {
    if (name == NULL || out == NULL) return -1;

    size_t nlen = strlen(name);
    if (nlen == 0 || nlen >= 100) return -1;        /* name field is 100 bytes */

    /* Reject dangerous names at the source (defence against tar-slip). */
    if (name[0] == '/') return -1;                  /* no absolute paths */
    for (const char *p = name; *p; ) {              /* reject any ".." *component* */
        const char *slash = strchr(p, '/');
        size_t seg = slash ? (size_t)(slash - p) : strlen(p);
        if (seg == 2 && p[0] == '.' && p[1] == '.') return -1;
        if (!slash) break;
        p = slash + 1;
    }

    /* size must fit in 11 octal digits (12-byte field minus the NUL). */
    if ((unsigned long long)size > 077777777777ULL) return -1;   /* 8^11 - 1 */

    memset(out, 0, 512);
    memcpy(out, name, nlen);                        /* bounded: nlen < 100 */

    /* numeric fields: zero-padded octal ASCII + NUL */
    snprintf((char *)out + 100, 8,  "%07o",  0644);                 /* mode  */
    snprintf((char *)out + 108, 8,  "%07o",  0);                    /* uid   */
    snprintf((char *)out + 116, 8,  "%07o",  0);                    /* gid   */
    snprintf((char *)out + 124, 12, "%011lo", (unsigned long)size); /* size  */
    snprintf((char *)out + 136, 12, "%011lo", 0UL);                 /* mtime */

    out[156] = '0';                                 /* typeflag: regular file */
    memcpy(out + 257, "ustar", 5);                  /* magic "ustar\0" (byte 262 already 0) */
    memcpy(out + 263, "00", 2);                     /* version */

    /* checksum: field = 8 spaces during the sum, then written back */
    memset(out + 148, ' ', 8);
    unsigned sum = 0;
    for (int i = 0; i < 512; i++) sum += out[i];    /* out is uint8_t -> unsigned bytes */
    snprintf((char *)out + 148, 7, "%06o", sum);    /* 6 octal digits + NUL at [154] */
    out[155] = ' ';                                 /* trailing space */

    return 0;
}

3. Verify — the fix rejects bad input and accepts good input

#include <stdlib.h>

/* Recompute and compare the stored checksum (used when auditing any header). */
static int checksum_ok(const uint8_t h[512]) {
    unsigned stored = (unsigned)strtoul((const char *)h + 148, NULL, 8);
    uint8_t tmp[512];
    memcpy(tmp, h, 512);
    memset(tmp + 148, ' ', 8);
    unsigned sum = 0;
    for (int i = 0; i < 512; i++) sum += tmp[i];
    return sum == stored;
}

int main(void) {
    uint8_t h[512];

    /* ACCEPT a clean relative name */
    if (write_ustar_header("findings/report.json", 128, h) != 0) {
        fprintf(stderr, "FAIL: rejected a good name\n"); return 1;
    }
    if (!checksum_ok(h))                    { fprintf(stderr, "FAIL: checksum\n"); return 1; }
    if (memcmp(h + 257, "ustar", 5) != 0)  { fprintf(stderr, "FAIL: magic\n");    return 1; }
    printf("accept clean name: OK (checksum verified)\n");

    /* REJECT the dangerous inputs */
    if (write_ustar_header("../../etc/cron.d/x", 10, h) == 0) {
        fprintf(stderr, "FAIL: accepted traversal name\n"); return 1;
    }
    if (write_ustar_header("/etc/passwd", 10, h) == 0) {
        fprintf(stderr, "FAIL: accepted absolute path\n"); return 1;
    }
    char big[200]; memset(big, 'a', sizeof big); big[199] = '\0';
    if (write_ustar_header(big, 10, h) == 0) {
        fprintf(stderr, "FAIL: accepted >=100 char name\n"); return 1;
    }
    printf("reject traversal / absolute / oversize: OK\n");
    printf("all checks passed\n");
    return 0;
}

Build and run (lab, localhost only):

$ cc -std=c11 -Wall -Wextra -fsanitize=address,undefined header.c -o header
$ ./header
accept clean name: OK (checksum verified)
reject traversal / absolute / oversize: OK
all checks passed

The sanitizers (-fsanitize=address,undefined) will crash loudly if any write strays outside the 512-byte buffer — run the insecure version under them with a 200-byte name to see the overflow reported.

Line by line

Walking the secure writer for the call write_ustar_header("findings/report.json", 128, h):

1. NULL guards — name and out are non-NULL, so we continue. A NULL here would be an immediate -1 instead of a crash.
2. Length check — nlen = 20. It is > 0 and < 100, so the name fits the field. (A 100+ byte name returns -1, which is what stops the insecure strcpy overflow.)
3. Absolute-path check — name[0] is 'f', not '/'. Pass.
4. Component scan — the loop splits on /: segments are findings (8) and report.json (11). Neither equals exactly .., so no traversal. Note report..json would also pass — only a whole .. component is rejected, avoiding the false positive.
5. Size range — 128 <= 8^11 - 1, pass.
6. Zero the block — all 512 bytes become 0; every field we do not touch stays 0.
7. Copy the name — memcpy(out, "findings/report.json", 20). Bounded by step 2, so it can never overrun.
8. Numeric fields — snprintf writes octal text. size 128 -> octal 200 -> "00000000200" at offset 124.
9. typeflag / magic / version — out[156]='0', "ustar" at 257, "00" at 263.
10. Checksum — set bytes 148–155 to spaces, sum all 512 unsigned bytes, then overwrite with "%06o" + a trailing space.

Trace of the size field:

Trace of the checksum field (illustrative values):

Common mistakes

On misconceptions: producing a header with a valid checksum proves only that the bytes are self-consistent — it does not prove the file inside is authentic or that the name is safe. Integrity of the bundle (e.g. a detached SHA-256 you record separately) and authenticity (a signature) are different guarantees.

Debugging tips

tar tf bundle.tar says "invalid header" or "checksum error".

• Recompute the checksum by hand with the field set to 8 spaces. If your value differs, you either summed the real field or summed signed bytes.
• Confirm the magic is exactly "ustar\0" at offset 257 and version "00" at 263. A wrong magic makes GNU tar treat it as an old-format archive.

The extracted file is truncated or has trailing garbage.

• The size field is almost certainly decimal instead of octal, or written at the wrong offset. Dump the header: xxd -l 512 bundle.tar and check bytes 124–135.

Sanitizer aborts with a heap/stack overflow.

• A write ran past offset 511 or past the name field. Check every snprintf width and that nlen < 100 is enforced before the memcpy.

A name you expected to reject was accepted (or vice-versa).

• Print the exact bytes of the name. A name like foo/../bar must be rejected (it has a .. component); foo..bar must be accepted. If both behave the same, your check is a substring match, not a component match.

Questions to ask when it fails:

• Is this field octal or binary? At what offset and width?
• Am I summing bytes as unsigned?
• Did I validate the name before copying it?
• Does a known-good tool (tar tvf) agree with my header?

Memory safety

Memory safety (C). Every write into out must be provably in-bounds. The one dangerous line is the name copy: guard it with strlen(name) < 100 and use memcpy(out, name, nlen) — never strcpy. Prefer snprintf (bounded) over sprintf (unbounded) for every numeric field. Sum checksum bytes through uint8_t/unsigned char so no byte is sign-extended. Build with -Wall -Wextra -fsanitize=address,undefined during development; a single stray write will be caught immediately rather than silently corrupting the next field.

Security & safety — detection and logging. Bundling and extraction are chain-of-custody events. Log them.

• Log (append-only): timestamp (UTC), operator/host identity, the engagement/case id (correlation id), the bundle filename, each entry's name and declared size, the computed SHA-256 of the whole bundle, and the security decision for each entry (accepted / rejected: traversal / rejected: absolute / rejected: oversize). On extraction, log the resolved output path and whether it stayed inside the target directory.
• Never log: the evidence contents, credentials or tokens captured during the engagement, session cookies, private keys, or full PANs/PII. Log the file's hash and size, not its bytes.
• Signals of abuse: any entry rejected for .. or a leading /; an entry whose declared size disagrees with the actual payload; a checksum that fails to recompute; an extraction whose resolved path escapes the target root.
• False positives: legitimate names containing .. as a substring (report..old.json), or deep-but-legal nested paths. Tune the rule to reject only whole .. components and true absolute paths, and keep an accept-list of expected top-level directories.

Real-world uses

Authorized use case. At the end of an engagement you assemble a deliverable bundle: report.pdf, findings.json, sanitised logs, and screenshots. A hardened bundler runs write_ustar_header per file, rejecting any name that could escape on extraction, and records a detached bundle.tar.sha256 plus a chain-of-custody log entry. The client can verify the hash before opening, and your extraction script refuses any entry whose resolved path leaves the target directory.

Best-practice habits.

• Validate input — treat every name and size as untrusted; reject rather than sanitise-and-hope.
• Least privilege — extract into a fresh, dedicated directory owned by an unprivileged user, never your home directory or as root.
• Secure defaults — refuse absolute paths and .. components by default; require an explicit opt-in for anything unusual.
• Logging & error handling — return an error and log the decision on any rejection; fail closed.

Only in an authorized lab. Any experiment that crafts a ../ header is done on localhost, a container, or a throwaway VM you own — never against third-party systems, and never shipped to anyone.

Practice tasks

Beginner 1 — Encode one octal field. Objective: write a helper void put_octal(uint8_t *dst, size_t width, unsigned long value) that stores value as zero-padded octal ASCII with a trailing NUL in a width-byte field. Requirements: use snprintf; verify byte-for-byte that put_octal(buf, 12, 128) yields "00000000200". Constraints: no writes past width. Hint: the format is "%0*lo" with width width-1. Concepts: octal ASCII, fixed-width fields.

Beginner 2 — Recompute and check a checksum. Objective: implement int checksum_ok(const uint8_t h[512]) that returns 1 iff the stored checksum matches a fresh computation. Input: a 512-byte header. Output: 1/0. Constraints: sum as unsigned; treat the field as 8 spaces during the sum. Hint: strtoul(h+148, NULL, 8). Concepts: self-referential checksum.

Intermediate 1 — Safe-name validator. Objective: int name_is_safe(const char *name) returning 1 only for names that are non-empty, < 100 bytes, have no leading /, and contain no whole .. component. Requirements: foo/../bar and /etc/passwd reject; findings/report.json and report..old.json accept. Constraints: component-wise check, not substring. Hint: split on /. Concepts: path-traversal defence, false positives. Defensive conclusion: this validator is the remediation — add each accepted/rejected example as a permanent test so a regression is caught immediately.

Intermediate 2 — Header round-trip test. Objective: write a header for logs/scan.txt of size 4096, then parse the size and name back out and assert they match; confirm tar tf (in a lab dir) lists the entry. Input: name + size. Output: parsed name/size. Constraints: run only in a throwaway directory. Hint: xxd -l 512 to eyeball offsets. Concepts: encode/decode symmetry, verification.

Challenge — Poisoned-bundle detector (lab only). Objective: given a .tar file, print each entry's name, declared size, and a verdict (safe / unsafe: traversal / unsafe: absolute / bad checksum) without extracting anything. Authorization checklist: (1) you own or are explicitly authorised to test the machine; (2) work on localhost / a container / a throwaway VM; (3) test archives are ones you crafted locally. Requirements: read 512 bytes at a time, validate name and checksum, skip payload by rounding size up to a 512 multiple. Constraints: never write files; read-only. Hint: build your own malicious test archive with the insecure writer from this lesson so you have a known bad input. Defensive conclusion: the tool is a detector — its output feeds a decision to quarantine and not extract; document how you would remediate (reject the bundle, notify the sender) and verify (re-scan after the sender re-issues a clean bundle). Cleanup/reset: delete the throwaway directory and any crafted test archives when done.

Summary

• A USTAR header is one fixed 512-byte block per file; each field sits at a known offset — index into the block, do not parse delimiters.
• Numeric fields (size, mode, mtime, checksum) are octal ASCII, zero-padded, NUL-terminated — never decimal, never raw binary.
• Checksum: set the field to 8 spaces, sum all 512 bytes as unsigned, then write the result as 6 octal digits + NUL + space.
• Security core: the name field is untrusted path input. Reject a leading / and any whole .. component to prevent tar-slip, on both write and extract. A valid checksum proves consistency, not authenticity or a safe name.
• Common mistakes: strcpy overflow, decimal size, signed-byte checksum, computing the checksum with the real field, and substring .. checks that false-positive report..old.json.
• Remember: validate at the source, verify with a recompute and a known-good tool, log the security decision (not the contents), and experiment with malicious names only in an isolated, authorized lab.

Practice with these exercises

• Total evidence size
• Does the bundle fit?
• Write a USTAR tar header for one file