Base64 and tokens — decode before you trust

Overview

Almost every secret you handle in security work — a session token, a TLS certificate, an API key, a JWT — travels across the network as base64. Base64 is a way of writing arbitrary bytes using only a small set of printable characters (letters, digits, and a couple of symbols) so they survive channels that were built for text, like HTTP headers, URLs, and email.

Here is the key mental shift: base64 is not encryption. It hides nothing. Anyone can decode it in one line. It is just a costume that lets raw bytes travel through text-only pipes. The security question is never "is it base64?" — it is "what do I do with the bytes after I decode them, and can I trust them?"

That decoding step is where bugs are born. You are turning attacker-controlled text into bytes in a fixed buffer, which is exactly the situation the C strings lesson warned you about: if you don't track your buffer's capacity, you overflow it. And decoding base64 means packing bits from four input characters into three output bytes, which leans directly on the shift-and-mask techniques from bitwise operators. This lesson combines both: careful buffer bounds plus bit manipulation, applied to untrusted input.

Once you can decode safely, we look at JWTs (JSON Web Tokens). A JWT is three base64url pieces joined by dots. Before you ever check a token's cryptographic signature, you must confirm it even has the right shape. Skipping that lets malformed or malicious tokens reach code that assumes they are well-formed — a classic source of authentication bugs.

Why it matters

You decode untrusted tokens constantly — every request to an authenticated API carries one. If your decoder trusts its input, a single crafted token can overflow a stack buffer, crash the service, or (worse) let an attacker overwrite memory. And if you verify a token's signature without first checking its structure, you hand malformed input to parsing code that never expected it.

Historically, JWT libraries that skipped these checks enabled real authentication bypasses: the infamous alg:"none" forgery, and confusion between signing algorithms. Getting the boring, defensive parts right — bounds checks and shape validation — eliminates an entire category of these failures before cryptography even enters the picture. In defensive security, the unglamorous discipline of "decode strictly, validate before you trust" is what keeps a token parser from becoming a foothold.

Core concepts

1. What base64 actually does

Definition. Base64 encodes binary data as text using a 64-character alphabet: A–Z, a–z, 0–9, +, and /. Each character represents exactly 6 bits (because 2^6 = 64).

Plain-language explanation. Raw bytes are 8 bits each. Text channels only reliably carry a limited set of printable characters. Base64 re-slices the bit stream into 6-bit chunks and maps each chunk to one safe character. Since 6 and 8 don't divide evenly, base64 works on the least common multiple: 24 bits = 3 bytes = 4 characters.

How it works internally. Take 3 bytes (24 bits), then cut them into four 6-bit groups and look each group up in the alphabet:

Input bytes:   M            a            n
ASCII:         77           97           110
Binary:        01001101     01100001     01101110

Regroup 24 bits into four 6-bit fields:
               010011   010110   000101   101110
Decimal:       19       22       5        46
Alphabet:      T        W        F        u

"Man"  ->  "TWFu"

Decoding runs this in reverse: map each character back to its 6-bit value, concatenate to 24 bits, and split into three bytes.

Padding. When the input isn't a multiple of 3 bytes, base64 pads the final group with = so the output is always a multiple of 4 characters:

When to use / not use. Use base64 to move bytes through text-only transports (headers, URLs, JSON, config files). Do not use it for confidentiality — it is trivially reversible. Do not use it where size matters greatly; it inflates data by ~33%.

Common pitfall. Treating base64 as security. Authorization: Basic ... is base64 of user:password — plaintext to anyone who decodes it. Base64 protects nothing.

Knowledge check. "Hi" is 2 bytes. How many base64 characters (including padding) will it encode to, and how many of those are =?

2. Encoded length and decoded length

Definition. The encoded length is always 4 * ceil(n / 3) characters for n input bytes. The decoded length depends on how many = pad characters trail the string.

Why it matters for safety. Before decoding, you must allocate an output buffer big enough for the bytes you'll produce — but never assume the input is honest about its own size. The maximum decoded size is (len / 4) * 3, adjusted down by the padding count. A safe decoder always checks each byte it's about to write against the buffer capacity, rather than trusting a length formula on untrusted input.

Encoded chars (len)    Max decoded bytes
4                      3
8                      6
"TWFu" (4, no pad)     3   ->  M a n
"TWE=" (4, one =)      2   ->  M a
"TQ==" (4, two =)      1   ->  M

Common pitfall. Sizing the output buffer from strlen(input) without dividing, then overflowing. Or forgetting that the input length must be a multiple of 4 for standard padded base64.

Knowledge check (predict the output). How many bytes does "QUJD" decode to, and what are they? (Hint: A=0, then work out the ASCII.)

3. Bounded, strict decoding

Definition. A bounded decoder is one that (a) validates every input character against the alphabet, (b) rejects padding that appears anywhere but the very end, and (c) refuses to write past the caller's buffer capacity, returning an error instead.

How it works. You process the input four characters at a time. Each character is looked up in a reverse table that maps a character to its 6-bit value (or to a sentinel meaning "invalid"). You assemble the 24-bit group, then write 3, 2, or 1 output bytes depending on padding — but before each write you check out_index < capacity.

When to use / not use. Always use strict decoding for untrusted input (tokens, uploaded files, network data). A permissive "skip unknown characters" decoder is fine only for trusted, well-formed data you generated yourself — and even then, strictness catches corruption early.

Common pitfall. Ignoring the return value. A decoder that returns -1 on bad input is useless if the caller proceeds to use a half-filled buffer as though decoding succeeded.

Strict decode of one 4-char group into a bounded buffer:

  in:  [ c0 ][ c1 ][ c2 ][ c3 ]
         |     |     |     |
     lookup each -> 6-bit value (or reject)
         v     v     v     v
  bits: 000000 000000 000000 000000   (24 bits)
         \_____ regroup into 3 bytes ____/
  out:  [ b0 ][ b1 ][ b2 ]
          ^ check out_index < cap before EACH byte

Knowledge check (find the bug). A decoder computes out[j++] = ... three times per group but only checks j < cap once, before the group. On the last group of a token sized exactly to fill cap, what can go wrong?

4. base64 vs base64url, and JWT shape

Definition. base64url is a URL- and filename-safe variant: it replaces + with -, / with _, and usually drops the = padding. A JWT is header.payload.signature, where each part is base64url of some bytes (JSON for the header and payload, raw signature bytes for the third).

How it works. Splitting on the two dots gives three segments. Structural validation means: exactly three segments, each non-empty, each containing only base64url characters. Only after that shape check do you decode the header, read its alg, and verify the signature.

When to use / not use. Use base64url anywhere the value goes in a URL, filename, or JWT. Don't feed a base64url string to a strict standard base64 decoder — the - and _ will be rejected (or, worse, silently misread).

Common pitfall. Accepting a token with empty segments (header..signature) or with extra dots. strtok and naive splitting can mask these; count segments explicitly.

Knowledge check (explain in your own words). Why must structural validation happen before signature verification, rather than after?

Syntax notes

A reverse-lookup table is the heart of a decoder. Instead of searching the alphabet for each character (slow, and easy to get wrong), you precompute an array indexed by the character's byte value:

// -1 means "not a base64 character". Index by (unsigned char)c.
static const signed char B64[256] = { /* ... filled at init ... */ };

int v = B64[(unsigned char)c];   // 0..63 if valid, -1 if not
if (v < 0) return -1;            // reject immediately

Key points:

• Cast to unsigned char before indexing. A plain char may be signed, and a byte >= 128 would index negatively — undefined behaviour.
• Assemble bits with shifts and masks (from the bitwise operators lesson): acc = (acc << 6) | v;
• Bound every write: if (j >= cap) return -1; before out[j++] = byte;
• For JWT shape, walk the string once, counting dots and checking each character is base64url or a dot.

Lesson

Why this matters

Tokens, certificates, and payloads usually travel as base64 — a text encoding that turns raw bytes into safe, printable characters.

Before you can reason about a JWT or a certificate, you have to decode it. Decoding untrusted input is exactly where bounds bugs creep in.

Base64, briefly

Base64 works in fixed groups:

• Every 4 characters encode 3 bytes.
• = is padding, and appears only at the very end (1 or 2 of them).

A safe decoder rejects:

• bad characters,
• padding in the middle of the data,
• any output that would overflow the caller's buffer.

JWT structure

A JWT (JSON Web Token) is three base64url segments joined by dots:

header.payload.signature

base64url is a URL-safe variant of base64. It swaps +/ for -_ and drops the = padding.

Before verifying a signature, first confirm the shape: exactly three non-empty segments made only of base64url characters.

The alg:"none" trap

A JWT header can claim alg:"none", which means "no signature."

Libraries that honoured this claim let attackers forge tokens. Structural validation is step one. Never accept alg:none for a token you are meant to verify.

Your job (two exercises)

• b64_decode(...) — a bounded base64 decoder.
• jwt_is_wellformed(...) — a structural JWT check (3 base64url segments).

What this is NOT

• A crypto verifier. We check encoding and shape, not signatures.

Code examples

#include <stdio.h> #include <stdint.h> #include <stddef.h> #include <string.h>

/* Map a base64 character to its 6-bit value, or -1 if invalid. / static int b64_value(char c) { if (c >= 'A' && c <= 'Z') return c - 'A'; /* 0..25 */ if (c >= 'a' && c <= 'z') return c - 'a' + 26; /* 26..51 */ if (c >= '0' && c <= '9') return c - '0' + 52; / 52..61 / if (c == '+') return 62; if (c == '/') return 63; return -1; / not base64 */ }

/*

• Strict, bounded standard-base64 decoder.

• Returns the number of decoded bytes, or -1 on any malformed input

• or if the output would exceed cap. */ int b64_decode(const char *in, uint8_t out, size_t cap) { size_t len = strlen(in); if (len % 4 != 0) return -1; / padded base64 is a multiple of 4 */ if (len == 0) return 0;

  size_t j = 0; /* next free slot in out / for (size_t i = 0; i < len; i += 4) { int pad = 0; int v[4]; for (int k = 0; k < 4; k++) { char c = in[i + k]; if (c == '=') { / padding only allowed in the last group, last 1-2 slots / if (i + 4 != len || k < 2) return -1; pad++; v[k] = 0; } else { if (pad) return -1; / real char after padding: reject / int d = b64_value(c); if (d < 0) return -1; / not a base64 character / v[k] = d; } } / Four 6-bit values -> 24-bit group -> up to 3 bytes. / uint32_t group = ((uint32_t)v[0] << 18) | ((uint32_t)v[1] << 12) | ((uint32_t)v[2] << 6) | (uint32_t)v[3]; int out_bytes = 3 - pad; / pad 0->3, 1->2, 2->1 / uint8_t b[3] = { (uint8_t)(group >> 16), (uint8_t)(group >> 8), (uint8_t)(group) }; for (int k = 0; k < out_bytes; k++) { if (j >= cap) return -1; / bound EVERY write */ out[j++] = b[k]; } } return (int)j; }

/* Is c a base64url character (A-Z a-z 0-9 - )? */ static int is_b64url_char(char c) { return (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '-' || c == ''; }

/*

• Structural JWT check: exactly three non-empty base64url segments

• separated by two dots. Returns 1 if well-formed, else 0.

• Does NOT verify the signature — shape only. */ int jwt_is_wellformed(const char *tok) { if (tok == NULL || tok[0] == '\0') return 0;

  int dots = 0; size_t seg_len = 0; /* length of current segment */ for (const char *p = tok; *p; p++) { if (p == '.') { if (seg_len == 0) return 0; / empty segment before this dot / dots++; if (dots > 2) return 0; / too many segments */ seg_len = 0; } else if (is_b64url_char(p)) { seg_len++; } else { return 0; / illegal character / } } return (dots == 2 && seg_len > 0); / 3 non-empty segments */ }

int main(void) { /* --- decode demo --- / uint8_t buf[64]; int n = b64_decode("TWFu", buf, sizeof buf); / "Man" */ if (n < 0) { fprintf(stderr, "decode failed\n"); return 1; } printf("decoded %d bytes: ", n); for (int i = 0; i < n; i++) putchar(buf[i]); putchar('\n');

/* a malformed input must be rejected, not silently accepted */
printf("bad input returns: %d\n", b64_decode("TW!u", buf, sizeof buf));

/* --- JWT shape demo --- */
const char *good = "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxIn0.abc123";
const char *bad  = "only.two";
printf("good token well-formed? %d\n", jwt_is_wellformed(good));
printf("bad token well-formed?  %d\n", jwt_is_wellformed(bad));
return 0;

}

**What it does.** `b64_decode` turns a standard padded base64 string into raw bytes in a caller-supplied buffer, refusing anything malformed or too large. `jwt_is_wellformed` confirms a token has exactly three non-empty base64url segments before any cryptographic work.

**Expected output:**

```text
decoded 3 bytes: Man
bad input returns: -1
good token well-formed? 1
bad token well-formed?  0

Edge cases handled. Zero-length input decodes to 0 bytes. Length not a multiple of 4 is rejected. = appearing anywhere but the final one or two slots is rejected, as is any real character following padding. Every output write is bounds-checked, so a long token can never overflow buf. The JWT check rejects NULL, empty strings, empty segments (a..b), too few or too many dots, and any non-base64url character.

Line by line

1. b64_value maps a single character to its 6-bit value using range checks in alphabet order (A=0 … /=63), returning -1 for anything else. This is the strictness gate for standard base64.
2. b64_decode length checks. strlen gives the input length; padded base64 must be a multiple of 4, so a non-multiple is rejected outright. Empty input trivially decodes to 0 bytes.
3. Outer loop steps i in blocks of 4 — one 24-bit group per iteration. j tracks how many output bytes we've written so far.
4. Inner loop reads the four characters of the group. An = is only legal in the last group (i + 4 == len) and only in the last one or two slots (k >= 2); otherwise reject. A normal character after padding has begun is illegal. Valid characters go through b64_value.
5. Bit assembly. group packs the four 6-bit values into a 24-bit integer: first value shifted left 18, next 12, next 6, last 0. This is the reverse of the encoding diagram in the concepts section.
6. Byte extraction. out_bytes = 3 - pad gives 3, 2, or 1. We extract the top, middle, and low bytes of group, but write only out_bytes of them.
7. The critical guard. Before each out[j++], we check j >= cap. This is what makes the decoder bounded — no amount of input can push past cap.

Trace of b64_decode("TWFu", buf, 64):

Returns 3.

8. jwt_is_wellformed walks the string once. dots counts separators; seg_len counts characters since the last dot. A dot with seg_len == 0 means an empty segment — reject. More than two dots — reject. Any non-base64url, non-dot character — reject. At the end, a valid token has exactly dots == 2 and a non-empty final segment.

Common mistakes

Mistake 1 — bounds check outside the inner write loop.

/* WRONG: checks capacity once, then writes up to 3 bytes */
if (j + 3 > cap) return -1;
out[j++] = b0;
out[j++] = b1;
out[j++] = b2;

Why it's wrong: on the final group you may only write 1–2 bytes, so j + 3 > cap can reject valid input that actually fits — or, written slightly differently, a subtle off-by-one lets a write slip past. Fix: check j >= cap immediately before each individual out[j++], as in the lesson code. Recognise it by testing a token sized to exactly fill the buffer.

Mistake 2 — indexing a lookup table with a signed char.

char c = in[i];
int v = table[c];        /* WRONG if c >= 128: negative index -> UB */

Fix: int v = table[(unsigned char)c];. Prevent it by always casting to unsigned char before using a character as an array index. This bug hides until someone feeds you a byte with the high bit set.

Mistake 3 — accepting misplaced padding.

/* WRONG: treats '=' as just another 0-valued character anywhere */
if (c == '=') { v = 0; continue; }

Why it's wrong: "AB=C" or "A===" should be rejected, but this accepts them, so two different strings decode to the same bytes (a malleability bug that can defeat signature checks that hash the encoded form). Fix: only allow = in the last group's last one or two slots, and reject any real character after padding begins.

Mistake 4 — splitting a JWT with strtok and not counting segments.

char *h = strtok(tok, ".");   /* WRONG: mutates the token, */
char *p = strtok(NULL, ".");  /* collapses "a..b" and ignores extra dots */

strtok skips empty fields, so "a..b" looks like two segments and "a.b.c.d" silently drops the fourth. Fix: scan once and count dots and segment lengths explicitly, as jwt_is_wellformed does. Also, strtok modifies its input — never do that to a token you don't own.

Debugging tips

Compiler errors and warnings.

• Build with -Wall -Wextra. A warning like "array subscript has type char" is the signed-index bug — fix the cast.
• "comparison is always true/false" on a char range check often means your platform's char is unsigned; use explicit ranges as shown rather than assuming signedness.

Runtime errors.

• A crash or AddressSanitizer report (-fsanitize=address) at out[j] means your bound check is wrong or missing. Rebuild with ASan and feed it an oversized input to confirm the decoder returns -1 instead of writing past cap.
• Garbage in the decoded output usually means the bit-assembly shifts are off (e.g. << 16 instead of << 18). Decode a known value like "TWFu" and check you get Man.

Logic errors.

• If valid tokens are rejected, print len % 4 and the offending character. base64url input (-,_) fed to the standard decoder is a common cause.
• If jwt_is_wellformed accepts bad tokens, log dots and seg_len at each step.

Questions to ask when it doesn't work.

1. Did I check the return value before using the buffer?
2. Is every array index cast to unsigned char?
3. Is there a bound check before every write, not just once per group?
4. Am I decoding standard base64 or base64url — and does my alphabet match?
5. Does an empty or all-padding input behave sensibly?

Memory safety

This topic sits directly on top of C's memory-safety hazards, because you are writing attacker-controlled data into a fixed buffer.

• Bounds. The single most important rule: check out_index < cap before every byte you write. The lesson's b64_decode does this inside the innermost loop. A decoder that trusts an input-derived length is one crafted token away from a stack or heap overflow.
• Signed-index UB. Indexing a table with a signed char that holds a byte >= 128 is undefined behaviour. Always cast to unsigned char.
• Initialisation. Only read out[0..n-1] where n is the decoder's return value. If it returned -1, the buffer contents are indeterminate — do not use them.
• Integer overflow in sizing. If you compute an output size as len / 4 * 3, be sure len can't overflow when multiplied. For untrusted input, prefer per-write bounds checks over trusting a formula.
• No implicit NUL. b64_decode writes raw bytes, not a C string. It does not append a '\0'. If the caller wants to treat the result as text, they must reserve room and terminate it themselves.

Security-specific, defensive practices.

• Structure before signature. jwt_is_wellformed is a shape gate. It is deliberately not a verifier — but running it first means your signature-verification and JSON-parsing code never sees garbage.
• The alg:"none" vulnerability (labelled). A JWT header can claim {"alg":"none"}, meaning "unsigned." A verifier that honours this accepts a token with an empty signature segment — an attacker forges any payload. Fix: never accept none for tokens you must verify; pin the expected algorithm explicitly and reject mismatches. Do not let the token choose its own verification algorithm.
• Malleability. Sloppy padding handling lets two different encodings decode to the same bytes. If any layer hashes or signs the encoded form, that difference matters — reject non-canonical padding.
• Least trust. Decode, validate shape, then validate signature, then finally trust the claims — in that order, never skipping a step. All examples here are lab-only; never test against tokens or systems you don't own.

Real-world uses

Concrete uses. Base64 and JWTs are everywhere: Authorization: Bearer <jwt> headers on virtually every REST/GraphQL API; OAuth 2.0 and OpenID Connect access/ID tokens; TLS certificates in PEM files (base64 between -----BEGIN CERTIFICATE----- markers); data: URLs embedding images; email attachments (MIME); Kubernetes Secrets; and Basic-auth credentials. A pentester decoding a captured token, or a backend engineer parsing one, runs exactly this code path.

Beginner best practices.

• Always check the decoder's return value before touching the buffer.
• Cast to unsigned char before indexing.
• Name buffers and lengths clearly (out, cap, j); comment only the non-obvious bit math.
• Validate a token's shape before decoding its contents.

Advanced best practices.

• Prefer a vetted library (OpenSSL's EVP base64, or a maintained JWT library) over hand-rolled code in production; write your own only to learn or where dependencies are impossible.
• Pin the accepted alg set and reject everything else, including none.
• Use constant-time comparison for signature bytes (see the previous lesson, constant-time compare) to avoid timing side channels.
• Enforce a maximum token length before decoding to bound work and memory.
• Treat decode and validation as separate, independently testable functions, and fuzz the decoder with malformed input.

Practice tasks

Beginner 1 — Encoded length. Write int b64_encoded_len(int n) returning the number of characters (including = padding) that n input bytes encode to. Objective: cement the 3-bytes-to-4-chars rule. Example: b64_encoded_len(1) -> 4, b64_encoded_len(3) -> 4, b64_encoded_len(4) -> 8. Constraint: assume n >= 0. Hint: it is 4 * ((n + 2) / 3). Concepts: encoded length.

Beginner 2 — Character classifier. Write int is_base64_char(char c) returning 1 if c is in the standard base64 alphabet (A–Z a–z 0–9 + /), else 0. Do not count =. Objective: the per-character guard. Example: is_base64_char('+') -> 1, is_base64_char('-') -> 0. Hint: range checks like the lesson's b64_value. Concepts: strict validation.

Intermediate 1 — Padding counter. Write int b64_pad_count(const char *s) that returns the number of trailing = (0, 1, or 2) in a valid standard base64 string, or -1 if the string's length isn't a multiple of 4 or padding is misplaced (a = before the last two positions). Example: "TWE=" -> 1, "TQ==" -> 2, "AB=C" -> -1. Hint: check length first, then inspect only the last two characters. Concepts: padding rules, malleability.

Intermediate 2 — Bounded decoder with terminator. Write int b64_decode_str(const char *in, char *out, size_t cap) that decodes standard base64 and, if there is room, appends a '\0' so out is a usable C string; return the byte count (excluding the terminator) or -1 if the terminator wouldn't fit. Objective: safe text handling. Constraint: never write past cap. Hint: reserve one spare slot for '\0'. Concepts: bounds, NUL termination.

Challenge — Structural + alg guard. Write int jwt_check(const char *tok, const char *expected_alg) that (1) confirms the token is well-formed (three base64url segments), (2) base64url-decodes only the header, (3) checks the decoded JSON contains "alg":"<expected_alg>" and explicitly rejects "alg":"none". Return 1 only if all pass, else 0. Do not verify the signature — this is the pre-crypto gate. Requirements: convert base64url to standard base64 (map -->+, _->/, re-pad to a multiple of 4) before decoding the header; bound every buffer. Hint: a tiny substring search on the decoded header is enough — you are not writing a full JSON parser. Concepts: base64url conversion, bounded decode, alg:none defence. Lab-only; use tokens you generate yourself.

Summary

• Base64 maps every 4 characters to 3 bytes using a 64-character, 6-bits-each alphabet; it is an encoding, never encryption.
• Padding (=) fills the last group and appears only at the end — one = means 2 real bytes, two means 1. Encoded length is 4 * ceil(n/3).
• Decode strictly and bounded: reject bad characters and misplaced padding, and check out_index < cap before every write. Always cast characters to unsigned char before indexing a table.
• base64url swaps +/ for -_ and drops padding; JWTs use it. A JWT is three non-empty base64url segments: header.payload.signature.
• Validate shape before signature, and never accept alg:"none" for a token you must verify.
• Most common mistakes: using the buffer without checking the return value, signed-char indexing, one bound check per group instead of per write, and strtok-based JWT splitting that hides empty or extra segments.
• Remember: decode -> validate shape -> verify signature -> trust the claims, in that order, skipping nothing.

Practice with these exercises

• Base64 encoded length
• Valid base64 character?
• Bounded base64 decoder
• Validate JWT structure