Hashing: integrity, identification, and its limits

Overview

Security objective. The asset this lesson protects is integrity and identity of data — the assurance that a file, message, or stored credential is exactly what it claims to be and has not been swapped or altered. The threats are corruption in transit, naive tampering by an attacker, and the accidental leaking of secrets through weak storage. By the end you will be able to detect a tampered file, spot a misused hash in a design, and reason about what a hash does and does not guarantee.

Imagine you download a 2 GB operating-system image and want to be sure not a single byte was corrupted in transit or swapped out by an attacker. You cannot eyeball 2 GB. Instead the publisher gives you a short string like e3b0c44298fc1c14... and you compute the same string from your copy. If they match, your file is intact. That short string is a cryptographic hash, and this lesson is about what it is, what it guarantees, and — just as important — what it does NOT guarantee.

A cryptographic hash is a function that takes any input — a password, a file, a whole disk image — and produces a fixed-size output called a digest (also called a hash, fingerprint, or checksum-with-stronger-guarantees). SHA-256, for example, always outputs 256 bits (32 bytes, usually shown as 64 hex characters) no matter whether the input was one letter or one gigabyte.

Hashing is one of the most widely used primitives in all of security and software. It underpins file-integrity checks, digital signatures, version-control systems like Git, blockchains, deduplication, and password storage. Because it is everywhere, misusing it is also everywhere — and that is exactly why this is a security lesson, not just a math lesson.

Connection to prerequisites. This lesson has no formal prereqs — it is the first lesson in the cryptography track and assumes no prior crypto knowledge. It does lean on one everyday idea you already have: the ability to compare two strings for exact equality. That single operation, applied to digests, is what turns hashing into a verification tool. After this you move on to Symmetric and asymmetric encryption, where you will see the crucial difference between hashing (no key, no way back) and encryption (has a key, designed to be reversible). Keeping those two ideas separate is the single most important takeaway here. As you read, watch for the recurring theme: a hash hides nothing on its own — it only commits you to a value.

Why it matters

Hashing shows up in nearly every security review, and so does its misuse. Understanding it well separates engineers who write safe systems from those who quietly ship vulnerabilities.

Where correct hashing protects real software:

• Integrity of downloads and updates. Package managers (apt, npm, pip), OS installers, and auto-updaters verify a hash before trusting a file. A wrong hash means corruption or tampering, and a fail-closed check refuses to install.
• Digital signatures. You do not sign a whole multi-megabyte document; you sign its hash. The security of the signature then depends on the hash being collision-resistant.
• Version control and content addressing. Git names every commit and file by its hash. Two files with the same hash are treated as the same content.
• Password storage. Done right (with a slow, salted hash), a database breach does not immediately hand attackers every plaintext password.

Where misuse causes breaches — the patterns reviewers flag again and again:

• "We hashed it, so it is encrypted." Hashing is not encryption. There is no key and no decryption. This sentence in a design doc is a red flag.
• Hashing guessable data for secrecy. Hashing a 4-digit PIN, a phone number, or a national ID and storing the digest does not hide it: the input space is tiny, so an attacker hashes every candidate and matches instantly.
• Using broken algorithms. MD5 and SHA-1 have practical collision attacks. Relying on them for signatures or certificates has caused real-world forged-certificate and supply-chain incidents.

Knowing how to recognize a hash by its format also drives defensive testing: when you audit a system and find a 32-hex-character field where passwords should be, you have likely found unsalted MD5 — a finding worth reporting, with a concrete remediation.

Core concepts

This section teaches each major idea separately. Read each definition, then the plain-language explanation, then the pitfall.

1. What a hash function actually is

Definition. A cryptographic hash function H takes a byte string of any length and returns a fixed-size digest: digest = H(input).

Plain language. Think of it as a meat grinder for data: you can put anything in and get a uniform output, but you can never reassemble the original from what comes out.

How it works internally (high level). The function reads the input in fixed-size blocks, mixing each block into an internal state through many rounds of bit operations (XOR, rotation, addition, substitution). Each round scrambles the state so thoroughly that the final state has no visible relationship to the input. You do not need the internal math to use a hash correctly — but you do need to trust that this scrambling is what makes the properties below hold.

  "hello"  --> [ H: read blocks, mix state over many rounds ] --> 2cf24dba5fb0a30e... (32 bytes)
  big.iso  --> [ same H, more blocks, same fixed output size ] --> 9f86d081884c7d65... (32 bytes)

Knowledge check: If SHA-256 outputs 256 bits regardless of input size, can it be reversed to recover the original input in general? Explain in your own words why or why not.

2. The four core properties

Pitfall. "One-way" is about infeasibility, not magic. If the set of possible inputs is small (a PIN, a yes/no answer), one-wayness gives you almost nothing, because an attacker can try them all. One-wayness protects large, unpredictable inputs.

3. The avalanche effect

Definition. Changing a single bit of the input flips, on average, about half of the output bits.

Plain language. There is no "close" with hashes. cat and bat have completely unrelated digests. This is exactly what makes a hash a good tamper detector: any change, however tiny, produces an obviously different digest.

H("The quick brown fox")  = d7a8fbb307d7809469ca9abcb0082e4f...
H("The quick brown f0x")  = 47e3e7f5a3...   <- one char changed, totally different digest

Knowledge check (predict the behavior): If you change the last byte of a 2 GB file, how much of its SHA-256 digest do you expect to change — a little or a lot? Why?

4. Collisions and why algorithm choice matters

Definition. A collision is two distinct inputs a != b with H(a) == H(b).

Plain language. Because outputs are fixed-size and inputs are unlimited, collisions must mathematically exist (pigeonhole principle). A good hash makes them infeasible to find. A broken hash is one where someone discovered a shortcut to find them.

• MD5 (128-bit) and SHA-1 (160-bit): collisions are now practical to construct. Do not use them where collisions matter (signatures, certificates, integrity against an active attacker).
• SHA-256 / SHA-512 (the SHA-2 family) and SHA-3: currently recommended for general-purpose hashing.

When to use / when NOT to use. Use a general fast hash (SHA-256) for integrity and identification. Do NOT use a fast hash for passwords — that needs a deliberately slow function (bcrypt, scrypt, Argon2), covered in the password lessons.

Pitfall. Picking an algorithm by speed. For passwords, fast is bad; for file integrity, fast is fine. The right choice depends on the threat, not on benchmarks.

5. Recognizing a hash by its format

Defenders and crackers both identify a hash by its length and character set:

32 hex chars  -> likely MD5        e.g. 5f4dcc3b5aa765d61d8327deb882cf99
40 hex chars  -> likely SHA-1
64 hex chars  -> likely SHA-256
$2b$...       -> bcrypt (note the $ structure and embedded salt/cost)
$argon2id$... -> Argon2 (embedded params, salt, and cost)

Knowledge check (find-the-bug): A teammate stores user passwords and shows you a column full of 32-character hex strings, identical for any two users who chose the same password. Name two distinct problems with this scheme, and say which log would let you notice bulk cracking after a breach.

6. Threat model — where a hash sits in a system

Hashing does not defend a system by itself; it defends a specific asset across a specific trust boundary. Here is the mental model for the two commonest uses in this lesson.

  ================= INTEGRITY OF A DOWNLOAD =================
  Asset:           the file you are about to trust and run
  Entry point:     the network path from publisher -> you (mirror, CDN, proxy)
  Trust boundary:  you do NOT trust the transport; you trust the publisher's
                   ANNOUNCED digest (ideally signed)
  What a hash gives: detection of accidental corruption and naive tampering
  What it does NOT:  protection if the attacker can rewrite BOTH the file and
                     the published digest  -> that needs a SIGNATURE

  ================= STORAGE OF A PASSWORD ===================
  Asset:           user passwords (and the accounts they protect)
  Entry point:     a database dump from SQL injection or a stolen backup
  Trust boundary:  the app trusts its DB; an attacker who READS the DB must
                   not thereby learn plaintext passwords
  Defense goal:    even with the full hash table, recovery must be SLOW and
                   PER-ACCOUNT (unique salt + slow KDF), never bulk/instant

Knowledge check (trust boundary): In the download case, which insecure assumption breaks the scheme — trusting the transport, or trusting the announced digest? Explain why a bare hash without a signature is not enough against an active attacker.

Syntax notes

Hashing is a concept rather than one language's syntax, but every platform exposes it the same way: feed bytes in, read a fixed digest out. Here is the canonical command-line form on Linux/macOS, annotated:

# Compute the SHA-256 digest of a file.
#  sha256sum      -> the tool (use `shasum -a 256` on macOS)
#  ubuntu.iso     -> the input file (read as raw bytes, not as text)
# Output: "<64-hex-digest>  ubuntu.iso"
sha256sum ubuntu.iso

# Hash a short string. `printf` avoids the trailing newline that `echo` adds,
# which would change the digest.
printf '%s' 'hello' | sha256sum

# Compare against a published checksum file automatically (fail-closed):
#   the file lists "<digest>  <filename>" lines; -c checks them.
sha256sum -c SHA256SUMS

Key structural points:

• The digest is raw bytes; it is displayed as hexadecimal (two hex chars per byte). 32 bytes -> 64 hex chars.
• The same input always yields the same hex string (deterministic).
• A trailing newline is part of the input. echo hello and printf '%s' hello hash to different values — a classic source of "my hashes don't match" confusion.
• sha256sum -c exits non-zero if any file fails, which is what lets a build or install step refuse a bad file automatically.

Lesson

A cryptographic hash maps any input to a fixed-size digest (the output). For example, SHA-256 always produces 256 bits, no matter how large the input is.

It is one-way and is foundational to modern security.

Properties

• Deterministic: the same input always gives the same digest.
• One-way (preimage resistance): it is infeasible to recover the original input from the digest.
• Collision resistance: it is infeasible to find two different inputs that produce the same digest.
• Avalanche effect: changing a single bit of the input flips about half of the output bits.

What hashing is for

• Integrity: compare digests to detect tampering or corruption. This is common for downloads and files.
• Identification and deduplication: content-addressing and fingerprints.
• Password verification: but only with the right, deliberately slow hash. See the password lessons.

What hashing is NOT

• Not encryption. There is no key, and there is no way back — that is by design. Saying "we hashed it, so it is encrypted" is wrong.
• Not confidentiality. If you hash a small or guessable input — like a phone number or a PIN — an attacker can simply hash every candidate until one matches.

Algorithm hygiene

MD5 and SHA-1 are broken for collision resistance. Never use them where collisions matter, such as for signatures or certificates.

Use SHA-256 or SHA-3 for general hashing.

Password hashing is a separate case: it needs functions that are deliberately slow (covered in a separate lesson).

Finally, recognizing a hash type — by its length and format — is how crackers decide which attack to use.

Code examples

Below is the recommended INSECURE -> SECURE -> VERIFY shape. Everything runs locally; nothing is downloaded or attacks any remote system.

1. WARNING: intentionally vulnerable — use only in a local, isolated, authorized lab. Do not deploy.

This snippet shows the classic mistake: "protecting" a low-entropy secret by storing its fast, unsalted hash. It looks safe because the digest is unreadable, but it is trivially reversible by brute force.

# INSECURE DEMO ONLY -- do NOT use for real secrets or in production.
import hashlib

def store_pin_insecurely(pin: str) -> str:
    # A fast, unsalted hash of a 4-digit PIN. Only 10,000 inputs exist.
    return hashlib.sha256(pin.encode()).hexdigest()

stored = store_pin_insecurely("0427")   # pretend this row is in a stolen DB dump

# An attacker who reads `stored` recovers the PIN in milliseconds:
for guess in range(10000):
    candidate = f"{guess:04d}"
    if hashlib.sha256(candidate.encode()).hexdigest() == stored:
        print("recovered PIN:", candidate)   # prints 0427
        break

Why it is broken: one-wayness is meaningless when the input space is tiny. The digest commits to the PIN but does not hide it. Adding a salt would stop precomputed rainbow tables but still would not save a 4-digit secret — the honest answer is that a PIN this short cannot be protected by hashing at all, so a real system rate-limits and locks the account instead.

2. SECURE — the correct use of a plain hash: fail-closed integrity verification

A bare hash is the right tool for integrity of a large, unpredictable input. This script recomputes a file's digest and refuses anything that does not match.

#!/usr/bin/env bash
# verify-integrity.sh -- check that a file matches its published SHA-256 digest.
set -euo pipefail   # stop on errors, undefined vars, and failed pipes

FILE="${1:-}"                 # path to the file we received
EXPECTED="${2:-}"            # the digest the publisher gave us (64 hex chars)

if [ -z "$FILE" ] || [ -z "$EXPECTED" ]; then
  echo "usage: $0 <file> <expected-sha256>" >&2
  exit 64
fi
if [ ! -f "$FILE" ]; then
  echo "error: no such file: $FILE" >&2
  exit 1
fi

# Compute the actual digest. `cut` keeps just the hex, dropping the filename.
ACTUAL="$(sha256sum "$FILE" | cut -d' ' -f1)"

# Normalize to lowercase so comparison is case-insensitive.
EXPECTED="$(printf '%s' "$EXPECTED" | tr 'A-Z' 'a-z')"
ACTUAL="$(printf '%s' "$ACTUAL"   | tr 'A-Z' 'a-z')"

if [ "$ACTUAL" = "$EXPECTED" ]; then
  echo "OK: integrity verified"
else
  echo "MISMATCH: file may be corrupted or tampered with" >&2
  echo "  expected: $EXPECTED" >&2
  echo "  actual:   $ACTUAL"   >&2
  exit 2          # non-zero exit so callers/CI can fail-closed
fi

3. VERIFY — prove the fix ACCEPTS good input and REJECTS bad input

# Good input: the digest of the exact bytes must verify.
printf '%s' 'hello' > greeting.txt
GOOD="$(sha256sum greeting.txt | cut -d' ' -f1)"
./verify-integrity.sh greeting.txt "$GOOD"
echo "exit=$?"        # expect: OK: integrity verified   /   exit=0

# Bad input: flip one byte; the check must refuse it.
printf '%s' 'hellO' > greeting.txt      # capital O, one byte changed
./verify-integrity.sh greeting.txt "$GOOD"
echo "exit=$?"        # expect: MISMATCH ...           /   exit=2

Expected output. The first call prints OK: integrity verified and exits 0. The second prints MISMATCH: file may be corrupted or tampered with and exits 2. That non-zero exit is the whole point: a build or installer wired to this script stops instead of trusting a bad file.

Edge cases to know. (1) A trailing newline in the file changes the digest, so generate the expected digest from the exact bytes. (2) This protects against accidental corruption and naive tampering, but if an attacker can replace BOTH the file and the published digest, integrity hashing alone is not enough — that is what digital signatures (next track) add. (3) This comparison is on a non-secret integrity value, so plain = is fine; never compare secret values (like password hashes) with early-exit string equality in a real service — use a constant-time comparison to avoid timing leaks.

Line by line

Walkthrough of the SECURE verification script and the data flowing through it.

Trace with input 'hello':

input bytes      : 68 65 6c 6c 6f            (h e l l o)
sha256sum output : 2cf24dba...938b9824  greeting.txt
after cut        : 2cf24dba...938b9824
after tr         : 2cf24dba...938b9824   (already lowercase)
EXPECTED arg     : 2cf24dba...938b9824
comparison       : equal  -> prints "OK: integrity verified", exit 0

Now imagine the file was tampered so it contains 'hellO' (capital O, byte 4f instead of 6f). Thanks to the avalanche effect the digest is not slightly different — it is entirely different, e.g. 9c6609fc.... The comparison fails and the script exits 2. The key insight: the script never needs to know what changed; any change at all breaks the match.

Common mistakes

Realistic mistakes learners make with hashing, each shown wrong-then-right.

Mistake 1: "Hashing is encryption"

WRONG mindset:
  store H(credit_card_number) and call the data "encrypted" / recoverable

Why it is wrong: there is no key and no inverse. You can never get the card number back from the digest — and for low-entropy data an attacker CAN get it back by brute force. Hashing gives integrity/identification, not confidentiality or reversibility.

CORRECTED:
  Need to read it back later?      -> use encryption (next lesson).
  Need to verify without storing?  -> use a hash, but only on high-entropy input.

How to recognize / prevent: flag any sentence pairing "hashed" with "so it's secure/encrypted/recoverable" in a design review.

Mistake 2: Hashing guessable input for secrecy

WRONG: digest = sha256("0427")   # a 4-digit PIN

Why it is wrong: only 10,000 possible PINs exist. An attacker precomputes sha256 of 0000..9999 once and matches your digest instantly. One-wayness is meaningless when the input space is tiny.

CORRECTED: do not hash a short secret for confidentiality at all.
           For passwords, use a SLOW, SALTED hash (bcrypt/scrypt/Argon2)
           so each guess is expensive and precomputation is useless.
           For a PIN, add rate-limiting and lockout on top.

How to recognize / prevent: watch for small, structured inputs (PINs, SSNs, phone numbers) being hashed "to protect them."

Mistake 3: Using MD5/SHA-1 where collisions matter

WRONG: sign or fingerprint a document with MD5

Why it is wrong: attackers can construct two different documents with the same MD5/SHA-1 digest, so a signature over one also "validates" the other.

CORRECTED: use SHA-256 or SHA-3 for any collision-sensitive use.

How to recognize / prevent: 32-hex-char (MD5) or 40-hex-char (SHA-1) digests in signatures, certificates, or integrity checks.

Mistake 4: The trailing-newline gotcha

WRONG: echo "hello" | sha256sum   # echo appends \n -> different digest!
CORRECTED: printf '%s' 'hello' | sha256sum

Why it is wrong: echo adds a newline, so you hash hello\n, not hello. This is the most common reason "my hashes don't match the example."

Mistake 5: Naive string comparison of secret hashes

WRONG (service code): if (provided_hash == stored_hash) { ... }   # may leak timing
CORRECTED: use a constant-time compare (e.g. hmac.compare_digest / CRYPTO_memcmp)

Why it is wrong: early-exit == can reveal how many leading bytes matched via response timing. For secret comparisons use a constant-time compare. (Plain file-integrity comparison, as in our SECURE script, is not secret, so = is fine there.)

Mistake 6: "The digest verified, so I trust the file"

WRONG: download file AND its digest from the same untrusted mirror, then
       verify file == digest and install

Why it is wrong: an active attacker who controls the mirror simply rewrites both. A matching hash proves internal consistency, not authenticity.

CORRECTED: verify the digest (or the file) against a SIGNED source -- a GPG
           signature or a digest fetched over an independently trusted channel.

Debugging tips

When hashing "doesn't work," the problem is almost always the input bytes or the algorithm, not the math. Work through these:

"My digest doesn't match the published one."

• Did a tool add a trailing newline? Compare echo vs printf '%s'.
• Are you hashing the same encoding? UTF-8 vs UTF-16, or CRLF vs LF line endings, change the bytes and thus the digest.
• Are you using the same algorithm? md5sum vs sha256sum give different lengths — count the hex chars (32 vs 64).
• Did the download finish? A truncated file has a different digest; check the file size first.

"Two different files have the same digest!" (with a modern hash)

• Almost certainly the files are actually identical (e.g., one is a copy or a symlink). Confirm with diff or by comparing sizes.
• If you genuinely produced a collision with MD5/SHA-1, that is expected — those are broken; switch algorithms.

"My password check always fails / always passes."

• Are you hashing the salt consistently on both store and verify?
• Are you comparing the full digest, in the same encoding (hex vs base64)?
• Did you accidentally re-hash an already-hashed value?

Concrete steps:

1. Print the exact bytes you are hashing (xxd or a hex dump) before computing the digest.
2. Compute a known test vector — SHA-256 of empty input is e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855. If your tool reproduces that, the tool is fine and your input is the issue.
3. Count the output length to confirm the algorithm (32/40/64 hex chars = MD5/SHA-1/SHA-256).
4. Check the exit code of a -c verification with echo $? — a silent script may still be failing.

Questions to ask when it fails: What exact bytes go in? Which algorithm, and which encoding of the output? Is there hidden whitespace, a BOM, or a newline? Am I comparing against a trusted, ideally signed, reference?

Memory safety

Security & safety

This is a defensive lesson. Everything below is about building systems that resist attackers and about auditing your own systems — never about attacking systems you do not own.

Authorization & ethics. Only compute or crack hashes on data and systems you are authorized to test: your own files, your own lab VMs, intentionally vulnerable training targets, or CTF challenges on localhost / containers. Cracking someone else's password hashes without explicit written authorization is illegal in most jurisdictions, full stop.

Authorization checklist before any hashing/cracking lab:

[ ] The data/hashes are mine, or I have written authorization to test them
[ ] The lab runs on localhost / a container / an isolated VM (no third party)
[ ] I am using my OWN sample secret (e.g. my own PIN) as the target
[ ] I have a cleanup step to delete generated hashes/wordlists afterward

Lab cleanup / reset. After a demo, remove the artifacts: rm -f greeting.txt pin.txt hashes.txt candidates.txt and clear shell history entries containing a sample secret if you typed one (history -d <n>). Never commit sample hashes or secrets to a repo.

Risks specific to hashing:

• Low-entropy inputs are not protected. Hashing a PIN/SSN/phone number for "secrecy" fails to brute force or rainbow tables. Do not do it; add rate-limiting/lockout and, where reversibility is needed, encryption.
• Unsalted password hashes let one precomputed table crack every user at once, and reveal that two users share a password. Always use a unique random salt per password plus a slow KDF.
• Broken algorithms. MD5/SHA-1 collisions enable forged signatures and certificates. Use SHA-256/SHA-3.
• Timing side channels. Comparing secret digests with early-exit equality can leak information. Use constant-time comparison for secrets.
• Consistency != authenticity. A matching hash from an untrusted source proves nothing; pair the hash with a signature.

Detection & logging guidance. For each security-relevant event, log a structured record: timestamp, source (IP / user id), the resource or file involved, the result (pass/fail), the security decision taken (allowed/blocked), and a correlation id to tie related events together. Concretely: log integrity-check failures (file, expected vs actual digest, source) and repeated authentication failures, which can indicate offline-cracking success being replayed. Never log the password, the raw salt+hash in a recoverable form, session tokens, cookies, private keys, or unnecessary PII. Use placeholders like API_KEY=<development-placeholder> in examples and configs.

Which events signal abuse: a spike in integrity mismatches from one source, a burst of failed logins across many accounts (credential stuffing), or successful logins immediately following a database-dump incident. False positives arise too: a legitimately updated file changes its digest, a proxy re-encodes content, or a user simply mistypes a password several times — so alert on rates and patterns, not single events, and correlate with change-management records before declaring an incident.

Real-world uses

Concrete authorized real-world use. A release engineer publishes a Linux ISO plus a signed SHA256SUMS file. Before the CI pipeline promotes the build, it runs sha256sum -c SHA256SUMS and verifies the GPG signature on that sums file; any mismatch fails the pipeline. This is entirely on infrastructure the team owns, and it protects every downstream user from a corrupted or swapped image.

Other concrete uses:

• Git names every object (commit, tree, blob) by its hash. When you git pull, content is identified and deduplicated by digest — change one byte and you get a different object id.
• Package managers and OS installers (apt, npm, ISO downloads) publish SHA-256 digests so you can verify you received the genuine, uncorrupted file before installing it.
• TLS certificates and code signing sign the hash of the certificate/binary; this is why collision-resistant hashes are mandatory there and why SHA-1 was phased out.
• Deduplication and CDNs use content hashes as cache keys so identical files are stored/served once.
• Password storage in any well-built web app uses a slow salted hash so a database breach does not equal instant account takeover.

Professional best-practice habits:

Beginner rules:

• Pick the algorithm by purpose: SHA-256/SHA-3 for integrity; bcrypt/scrypt/Argon2 for passwords.
• Never call hashing "encryption." If you need the data back, you need encryption.
• Never hash low-entropy secrets for confidentiality.
• Always verify downloads against a published — ideally signed — digest, and fail closed.

Advanced rules:

• Salt every password and tune the KDF cost to your hardware; revisit the cost over time (least-privilege applies: the verification service needs only the hash, never the plaintext).
• Prefer signed digests (a hash plus a signature) when defending against active tampering, not a bare hash.
• Use constant-time comparison for any secret value.
• Plan for crypto agility: store which algorithm/version produced each digest so you can migrate when an algorithm weakens (as MD5 and SHA-1 did).
• Secure defaults + error handling: fail closed on mismatch, log the failure with a correlation id, and never leak the secret in the error message.

Practice tasks

Work these in order; each builds on the last. Solutions are intentionally not given. Any task touching secrets is lab-only, on your own data.

Beginner 1 — Compute and compare

Objective: get comfortable producing digests. Requirements: create a text file with the single word hello (no newline), compute its SHA-256, then add one character and recompute. Input/Output: two 64-hex-char digests that are completely different. Constraints: local files only. Hint: use printf '%s' to avoid an accidental newline. Concepts: deterministic property, avalanche effect.

Beginner 2 — Identify the hash type

Objective: classify digests by format. Requirements: given these strings, label each as MD5, SHA-1, or SHA-256 and justify by length:

5f4dcc3b5aa765d61d8327deb882cf99
da39a3ee5e6b4b0d3255bfef95601890afd80709
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Constraints: decide using length / character set only. Concepts: recognizing a hash by format.

Intermediate 1 — Fail-closed integrity gate

Objective: turn the SECURE script into a guard. Requirements: write a script that verifies a file against an expected SHA-256 and exits non-zero on mismatch, then prove it by flipping one byte of the file and re-running. Input/Output: OK and exit 0 on the original; a clear MISMATCH and non-zero exit after tampering. Constraints: reuse the lowercase-normalization and cut ideas; verify the exit code with echo $?. Hint: fail-closed means the default path on any doubt is refusal. Defensive conclusion: this demonstrates detection of tampering; note in a comment that a bare hash still needs a signature against an active attacker, and remove the test files when done. Concepts: integrity checking, fixed-size digest comparison, fail-closed design.

Intermediate 2 — Why a PIN hash leaks (lab-only)

Objective: demonstrate the low-entropy weakness on your OWN data in an isolated lab. Requirements: write your OWN 4-digit PIN to a file, hash it, then write a short loop that hashes 0000..9999 locally and reports which candidate matches your stored digest. Use only your own PIN, on localhost. Constraints: localhost only; authorization checklist completed first; this illustrates why the scheme is unsafe, not how to attack anyone. Hint: 10,000 iterations is trivial — that is the whole point. Defensive conclusion: state the remediation (never hash short secrets for secrecy; add rate-limiting/lockout; use encryption if reversibility is needed), then delete the PIN and hash files to reset the lab. Concepts: one-wayness vs input entropy, why hashing guessable data fails.

Challenge — Design a safe password-storage scheme

Objective: specify (in writing, no full code) how to store user passwords safely against the database-dump threat model in this lesson. Requirements: describe the algorithm choice, the role and storage location of the salt, how verification works, how you compare hashes safely, exactly what you log (fields) and what you never log, and how you migrate if the algorithm later weakens. Constraints: must defend against the DB-dump threat; no plaintext or reversible storage; no MD5/SHA-1. Hint: think "slow + unique salt + crypto-agile + constant-time compare + fail-closed logging." Defensive conclusion: include a retest step — how you would confirm, after implementing, that the scheme rejects wrong passwords and that no plaintext or secret appears in logs. Concepts: threat modeling, slow KDFs, salting, crypto agility, secure comparison, logging without secrets.

Summary

• A cryptographic hash maps any input to a fixed-size digest and has four core properties: deterministic, one-way (preimage-resistant), collision-resistant, and fixed-size. The avalanche effect means any one-bit change scrambles the whole output, which is what makes hashes great tamper detectors.
• Use hashing for integrity (verify downloads/updates, fail closed), identification/deduplication (Git, content addressing), and password verification — but passwords need a deliberately slow, salted KDF (bcrypt/scrypt/Argon2), not a fast one.
• Key syntax to remember: sha256sum file (or shasum -a 256), printf '%s' 'text' | sha256sum to hash a string without an accidental newline, and sha256sum -c SHA256SUMS to fail closed on a bad file.
• Common mistakes: calling hashing "encryption"; hashing guessable inputs (PINs, phone numbers) for secrecy; using broken MD5/SHA-1 where collisions matter; the trailing-newline mismatch; comparing secret hashes with non-constant-time equality; and trusting a hash fetched from the same untrusted source as the file (you need a signature).
• Detection & logging: record integrity-check and auth failures with timestamp, source, resource, result, decision, and correlation id; never log passwords, salt+hash, tokens, or private keys; alert on rates and patterns, not single events.
• What to remember: a hash hides nothing on its own — it only commits you to a value. It gives integrity and identity, never confidentiality; nothing is ever "completely secure." For confidentiality you need encryption, the next lesson. Always test on authorized, local systems only, and clean up afterward.

Practice with these exercises

• Validate a Canonical SHA-256 Hex Digest
• Constant-Time Digest Comparison
• Internet checksum (16-bit ones-complement)
• XOR checksum
• Compute a CRC-32 checksum