Mutexes — pthread_mutex_t

Overview

In the previous lesson on race conditions you saw that when two threads touch the same variable without coordination, updates get lost: counter++ is really load, add, store, and two threads can interleave those steps so one increment vanishes. That lesson showed you the problem. This lesson gives you the standard fix: the mutex.

A mutex (mutual-exclusion lock) is an object that only one thread can "hold" at a time. You wrap the dangerous code — the critical section — between a lock and an unlock. While one thread holds the lock, any other thread that tries to lock waits its turn. That single guarantee turns the non-atomic counter++ into an operation no other thread can interleave with, which is exactly what was missing before.

Why it matters

Nearly every real multithreaded program — a web server tracking connection counts, a database buffer pool, a logging subsystem, a reference-counted object — protects its shared state with a mutex. Getting it wrong produces the nastiest class of bugs in systems programming: data races that corrupt memory intermittently and vanish under a debugger, and deadlocks that hang the whole process. In security terms, an unsynchronized shared counter or free-list is a classic source of double-free and use-after-free vulnerabilities, because a torn or lost update can leave an allocator or reference count in an impossible state that an attacker can steer.

Core concepts

The critical section and the lock/unlock pattern

A critical section is any span of code that reads or writes shared mutable state and would misbehave if two threads ran it at once. A mutex protects a critical section. The pattern never changes:

lock → touch shared state → unlock

pthread_mutex_t m = PTHREAD_MUTEX_INITIALIZER;
pthread_mutex_lock(&m);
/* critical section: exactly one thread is here at a time */
counter++;
pthread_mutex_unlock(&m);

Think of the lock as a single key on a hook. To enter the room you must take the key; if it is gone, you wait by the hook. When you leave you hang the key back. Only one person is ever in the room.

  Thread A            mutex (key)          Thread B
  --------            -----------          --------
  lock()  ---------->  [taken by A]
  counter load                             lock()  --. 
  counter store                            (blocked)  |  waits
  unlock() --------->  [free]                       <-'
                       [taken by B] <------ lock() resumes
                                            counter load
                                            counter store
                                            unlock()

Because B physically cannot execute its load/store until A has finished and unlocked, the two increments can no longer interleave. The lost update from the race-conditions lesson is gone.

Initialization: static vs dynamic

A mutex must be initialized exactly once before first use. There are two ways:

The static form is simplest and is what you use for a global lock. The dynamic form is required when the mutex is a field inside an object you allocate at runtime, because a struct you malloc cannot use the static initializer.

Knowledge check: you added a pthread_mutex_t lock; field to a struct account that you allocate with malloc. How do you initialize and later dispose of that lock?

Call pthread_mutex_init(&acct->lock, NULL) right after allocating (you cannot assign PTHREAD_MUTEX_INITIALIZER to a runtime-allocated field portably), and call pthread_mutex_destroy(&acct->lock) before free(acct). Destroying after free, or freeing while another thread might still lock it, is undefined behaviour.

Lock all accesses, not just writes

A subtle but critical rule: reads of the shared variable must also be under the lock, not just writes. If one thread updates a multi-word value while another reads it lock-free, the reader can observe a torn value — a mix of old and new bytes that was never a valid state. Even for a single long, reading without the lock is a data race, which the C standard declares undefined behaviour; the compiler may then cache the value in a register and never see another thread's update. The rule: every thread that touches a piece of shared state must hold the same mutex while doing so.

Keep the critical section small

While a thread holds the lock, every other thread that needs it is stalled. So do the minimum inside the lock and push slow work (I/O, malloc, formatting, long loops) outside it.

/* BAD: holds the lock across a slow syscall — everyone stalls */
pthread_mutex_lock(&m);
read(fd, buf, sizeof buf);   /* could block for seconds */
total += n;
pthread_mutex_unlock(&m);

/* GOOD: slow work outside, lock only the shared update */
ssize_t n = read(fd, buf, sizeof buf);
pthread_mutex_lock(&m);
total += n;
pthread_mutex_unlock(&m);

Knowledge check: why is holding a mutex across a blocking read() dangerous beyond just being slow?

Besides serializing all threads behind the slow call, it can cause deadlock: if the thread is waiting on data that only arrives after another thread — which is now blocked on your mutex — does its work, neither can proceed. Locks should protect memory, not I/O.

Syntax notes

/* Static initializer: default attributes, no destroy required. */
pthread_mutex_t m = PTHREAD_MUTEX_INITIALIZER;

/* Dynamic init. attr = NULL means default attributes. Returns 0 on success,
   or an errno-style positive error code (does NOT set errno). */
int pthread_mutex_init(pthread_mutex_t *mutex, const pthread_mutexattr_t *attr);

/* Blocks until the calling thread owns the lock. Returns 0, or an error code
   (e.g. EDEADLK if a default mutex detects self-relock, on some systems). */
int pthread_mutex_lock(pthread_mutex_t *mutex);

/* Non-blocking attempt. Returns 0 if acquired, EBUSY if already held. */
int pthread_mutex_trylock(pthread_mutex_t *mutex);

/* Releases a lock the calling thread holds. Unlocking a mutex you do not own,
   or one already unlocked, is undefined behaviour. Returns 0 or an error code. */
int pthread_mutex_unlock(pthread_mutex_t *mutex);

/* Releases resources of an initialized, currently-unlocked mutex. Destroying a
   locked mutex is undefined behaviour. Required for dynamically-init'd mutexes. */
int pthread_mutex_destroy(pthread_mutex_t *mutex);

Key conventions: these functions return 0 on success and a positive error number on failure — they do not set errno, so check the return value directly (e.g. int rc = pthread_mutex_lock(&m); if (rc) { /* handle rc */ }). Compile and link with -lpthread (or -pthread). A statically-initialized mutex needs no destroy; a pthread_mutex_init'd one must be destroyed before its memory is freed, and only while unlocked.

Lesson

What a mutex is

A mutex (short for mutual exclusion lock) makes sure that only one thread at a time can enter a critical section — a region of code that touches shared data.

The pattern is always the same:

lock → modify shared state → unlock

pthread_mutex_t m = PTHREAD_MUTEX_INITIALIZER;
pthread_mutex_lock(&m);
/* … exactly one thread is here … */
pthread_mutex_unlock(&m);

While one thread holds the lock, any other thread that calls pthread_mutex_lock waits until the lock is released.

Rules to follow

• Lock before every read and every write of the shared variable. Reading without the lock can return a torn value — a half-updated value caught mid-change.
• Unlock on every exit path, including error paths. A forgotten unlock leaves the lock held forever, which causes a deadlock (other threads wait forever).
• Hold the lock for as little code as possible. While a mutex is held, every other thread that needs it is blocked.
• Call pthread_mutex_destroy when you are done with a mutex that you allocated on the heap.

Code examples

#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>

#define ITERS 1000000

/* Shared state, guarded by one mutex. */
static long            guarded_counter = 0;
static long            naive_counter   = 0;
static pthread_mutex_t counter_lock    = PTHREAD_MUTEX_INITIALIZER;

/* Increments both counters: one under the lock, one without. */
static void *worker(void *arg) {
    (void)arg;
    for (int i = 0; i < ITERS; i++) {
        /* Unsafe: a data race. Kept only to show the lost-update bug. */
        naive_counter++;

        /* Safe: lock -> modify -> unlock. Keep the region tiny. */
        pthread_mutex_lock(&counter_lock);
        guarded_counter++;
        pthread_mutex_unlock(&counter_lock);
    }
    return NULL;
}

int main(void) {
    pthread_t t[4];
    for (int i = 0; i < 4; i++) {
        if (pthread_create(&t[i], NULL, worker, NULL) != 0) {
            perror("pthread_create");
            return EXIT_FAILURE;
        }
    }
    for (int i = 0; i < 4; i++)
        pthread_join(t[i], NULL);

    long expected = 4L * ITERS;
    printf("expected        = %ld\n", expected);
    printf("guarded_counter = %ld  (mutex-protected)\n", guarded_counter);
    printf("naive_counter   = %ld  (unprotected; usually wrong)\n", naive_counter);
    printf("guarded is correct: %s\n", guarded_counter == expected ? "yes" : "no");

    /* Statically-initialized mutexes need no destroy, but it is legal. */
    pthread_mutex_destroy(&counter_lock);
    return 0;
}

Line by line

• #include <pthread.h> brings in pthread_mutex_t, the lock/unlock functions, and thread creation. Remember to link with -lpthread.
• static long guarded_counter / naive_counter: two shared counters. One will be updated under the lock, the other deliberately without, so you can see the difference in one run.
• static pthread_mutex_t counter_lock = PTHREAD_MUTEX_INITIALIZER;: the global lock, initialized statically. Because it is a file-scope object with a fixed lifetime, the static initializer is the right tool and no pthread_mutex_init call is needed.
• Inside worker, naive_counter++; runs with no protection. It is a data race across four threads; increments will be lost.
• The three lines pthread_mutex_lock(&counter_lock); guarded_counter++; pthread_mutex_unlock(&counter_lock); are the whole point: the critical section is exactly one statement, held for the shortest possible time. Only one thread can be between the lock and unlock, so guarded_counter++ becomes effectively atomic.
• In main, we spawn four threads with pthread_create, checking its return value (it returns an error code, not -1). Four threads make the race in naive_counter almost always visible.
• pthread_join waits for each thread to finish. Without joining, main could print and exit while workers are still running — and reading the counters then would itself be a race.
• expected = 4L * ITERS is the arithmetically correct total. The guarded_counter matches it every time; naive_counter almost never does.
• pthread_mutex_destroy(&counter_lock) is optional for a static mutex but shown for completeness; it must only ever be called when the mutex is unlocked and no thread will lock it again — here, after all joins.

Common mistakes

• Returning while still holding the lock. Wrong: pthread_mutex_lock(&m); if (err) return -1; pthread_mutex_unlock(&m); — the error path never unlocks, so the next thread to lock blocks forever (a deadlock). Fixed: unlock on every path — pthread_mutex_lock(&m); if (err) { pthread_mutex_unlock(&m); return -1; } /* ... */ pthread_mutex_unlock(&m);, or route all exits through a single goto cleanup; label that unlocks once.
• Locking writes but reading lock-free. Wrong: writer does lock(); balance -= amt; unlock(); while a reporter thread does printf("%ld", balance); with no lock. That read is a data race and may see a torn or stale value. Fixed: take the same lock around the read — lock(); long b = balance; unlock(); printf("%ld", b);.
• Using PTHREAD_MUTEX_INITIALIZER on a heap struct. Wrong: struct S *s = malloc(sizeof *s); s->m = PTHREAD_MUTEX_INITIALIZER; — assigning the initializer macro to a runtime field is not portable/valid. Fixed: pthread_mutex_init(&s->m, NULL); after allocation, and pthread_mutex_destroy(&s->m); before free(s);.
• Re-locking a mutex you already hold. Wrong: a function locks m, then calls a helper that also locks m — a default (non-recursive) mutex deadlocks against itself. Fixed: either restructure so the helper assumes the lock is already held (document it as "caller must hold m"), or split the shared work into a lock-free inner function called from within the critical section.

Debugging tips

• Symptom: total is wrong / non-deterministic. That is a missing or inconsistent lock. Run under a thread sanitizer: cc -std=c11 -fsanitize=thread -g prog.c -lpthread && ./a.out — TSan pinpoints the exact unsynchronized read/write pair and both stack traces.
• Symptom: the program hangs. That is a deadlock or a lock held on an error path. Attach with gdb -p <pid>, then thread apply all bt; a thread stuck in __lll_lock_wait / pthread_mutex_lock shows who is waiting, and you can often see the owner elsewhere holding it.
• Check return codes. These functions return an error number, not -1, and do not set errno. Temporarily wrap calls: int rc = pthread_mutex_lock(&m); if (rc) fprintf(stderr, "lock: %s\n", strerror(rc));.
• Reproduce races reliably. Increase iteration counts and thread counts, and add tiny sched_yield() calls between the load and store of a suspected unprotected variable to widen the race window.
• On Linux, strace -f -e trace=futex ./a.out shows the futex syscalls the mutex uses under contention; a thread blocked forever in FUTEX_WAIT is a stuck lock.

Memory safety

The whole point of a mutex is to eliminate data races, which the C11 memory model defines as undefined behaviour: two threads accessing the same non-atomic object where at least one writes, without synchronization, gives the compiler and CPU freedom to reorder, cache, or tear the access — results are not merely 'sometimes wrong' but formally undefined. A mutex establishes a happens-before relationship: everything one thread did before unlock is visible to the next thread after it locks, so no explicit memory barriers are needed. Hazards specific to mutexes: (1) use-after-destroy / destroy-while-locked — never pthread_mutex_destroy a locked mutex or one another thread may still use, and never free a struct containing a live mutex. (2) Guarding the wrong lock — protecting a variable with different mutexes in different places gives zero protection; one variable, one lock. (3) Recursive self-lock — a default mutex is not recursive; relocking it in the same thread is undefined/deadlock. (4) Uninitialized use — locking a mutex that was never initialized is undefined behaviour. For a plain counter, an alternative is _Atomic long with atomic_fetch_add, which needs no lock at all; reach for a mutex when you must update several related fields together as one indivisible transaction.

Real-world uses

Mutexes appear everywhere shared mutable state meets threads: reference counts in shared_ptr-like schemes and in the CPython/PHP interpreters, connection and request counters in web servers (nginx worker stats, Apache scoreboards), buffer pools and lock-managers in databases (PostgreSQL, SQLite in multithread mode), free-lists inside malloc implementations, and the internal state of thread-safe logging libraries. Best practice: keep one lock per logical piece of state and document what each lock protects; keep critical sections tiny; never call into unknown code (callbacks, allocation, I/O) while holding a lock; establish and document a consistent lock-ordering when you need more than one lock (the topic of the next lesson on deadlocks); and prefer higher-level abstractions — atomics for single counters, read-write locks for read-mostly data, or a message-passing design that avoids sharing altogether — when they fit better than a plain mutex.

Practice tasks

1. Take the naive counter from the example and add a second mutex-protected counter that four threads increment; confirm the protected one always equals threads * iterations while the naive one drifts low.

2. Convert a global pthread_mutex_t and the counter it guards into a heap-allocated struct counter { long value; pthread_mutex_t lock; }. Use pthread_mutex_init / pthread_mutex_destroy correctly and free the struct only after all threads join.

3. Write int withdraw(struct account *a, long amt) that locks, checks balance >= amt, subtracts, and unlocks — returning -1 (still unlocking!) when funds are insufficient. Verify with a goto cleanup version that every path unlocks exactly once.

4. Add a reporter thread that periodically prints the counter. First read it without the lock and observe (with TSan) that it is a data race; then fix it by reading a snapshot under the same lock and confirm TSan goes quiet.

5. Deliberately create a self-deadlock: have a locked function call a helper that locks the same default mutex, and observe the hang under gdb. Then fix it two ways — (a) refactor so the helper needs no lock, and (b) research and use a recursive mutex attribute — and explain the trade-off.

Summary

• A pthread_mutex_t enforces mutual exclusion: only one thread at a time can be inside the critical section it guards.
• The pattern is always lock → modify shared state → unlock; this makes non-atomic operations like counter++ safe.
• Guard every access — reads as well as writes — with the same mutex; one piece of state, one lock.
• Initialize statically with PTHREAD_MUTEX_INITIALIZER for globals, or dynamically with pthread_mutex_init for heap structs (and pthread_mutex_destroy before freeing, only when unlocked).
• Unlock on every exit path, including errors — a forgotten unlock deadlocks the program.
• Hold the lock for as little code as possible; never do slow I/O or call unknown code while holding it.
• These functions return an error code (not -1) and do not set errno; check the return value. Link with -lpthread.

Practice with these exercises

• Are lock/unlock balanced?
• Critical section nesting
• Fix the race with a pthread_mutex_t
• Build a tiny thread-safe counter type