Bit fields: pack and unpack

Overview

Protocol headers and hardware registers pack several small numbers into one word: a 3-bit mode here, a 12-bit length there. Reading one out is a two-step move — shift the field down to position 0, then mask off everything above it. Writing one back is three steps — clear the old field, mask the incoming value to the field's width so a too-large argument cannot spill into its neighbours, then OR it into place. The mask itself is (1u << width) - 1, a value of width consecutive 1 bits, which needs a guard when width is 32 because shifting by the full width is undefined.

Why it matters

Every binary format you will ever parse does this. An IPv4 header packs version and header-length into one byte; a floating-point number is a sign bit, an exponent field and a mantissa field; an instruction encoding splits opcode from operands. If you extract a field with the wrong shift or forget to mask an inserted value, you silently corrupt the fields next to it — and the bug surfaces far away, as a malformed packet or a wrong register setting.

Core concepts

The width mask. (1u << width) - 1 produces width low 1 bits: width 3 → 0b111. This is the same suffix-flip identity from the popcount lesson, used constructively.

The width-32 guard. 1u << 32 is undefined behaviour on a 32-bit unsigned, so a full-width field needs a special case: mask = (width >= 32) ? ~0u : ((1u << width) - 1u).

Extracting. (x >> pos) & mask slides the field down to bit 0 and clears everything above it. Shift first, then mask — the reverse order needs a different mask and is easier to get wrong.

Inserting. (x & ~(mask << pos)) | ((val & mask) << pos). The left half clears the old field in place; the right half clamps val to the field width and moves it into position. The val & mask is the safety step: without it, an oversized value overwrites neighbouring fields.

Clear-then-set. Inserting is the general form of the read-modify-write pattern: never OR a new value over an old one without clearing first, or leftover 1 bits survive.

Syntax notes

static unsigned field_mask(int width) {
    return (width >= 32) ? ~0u : ((1u << width) - 1u);   // guard the full-width case
}

unsigned extract_bits(unsigned x, int pos, int width) {
    return (x >> pos) & field_mask(width);
}

unsigned insert_bits(unsigned x, int pos, int width, unsigned val) {
    unsigned m = field_mask(width);
    return (x & ~(m << pos)) | ((val & m) << pos);   // clear, clamp, place
}

Key points:

• (1u << width) - 1u is the width mask; guard width >= 32.
• val & m clamps the incoming value — do not skip it.
• pos + width must not exceed 32, or the field runs off the end of the word.

Lesson

Compact formats cram several values into one word — an RGB565 pixel, a permission triple, an instruction opcode. Extract a field by shifting it down and masking; insert by clearing the field and OR-ing the new value in. The width==32 case needs care: 1u << 32 is undefined.

The demo packs and unpacks an RGB565 pixel.

Code examples

#include <stdio.h>
static unsigned extract_bits(unsigned x,int pos,int width){ unsigned m=(width>=32)?~0u:((1u<<width)-1u); return (x>>pos)&m; }
static unsigned insert_bits(unsigned x,int pos,int width,unsigned val){ unsigned m=(width>=32)?~0u:((1u<<width)-1u); return (x&~(m<<pos))|((val&m)<<pos); }
int main(void){
    /* Pack an RGB565 pixel: R:5 at 11, G:6 at 5, B:5 at 0. */
    unsigned px = 0;
    px = insert_bits(px, 11, 5, 25);        /* red   */
    px = insert_bits(px,  5, 6, 50);        /* green */
    px = insert_bits(px,  0, 5, 12);        /* blue  */
    printf("packed = 0x%04X\n", px);
    printf("R=%u G=%u B=%u\n", extract_bits(px,11,5), extract_bits(px,5,6), extract_bits(px,0,5));
    return 0;
}

Line by line

Common mistakes

Shifting by 32 (undefined); not masking the inserted value to its width.

Debugging tips

Compiler errors and warnings:

• warning: left shift count >= width of type — width reached 32 and you skipped the guard.
• warning: right shift count >= width of type — pos is out of range; validate it.

Runtime symptoms:

• A neighbouring field changed. You forgot val & m on insert, so an oversized value spilled over. This is the classic bug.
• Old bits survive. You ORed without clearing first; use (x & ~(m << pos)) | ....
• The extracted value is far too large. You masked before shifting, or used the mask unshifted.
• Everything breaks at width 32. The guard is missing — 1u << 32 is undefined, and in practice often evaluates to 1, giving a mask of 0.

Technique: print x, the mask, and the result in hex on one line. Fields are nibble-aligned surprisingly often, which makes hex a natural microscope for this bug class.

Memory safety

Field packing is where undefined shifts and silent corruption meet, so validate aggressively:

• Shift range. Both pos and width must be validated: require 0 <= pos < 32, 0 < width <= 32, and pos + width <= 32. Values from a file or network must never reach a shift unchecked.
• The full-width special case. Always guard width >= 32; the undefined 1u << 32 is a real portability bug that appears only on some targets.
• Clamp on insert. val & m is a safety boundary, not an optimisation — it is what stops a caller's bad value from corrupting adjacent fields.
• Bit-field structs are not a shortcut. C's struct { unsigned mode : 3; } syntax has implementation-defined layout and endianness, so it is unsuitable for parsing external formats. Do it explicitly with shifts and masks.

Real-world uses

Concrete uses: Parsing the IPv4 version/IHL byte or the TCP data-offset/flags word. Decoding an IEEE-754 float into sign, exponent and mantissa. Reading a RISC instruction's opcode and register fields. Configuring a UART's parity and word-length fields inside one control register. Unpacking a colour value into R, G, B and alpha channels.

Professional best practices:

Beginner:

• Write extract/insert helpers once and call them everywhere instead of hand-rolling shifts.
• Name each field's position and width as constants.

Intermediate:

• Validate pos/width at the boundary and treat violations as errors, not as clamped best effort.
• Prefer explicit shift-and-mask over C bit-field structs for anything crossing a machine boundary — layout is not portable.
• Round-trip test: extract(insert(x, p, w, v), p, w) == (v & mask) for random inputs is a strong invariant.

Practice tasks

1. (Beginner) Build a width mask. Implement unsigned field_mask(int width) returning width low 1 bits, with the width >= 32 guard. Example: field_mask(3) → 7; field_mask(32) → 0xFFFFFFFF. Concepts: (1u<<w)-1, undefined shifts.

2. (Beginner) Extract a nibble. Implement unsigned extract_bits(unsigned x, int pos, int width). Example: extract_bits(0xAB, 4, 4) → 0xA. Concepts: shift-then-mask.

3. (Intermediate) Insert safely. Implement insert_bits with clear-then-set and clamping. Requirements: inserting an oversized value must not disturb neighbouring fields. Example: insert_bits(0xAB,4,4,0x1F) → 0xFB, not garbage. Concepts: clamping, read-modify-write.

4. (Intermediate) Decode an IPv4 first byte. Given 0x45, extract the version (high 4 bits) and header length (low 4 bits) and print both. Example: version 4, IHL 5 (= 20 bytes). Concepts: real-world field layout.

Summary

A field is a shift plus a mask: extract with (x >> pos) & mask, insert with clear-then-set — (x & ~(mask << pos)) | ((val & mask) << pos). The mask (1u << width) - 1 needs a guard when width is 32 because shifting by the full width is undefined, and the val & mask clamp is what prevents an oversized value from corrupting the fields beside it. Validate pos and width whenever they come from outside your program, and prefer explicit shifts to C bit-field structs for any format that crosses a machine boundary.

Practice with these exercises

• Extract a bit field
• Insert a bit field