Detect a byte-order mark

Challenge

Detect a byte-order mark at the start of a buffer — the first step in robust text loading, since a stray BOM silently corrupts parsing.

Task

Implement int detect_bom(const uint8_t *buf, size_t n) that inspects the leading bytes of buf and reports which BOM (if any) is present.

Input

• buf: a byte buffer (may be NULL).
• n: number of valid bytes in buf.

Output

Return:

• 1 for the UTF-8 BOM EF BB BF,
• 2 for UTF-16LE FF FE,
• 3 for UTF-16BE FE FF,
• 0 for no BOM, including NULL or too-short input.

Example

{0xEF,0xBB,0xBF,...}   ->   1   (UTF-8)
{0xFF,0xFE,...}        ->   2   (UTF-16LE)
{0xFE,0xFF,...}        ->   3   (UTF-16BE)
{'h','e','l',...}      ->   0
{0xFF} (n=1)           ->   0   (too short)

Edge cases

• NULL buffer or fewer bytes than the signature: 0.
• FF FE vs FE FF distinguishes LE from BE — order matters.

Rules

• Check the 3-byte UTF-8 BOM before the 2-byte UTF-16 BOMs; never read past n.

Why this matters

A leading BOM silently corrupts parsing if you don't skip it; detecting one is the first step in robust text loading.

Input format

buf: byte buffer (may be NULL). n: valid byte count.

Output format

int: 1=UTF-8, 2=UTF-16LE, 3=UTF-16BE, 0=none/NULL/too-short.

Constraints

Check the 3-byte BOM first; respect n so you never read past the buffer.

Starter code

#include <stdint.h>
#include <stddef.h>
int detect_bom(const uint8_t *buf, size_t n) {
    /* TODO */
    (void)buf; (void)n;
    return 0;
}

Common mistakes

Swapping LE/BE. Checking the 2-byte BOM before the 3-byte one. Reading past a short buffer.

Edge cases to handle

No BOM. Only one byte present. NULL.

Complexity

O(1).

Background lessons

• Binary files

Up next

• Extract the final path component