Detect a glibc-style stack canary pattern

Challenge

Scan a stack frame's bytes for the tell-tale shape of a glibc stack canary — the low byte is 0x00 and the rest is random.

Task

Implement int has_canary_pattern(const uint8_t *frame, size_t n). Treat the frame as a sequence of 8-byte aligned slots and look for any slot that is canary-shaped:

• byte 0 (the low byte of the slot) is 0x00, and
• at least one of bytes 1..7 is non-zero.

Return:

• 1 if any slot matches,
• 0 if none does,
• -1 if frame == NULL or n is not a multiple of 8.

Input

• frame: a fixed byte buffer the grader provides (may be NULL).
• n: the buffer length in bytes (must be a multiple of 8 to be valid).

Output

Returns 1, 0, or -1 as described above.

Example

all-zero 24-byte frame                              ->   0   (no canary)
slot = {00 AB CD 00 00 00 00 12}                    ->   1   (low byte 0, others non-zero)
slot = {01 AB CD EF FE DC BA 98}                    ->   0   (low byte non-zero)
slot = {00 00 00 00 00 00 00 00}                    ->   0   (all zero, not a canary)
has_canary_pattern(frame, 23)                       ->   -1  (n not a multiple of 8)
has_canary_pattern(NULL, 8)                         ->   -1

Edge cases

• An all-zero slot is NOT a canary (the upper 7 bytes must include a non-zero).
• n not divisible by 8, or frame == NULL: return -1.

Rules

• Walk 8 bytes at a time. Predicate: byte 0 == 0 AND (byte 1 | ... | byte 7) != 0.

Why this matters

Recognising the canary's shape is foundational defensive knowledge — it explains why naive overflows fail and why mitigations matter.

Input format

A fixed byte buffer frame (may be NULL) and its length n (valid only when a multiple of 8).

Output format

1 if any 8-byte slot is canary-shaped, 0 if none, -1 if frame is NULL or n % 8 != 0.

Constraints

Stride 8; a slot matches when its low byte is 0 and at least one upper byte is non-zero.

Starter code

#include <stdint.h>
#include <stddef.h>
int has_canary_pattern(const uint8_t *frame, size_t n) {
    /* TODO */
    (void)frame; (void)n;
    return -1;
}

Common mistakes

Treating all-zero slot as canary. Ignoring the n % 8 != 0 invariant. Walking unaligned.

Edge cases to handle

Empty frame. Single slot. Canary at the very last slot.

Complexity

O(n).

Background lessons

• Detect a stack canary pattern in a memory snapshot

Up next

• A libFuzzer-shape target that never crashes