Detect disguised executable magic bytes

Challenge

Attackers upload malicious executables disguised with benign file extensions (e.g. resume.pdf.exe or avatar.png). File validation must inspect magic header bytes rather than trusting file extensions.

Your Task

Implement:

int is_executable_blob(const unsigned char *buf, size_t len);

Inspect buf to detect executable magic signatures.

Recognized Signatures

1. Linux ELF: len >= 4 and starts with 0x7F, 'E', 'L', 'F'
2. Windows PE / DOS: len >= 2 and starts with 'M', 'Z'
3. Java Class / Mach-O Fat: len >= 4 and starts with 0xCA, 0xFE, 0xBA, 0xBE
4. Mach-O 32/64-bit: len >= 4 and starts with 0xFE, 0xED, 0xFA, 0xCE or 0xFE, 0xED, 0xFA, 0xCF (or reverse endian 0xCE/0xCF, 0xFA, 0xED, 0xFE)
5. Script Shebang: len >= 2 and starts with '#', '!'

Rules

• If buf == NULL, return -1.
• If any executable magic matches, return 1.
• If clean/non-executable or len is too short to match, return 0.

Input format

buf: byte buffer; len: byte count.

Output format

Returns 1 if executable, 0 if non-executable, -1 on NULL pointer.

Constraints

Freestanding C11. Read-only buffer inspection.

Starter code

#include <stddef.h>

/* Inspect buf for executable magic (ELF, PE, Mach-O, Java class, shebang).
   Return 1 if executable, 0 if safe/non-executable, or -1 if buf is NULL. */
int is_executable_blob(const unsigned char *buf, size_t len) {
    (void)buf; (void)len;
    return -1;
}

Common mistakes

Reading past len when checking 4-byte signatures; ignoring script shebang (#!).

Edge cases to handle

len < 2 returns 0; buffer with ELF prefix but len == 3 returns 0; NULL pointer returns -1.

Background lessons

• Classify a firmware blob from its magic bytes