cybersecurity · intermediate · ~15 min

Subtract two ints with overflow detection

Pre-check overflow without performing the unsafe arithmetic.

Challenge

Implement int sub_safe(int a, int b, int *out).

Return:

  • 0 on success; *out = a - b.
  • -1 if the subtraction would overflow int.

Hint: a - b overflows above INT_MAX if b < 0 && a > INT_MAX + b. Below INT_MIN if b > 0 && a < INT_MIN + b.

Why this matters

Signed subtraction overflows in different cases than addition. Audit code rarely checks; the bug class is real in size calculations.

Input format

Two ints + output pointer.

Output format

0/-1 + (maybe) *out.

Constraints

No GCC builtins; portable C.

Starter code

#include <limits.h>
int sub_safe(int a, int b, int *out) { /* TODO */ return -1; }

Common mistakes

Doing a - b first and then comparing — undefined.

Edge cases to handle

INT_MIN - 1; INT_MAX - (-1); 0 - 0; INT_MIN - INT_MIN.

Complexity

O(1).

Background lessons

Up next

Solve this exercise in the browser editor — compile and run against the test harness, no setup required.