Multiply two ints with overflow detection

Challenge

Implement int mul_safe(int a, int b, int *out).

Return:

• 0 on success; *out holds a * b.
• -1 on overflow; *out is unchanged.

The check is more involved than addition because of sign combinations. The simplest approach is:

• If a == 0 or b == 0: result is 0, no overflow.
• Compute result = (long long)a * (long long)b.
• Overflow iff result < INT_MIN || result > INT_MAX.

Examples

int out;
mul_safe(3, 4, &out)            -> 0, out == 12
mul_safe(0, INT_MAX, &out)      -> 0, out == 0
mul_safe(INT_MAX, 2, &out)      -> -1
mul_safe(INT_MIN, -1, &out)     -> -1   // would be -INT_MIN which overflows
mul_safe(-3, -4, &out)          -> 0, out == 12

Why this matters

Computing malloc(count * size) without checking overflow is the textbook 'OpenSSL Heartbleed-class' bug: the multiplication wraps to a small value, the allocator hands back a tiny buffer, and the subsequent loop writes past the end. The fix is a pre-check.

Input format

Two ints + output pointer.

Output format

0/-1 + maybe *out.

Constraints

Portable C; no __builtin_mul_overflow.

Starter code

#include <limits.h>
int mul_safe(int a, int b, int *out) { /* TODO */ return -1; }

Common mistakes

Doing the multiplication in int first, then comparing — undefined behaviour. Forgetting that INT_MIN * -1 overflows by 1.

Edge cases to handle

Zero times anything; INT_MIN * -1; INT_MAX * 1; INT_MIN * INT_MIN.

Complexity

O(1).

Background lessons

• Data types
• Integer overflow

Up next

• Grow a buffer and zero only the new region
• Add two ints with overflow detection