Multiply two ints with overflow detection

Challenge

Multiply two ints, detecting overflow before it corrupts a size calculation — the classic malloc(count * size) bug where the product wraps and the buffer comes back too small.

Task

Implement int mul_safe(int a, int b, int *out) that stores a * b in *out only when it fits in an int.

A clean portable approach: compute the product in a wider type (long long) and range-check it against INT_MIN/INT_MAX.

Input

• a, b: the two ints to multiply.
• out: where the product is written on success.

Output

Returns 0 on success (with *out = a * b), or -1 on overflow (leaving *out unchanged).

Example

int out;
mul_safe(3, 4, &out)         ->   0, out == 12
mul_safe(0, INT_MAX, &out)   ->   0, out == 0
mul_safe(INT_MAX, 2, &out)   ->   -1
mul_safe(INT_MIN, -1, &out)  ->   -1   (-INT_MIN overflows by one)
mul_safe(-3, -4, &out)       ->   0, out == 12

Edge cases

• Zero times anything is 0 (no overflow).
• INT_MIN * -1 overflows by one.
• INT_MAX * 1 succeeds.

Rules

• Portable C; no __builtin_mul_overflow.
• Do not compute the product in int first and then compare — that is undefined behaviour.

Why this matters

Computing malloc(count * size) without checking overflow is the textbook 'OpenSSL Heartbleed-class' bug: the multiplication wraps to a small value, the allocator hands back a tiny buffer, and the subsequent loop writes past the end. The fix is a pre-check.

Input format

Two ints a, b, and an output pointer out.

Output format

0 on success (with *out = a * b); -1 on overflow (with *out unchanged).

Constraints

Portable C; no __builtin_mul_overflow; never multiply in int before checking.

Starter code

#include <limits.h>
int mul_safe(int a, int b, int *out) { /* TODO */ return -1; }

Common mistakes

Doing the multiplication in int first, then comparing — undefined behaviour. Forgetting that INT_MIN * -1 overflows by 1.

Edge cases to handle

Zero times anything; INT_MIN * -1; INT_MAX * 1; INT_MIN * INT_MIN.

Complexity

O(1).

Background lessons

• Data types
• Integer overflow

Up next

• Grow a buffer and zero only the new region
• Add two ints with overflow detection