rlimit — capping CPU, memory, fds

Overview

setrlimit puts a per-process cap on a single resource, such as memory or CPU time.

Each resource has two limits:

• The soft limit is the value currently enforced.
• The hard limit is the ceiling for the soft limit. A normal process may raise its soft limit only up to the hard limit.

Once you lower the hard limit, only root can raise it again. This is a one-way door, which makes it useful for dropping your own privileges before running risky code.

Why it matters

Sandboxing in C starts with resource limits.

Every Docker container, every CI runner, and every code-execution service tunes them to keep workloads contained.

Core concepts

Common resources you can cap

• RLIMIT_AS — total address space (anonymous memory plus mmap'd regions). This effectively caps how much malloc can hand out.
• RLIMIT_CPU — CPU seconds consumed. The kernel sends SIGXCPU when the soft limit is reached, and SIGKILL at the hard limit.
• RLIMIT_FSIZE — maximum size of a file the process may write. Overrunning it raises SIGXFSZ.
• RLIMIT_NOFILE — maximum number of open file descriptors.
• RLIMIT_NPROC — maximum number of processes for the user.

Pentester mindset

When auditing a sandbox runner, check whether it sets resource limits at all.

• Missing RLIMIT_CPU means an attacker can pin a CPU core forever.
• Missing RLIMIT_AS means an attacker can exhaust memory and trigger the out-of-memory (OOM) killer.

Defensive coding habit

Lower both the soft and hard limits at the start of a section that runs with privileges before dropping them.

Never raise a hard limit that you set yourself. You cannot, unless you are root.

Syntax notes

#include <sys/resource.h>

int getrlimit(int resource, struct rlimit *rlim);
int setrlimit(int resource, const struct rlimit *rlim);

getrlimit reads the current limits for a resource. setrlimit changes them.

Lesson

setrlimit and getrlimit configure per-process resource caps: CPU seconds, address space, file size, and number of open file descriptors.

They are the user-space foundation of every sandbox, including ours.

Code examples

struct rlimit r = { .rlim_cur = 128 * 1024 * 1024, .rlim_max = 128 * 1024 * 1024 };
setrlimit(RLIMIT_AS, &r);     /* 128 MB max address space */

Line by line

struct rlimit r;
r.rlim_cur = r.rlim_max = 5;              /* 5 CPU-seconds */
setrlimit(RLIMIT_CPU, &r);
r.rlim_cur = r.rlim_max = 128 * 1024L * 1024L;   /* 128 MB */
setrlimit(RLIMIT_AS, &r);

Common mistakes

• Raising a hard limit after dropping it. Once you lower a hard limit, you cannot raise it again unless you are root.

Debugging tips

Several tools let you inspect resource limits:

• ulimit -a in a shell shows your current limits.
• prlimit --pid PID queries any running process.
• cat /proc/PID/limits shows the kernel's view for a given process.

Memory safety

Setting RLIMIT_AS too low can make a later malloc fail in ways you did not expect.

Always test under realistic load before relying on a tight limit.

Real-world uses

• Every code runner.
• Every cgroup ancestor.
• The shell builtin ulimit.

Practice tasks

1. Cap RLIMIT_AS to 128 MB, then watch a 200 MB malloc fail.
2. Cap RLIMIT_CPU to 5 seconds, then run an infinite loop and observe the process being killed.
3. Lower RLIMIT_NOFILE to 16, then watch the 17th open call fail.

Summary

• setrlimit caps a per-process resource (CPU, memory, file size, open fds).
• Each resource has a soft limit (enforced) and a hard limit (the ceiling).
• Lowering the hard limit is permanent for a non-root process, so use it to drop privileges.
• Resource limits are the kernel-backed foundation of every sandbox.

Practice with these exercises

• Clamp to a soft limit
• Over the hard limit?
• Will this allocation fail under the rlimit?