Linux System Programming · intermediate · ~20 min
- By the end you can read a `/proc/<pid>/status`-style file into a fixed, bounded line buffer and never overflow, even on hostile input. - By the end you can parse each field with width-limited `sscanf` and check its return value, defaulting to `unknown` (a sentinel) instead of `0` on failure. - By the end you can reason about the per-PID race — a process vanishing mid-read — and handle `ENOENT` on every access after `opendir`. - By the end you can tell stable `/proc` fields from volatile ones and refuse to trust anything that does not match the documented format. - By the end you can build a small forensic parser that stays correct and non-crashing against fast-cycling PIDs (a denial-of-defense test).
The recipes prerequisite showed you how to use /proc: which files to cat, what status, maps, and fd/ reveal, and how to answer forensic questions by hand. This lesson flips the role — you are now the engineer writing the tool that reads those files a thousand times a second inside a security agent. You already know what the fields mean; here you learn how to extract them safely and reliably so a parser bug never becomes a missed detection.
/proc is a virtual filesystem: the kernel synthesizes each file's text on read, there is no bytes-on-disk, and the content can change between two reads of the same file. That reality — text that is generated on demand, for a target that may disappear — is exactly why naive parsing (atoi, unbounded reads, trusting field order) breaks in production. Everything below builds defensive parsing on top of the /proc layout you already understand.
Essentially every Linux DFIR and observability tool — osquery, Falco, auditd, htop, and the endpoint agents that ship on servers — is a /proc parser at its core. A single mis-parsed field there is not a cosmetic bug: it is a blind spot an attacker can hide in, or a crash that takes your monitoring offline (a "denial-of-defense"). Because /proc content is attacker-influenced (a process controls its own comm, its cmdline, its environment), the parser is a trust boundary, and treating it like trusted config is how detection tools get evaded or DoS'd.
When you open("/proc/self/status") and read(), the kernel runs a function that formats the answer right then. There is no stored file, no stable size, and stat() typically reports size 0. Two consequences follow immediately: (1) you cannot mmap most of these files or trust st_size to pre-size a buffer, and (2) the bytes can differ between reads because the process kept running. Read sequentially, from the top, into a bounded buffer.
user space kernel
---------- ------
open("/proc/4242/status") ----> look up task 4242
read(fd, buf, 4096) ----> format_status(task) -> text
<---- "Name:\tfoo\nState:\tR..."
read(fd, buf, 4096) ----> 0 (EOF)
A forensic parser must survive input it did not expect. Read one line at a time into a fixed buffer (fgets from a stream does exactly this) and decide, explicitly, what happens when a line does not fit. The safe default is to drain and discard the overflow so the next read starts on a real line boundary, and to skip the truncated line rather than parse half of it. The Cgroup line, for example, can be very long; if it overruns your buffer and you parse the fragment, you may match a field that was never really there.
sscanf is safe here only if you (a) give every %s/%[ a width that matches the destination, and (b) check the return value — the count of successful conversions. %ld that fails to match leaves the target untouched, so if you pre-seed the target with a sentinel (-1), a failed parse cannot masquerade as a real 0.
Knowledge check:
long rss = 0; sscanf(line, "VmRSS:\t%ld kB", &rss);— if the line readsVmRSS:\tnot-a-number kB, what isrssafterward, and why is that dangerous?Answer
`sscanf` fails to convert, returns `0`, and **leaves `rss` unchanged** — so `rss` is still `0`. A downstream check like "alert if RSS < 1024 kB" now fires, or worse, "skip idle processes with RSS 0" silently hides this one. Initialize to `-1` and branch on the return value: `if (sscanf(...) == 1) { got it } else { unknown }`. Never let a parse failure become a plausible-looking number.
Not every field is equally dependable. Build detections on stable fields; treat volatile ones as best-effort.
| Field | Stability | Notes |
|---|---|---|
Name, Pid, PPid, Uid, Gid |
Stable | Present for many kernel generations |
VmRSS, VmPeak, VmSize, Threads |
Stable | Memory/thread accounting, long-standing |
State |
Stable | Single letter + parenthetical text |
Cgroup, NSpgid, NStgid |
Less stable | Newer, namespace/cgroup dependent; may be absent |
SigQ, Seccomp |
Less stable | Format has shifted between versions |
| Anything with an absolute timestamp | Volatile | Changes every read; never key logic on it |
Always parse by field name, never by line number — the kernel adds and reorders lines between versions.
A PID can die between the moment you list /proc and the moment you open a file inside /proc/<pid>/. This is a time-of-check/time-of-use gap you cannot close, only handle: every open/read after the directory scan may return ENOENT (no such file) or ESRCH, and that is normal, not an error to crash on.
T0 opendir("/proc") -> sees 4242
T1 process 4242 exits <-- the race window
T2 open("/proc/4242/status") -> ENOENT <-- must be tolerated
A defender's tool that aborts here fails exactly when an attacker spawns short-lived processes on purpose. Treat ENOENT/ESRCH mid-scan as "process gone, move on," and reserve hard errors for the unexpected ones (EACCES, EIO).
Everything here is read-only introspection of the local machine — the defensive core of DFIR tooling. The one "offensive" idea, churning PIDs, is a robustness test you run against your own parser in a lab (bash -c 'while true; do :; done' &), to prove your tool degrades gracefully instead of crashing. That defensive hardening is the whole point: a monitor that survives adversarial input is worth more than one that reads pretty output.
FILE *fopen(const char *path, const char *mode);
// Returns NULL on failure (check errno). For /proc use "r". Must fclose().
char *fgets(char *buf, int size, FILE *fp);
// Reads at most size-1 bytes OR up to and including one '\n', NUL-terminates.
// Returns buf, or NULL at EOF/error. If no '\n' was stored, the line was
// longer than the buffer — the rest is still in the stream (drain it).
int sscanf(const char *str, const char *fmt, ...);
// Returns the NUMBER of input items successfully assigned (not bytes).
// A failed %ld/%c leaves its target UNCHANGED — seed targets with sentinels.
// Always give %s and %[...] a width (e.g. %63[^\n]) <= dest size minus 1.
DIR *opendir(const char *name); // for scanning /proc for PIDs
struct dirent *readdir(DIR *dirp); // d_name holds the PID as text
int closedir(DIR *dirp);
// After readdir sees a PID, any open of files under it may fail with ENOENT
// or ESRCH if the process exited — check errno and continue, don't abort.
int open(const char *path, int flags); // returns -1 + errno on failure
ssize_t read(int fd, void *buf, size_t n); // /proc files: read sequentially;
// short reads are normal, loop.
Canonical reference: man 5 proc. Must-free/close: every fopen→fclose, every opendir→closedir, every open→close.
The recipes lesson showed how to use /proc. This lesson is for engineers writing the forensic tools themselves.
When you build those tools:
/* procfs-forensics: robust parsing of /proc/<pid>/status-style text.
*
* On a real Linux box you would fopen("/proc/self/status"). To keep this
* demo hermetic (and to force the interesting error cases), we parse a
* fixed in-memory buffer that mimics the kernel's text format, including
* a malformed line and an over-long line that a naive parser would trust.
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
/* A parsed subset of /proc/<pid>/status. Sentinels mark "unknown". */
struct pstatus {
char name[64]; /* Name: process comm (may be truncated by kernel) */
long pid; /* Pid: */
long ppid; /* PPid: */
long vm_rss_kb; /* VmRSS: resident set size in kB */
long threads; /* Threads: */
char state; /* State: R/S/D/Z/T ... */
};
static void pstatus_init(struct pstatus *s) {
s->name[0] = '\0'; /* empty name == unknown */
s->pid = s->ppid = s->vm_rss_kb = s->threads = -1; /* -1 == unknown */
s->state = '?'; /* '?' == unknown */
}
/* Parse one already-bounded line into *s. Returns 1 if a field matched. */
static int parse_line(const char *line, struct pstatus *s) {
/* %63[^\n] caps the copy at the buffer size; width specifiers matter. */
if (sscanf(line, "Name:\t%63[^\n]", s->name) == 1) return 1;
if (sscanf(line, "State:\t%c", &s->state) == 1) return 1;
if (sscanf(line, "Pid:\t%ld", &s->pid) == 1) return 1;
if (sscanf(line, "PPid:\t%ld", &s->ppid) == 1) return 1;
if (sscanf(line, "VmRSS:\t%ld kB", &s->vm_rss_kb) == 1) return 1;
if (sscanf(line, "Threads:\t%ld", &s->threads) == 1) return 1;
return 0; /* unknown / uninteresting line: ignore, never abort */
}
/*
* Read from `text` one line at a time into a fixed buffer, exactly as fgets
* would from a stream. If a line is longer than the buffer, we consume and
* discard the overflow so the next iteration starts on a real line boundary.
* Returns the number of interesting fields matched.
*/
static int parse_status(const char *text, struct pstatus *out) {
char line[256]; /* fixed, bounded line buffer */
size_t pos = 0, matched = 0;
pstatus_init(out);
while (text[pos] != '\0') {
size_t n = 0;
int truncated = 0;
/* Copy up to sizeof(line)-1 chars or until newline. */
while (text[pos] != '\0' && text[pos] != '\n' && n < sizeof line - 1)
line[n++] = text[pos++];
line[n] = '\0';
/* If we stopped because the buffer filled, this line is too long. */
if (text[pos] != '\0' && text[pos] != '\n') {
truncated = 1;
while (text[pos] != '\0' && text[pos] != '\n') pos++; /* drain */
}
if (text[pos] == '\n') pos++; /* step past the newline */
if (truncated) {
fprintf(stderr, "skip: over-long line capped at %zu bytes\n",
sizeof line - 1);
continue; /* refuse to trust a line that did not fit */
}
matched += (size_t)parse_line(line, out);
}
return (int)matched;
}
int main(void) {
/* A realistic status blob: note the bogus VmRSS and the giant Cgroup. */
static char big[512];
memset(big, 'A', sizeof big - 1);
big[sizeof big - 1] = '\0';
char text[1200];
snprintf(text, sizeof text,
"Name:\tdemo-agent\n"
"State:\tR (running)\n"
"Pid:\t4242\n"
"PPid:\t1\n"
"VmRSS:\tnot-a-number kB\n" /* malformed: must NOT become 0 */
"Threads:\t7\n"
"Cgroup:\t%s\n" /* over-long line: must be skipped */
"VmRSS:\t20480 kB\n", /* the real value, later in the file */
big);
struct pstatus s;
int fields = parse_status(text, &s);
printf("parsed %d fields\n", fields);
printf("Name : %s\n", s.name[0] ? s.name : "unknown");
printf("State : %c\n", s.state);
printf("Pid : %ld\n", s.pid);
printf("PPid : %ld\n", s.ppid);
printf("Threads : %ld\n", s.threads);
if (s.vm_rss_kb < 0) printf("VmRSS : unknown\n");
else printf("VmRSS : %ld kB\n", s.vm_rss_kb);
/* A defensive tool decides based on the sentinel, not a silent 0. */
if (s.vm_rss_kb < 0)
fprintf(stderr, "note: VmRSS unknown, not treating as 0\n");
return 0;
}
struct pstatus + pstatus_init — every numeric field is seeded to -1 and state to '?'. This is the whole trick behind "default to unknown, not 0": a parse that never runs, or that fails, leaves the sentinel in place, so downstream code can tell "we didn't measure this" from "this is genuinely zero."parse_line — one sscanf per field, each keyed by the field's name ("Pid:\t%ld"), never by position. Name uses %63[^\n]: the 63 caps the copy at one less than name[64], so a long process name can't overflow. Each match is checked with == 1; a non-matching line simply returns 0 and is ignored rather than treated as an error.parse_status inner copy loop — mimics fgets: it copies bytes until a newline, end of input, or the buffer fills (n < sizeof line - 1). This is the bound that makes hostile input safe.truncated branch — if we stopped because the buffer filled (not because we hit \n), the line was too long; we set truncated, then spin pos forward to drain the rest of the oversized line so the next iteration starts cleanly on the following line. We then continue, refusing to parse a fragment. This is what stops the giant Cgroup line from corrupting our parse.if (text[pos] == '\n') pos++ — step past the delimiter so we don't reprocess it.main building text — deliberately plants three hazards: a VmRSS with a non-numeric value (must stay unknown), an over-long Cgroup line (must be skipped), and the real VmRSS further down (proving we scan the whole file, not just the first match).unknown for vm_rss_kb < 0. Because the malformed line failed and the good line matched later, the output shows 20480 kB, and the field count is 6. The over-long line produced the skip: diagnostic on stderr.long rss = 0;
sscanf(line, "VmRSS:\t%ld kB", &rss);
use(rss);
Why it breaks: A failed conversion returns 0 items and leaves rss at its initial value. Seeded to 0, a malformed or absent field silently becomes a real-looking 0 — corrupting any threshold logic.
long rss = -1;
if (sscanf(line, "VmRSS:\t%ld kB", &rss) == 1) use(rss);
else mark_unknown();
char name[64];
sscanf(line, "Name:\t%s", name);
Why it breaks: %s has no width limit, so a long process name (a process controls its own comm) overflows the 64-byte buffer — a classic stack smash on attacker-influenced input.
char name[64];
sscanf(line, "Name:\t%63[^\n]", name); /* width caps the copy */
// scan /proc, then for each pid:
FILE *f = fopen(path, "r");
fgets(line, sizeof line, f); /* segfaults: f may be NULL */
Why it breaks: Between listing /proc and opening the file the process can exit, so fopen returns NULL with errno == ENOENT. Dereferencing it crashes the monitor exactly when short-lived processes appear.
FILE *f = fopen(path, "r");
if (!f) { if (errno == ENOENT) continue; /* gone */ else report(errno); }
// assume Threads is always the 20th line
fgets over 19 lines; sscanf(line, "%ld", &threads);
Why it breaks: Kernels add and reorder /proc fields between versions; positional parsing reads the wrong line on a different kernel and returns garbage.
while (fgets(line, sizeof line, f))
if (sscanf(line, "Threads:\t%ld", &threads) == 1) break; /* by name */
ps/ls: ps -o pid,ppid,rss,nlwp,stat,comm -p <pid> should agree with your parser (nlwp = threads, rss in kB, stat = state). Divergence means a field-name or unit bug.strace -e trace=openat,read,close ./tool shows exactly which /proc files you touch, the short reads, and any ENOENT/ESRCH you must be tolerating. If you see a crash right after an openat returning -1 ENOENT, you skipped the null/errno check.valgrind --error-exitcode=1 ./tool catches an unbounded %s overflowing your line/name buffer, and reads past the end of a truncated line.unknown where it should be, not 0.bash -c 'while :; do :; done' & (a churning PID) or spawn/kill processes in a loop while your scanner runs; the tool must keep going without a single crash or leaked fd.%s/%[ is a stack overflow waiting to happen — /proc content (comm, cmdline, environ) is influenced by the target process, i.e. attacker-controlled. Always pair the destination size with a sscanf width (%63[^\n] for char[64]).fgets returns a line with no \n, the rest is still buffered; parsing the fragment can match a field that isn't really present. Drain to the next newline before continuing.unknown, not indeterminate memory. valgrind flags the difference.st_size for /proc files (usually 0); don't malloc(st_size) and read into it — you'll allocate 0 bytes and overflow. Read incrementally into a bounded buffer.fclose/close per PID leaks fds fast when you poll thousands of processes; the process eventually hits EMFILE and stops monitoring (another denial-of-defense).char line[] across threads is a genuine data race.Every serious Linux endpoint and observability tool is, underneath, a hardened /proc parser: osquery exposes /proc as SQL tables, Falco and auditd correlate process metadata for detections, htop/top/ps render status and stat, and cAdvisor/node-exporter scrape memory and thread counts for metrics. Best practice mirrors what those projects do: parse by field name, cap every buffer, treat ENOENT/ESRCH as normal during scans, default to unknown sentinels, and pin your assumptions to man 5 proc while testing across the kernel versions you actually ship on. Treat /proc as an untrusted input source and your parser as a security boundary, because in practice it is one.
VmRSS by name. Write a function that reads /proc/self/status (or the lesson's mock buffer) with a bounded line buffer and returns VmRSS in kB, or -1 if the field is absent or malformed. Verify against ps -o rss.
Full status struct. Extend it to fill a struct with Name, State, Pid, PPid, Threads, and VmRSS, every field seeded to a sentinel, each parsed by name. Print unknown for anything missing.
Scan all PIDs. Use opendir("/proc") + readdir to iterate numeric entries, read each one's comm and state, and print a table. Skip non-numeric names and tolerate ENOENT on every open.
Win the race. While your scanner runs, spawn and kill short-lived processes in a loop; make the scanner report a per-scan count of "processes that vanished mid-read" without ever crashing or leaking a descriptor.
Adversarial fixture harness. Build a test that feeds your parser hand-crafted buffers — over-long line, non-numeric number, duplicated field, missing trailing newline, empty file — and asserts the parser returns unknown (never 0 or garbage) and never overflows under valgrind.
/proc is kernel-generated text for a possibly-vanishing target: read it sequentially into a bounded buffer, never mmap it or trust st_size.%s/%[ a width; and check sscanf's return — a failed conversion leaves its target unchanged.-1/unknown), never 0, so a parse failure can't masquerade as a real value.ENOENT/ESRCH on any access after the directory scan is normal — continue, don't crash.Pid, PPid, VmRSS, Threads, State) from volatile ones, and treat /proc as an untrusted, security-relevant input boundary.