open / read / write / close

Overview

In the previous lesson, File descriptors, you learned that the kernel hands your program a small non-negative integer — a file descriptor (fd) — to represent something it has opened on your behalf. By convention 0 is standard input, 1 is standard output, and 2 is standard error. This lesson is about the four system calls that let you create and use those descriptors yourself: open, read, write, and close.

These four calls are the bedrock of file I/O on Linux and other POSIX systems. The friendly C standard-library functions you may already know — fopen, fread, fwrite, fclose — are built on top of these syscalls. The stdio layer adds buffering and a FILE * wrapper for convenience. When you go one level down to open/read/write/close, you trade that convenience for direct, predictable control over exactly what the kernel does. That control is what device drivers, network servers, shells, and database engines need.

The single most important idea in this lesson is that read and write are requests, not guarantees. You ask to move up to N bytes; the kernel tells you how many it actually moved. A correct program never assumes the number it asked for is the number it got. Once that idea clicks, raw I/O stops being scary and becomes a small, dependable toolkit.

Terminology to keep straight as we go: a system call (syscall) is a request from your program into the kernel; a flag is a bit you OR together to describe how to open a file; a mode is the permission bits a newly created file should get; EINTR is the error code meaning "a signal interrupted me before I finished — try again."

Why it matters

Every byte that leaves or enters a running Linux program passes through these calls or something layered on them. A web server reading a request, a logger appending a line, cp copying a file, a backup tool streaming gigabytes — all of it is read and write under the hood.

Understanding the raw layer matters for three concrete reasons:

• Correctness. Code that assumes write always writes everything you gave it will silently corrupt data the first time it talks to a slow disk, a full pipe, or a network socket. The bug is invisible in testing and catastrophic in production.
• Performance and control. stdio buffering is great for line-oriented text, but it gets in the way when you need exact control over when bytes hit the device, when you are copying large blocks, or when you are talking to a pipe or socket rather than a regular file.
• Robustness under signals. Long-running programs receive signals. A syscall interrupted mid-flight returns -1 with errno == EINTR. Programs that do not handle this crash or behave erratically when, for example, the terminal is resized or a timer fires.

These are exactly the failure modes that distinguish toy code from software people can depend on.

Core concepts

1. open — turning a path into a file descriptor

open asks the kernel to set up access to a file named by a path and return a fresh file descriptor pointing at it.

int fd = open(path, flags, mode);

• path — the filename (relative to the current directory, or absolute).
• flags — an integer built by OR-ing together one access mode and any number of behavior flags.
• mode — the permission bits for a file you are creating; only used when O_CREAT is in the flags, ignored otherwise.

Exactly one access mode is required: O_RDONLY (read only), O_WRONLY (write only), or O_RDWR (both). Common behavior flags you OR on top:

Internally, the kernel walks the path, checks permissions, finds the smallest unused descriptor number in your process, links it to an open file description (which holds the current offset and the flags), and returns that number. On failure it returns -1 and sets errno.

When to use which mode: A new file you create gets mode like 0644 (owner read/write, others read). The kernel then masks it with the process umask, so the file may end up with fewer bits than you asked for — that is expected.

Pitfall: passing a mode but forgetting O_CREAT does nothing useful (mode is ignored), and passing O_CREAT without a mode leaves the new file's permissions as garbage. Always pair them.

open("data.txt", O_RDONLY)
        |
        v
  kernel: resolve path, check perms
        |
        v  smallest free fd = 3
  +-----------------------------+
  | process fd table            |
  |  0 -> stdin                 |
  |  1 -> stdout                |
  |  2 -> stderr                |
  |  3 -> open file description -> [offset=0, flags] -> inode(data.txt)
  +-----------------------------+
        |
        v
  returns 3

Knowledge check (concept): You call open("log.txt", O_WRONLY | O_CREAT | O_APPEND, 0644). What are two distinct behaviors this combination of flags produces?

2. read — pulling bytes in

ssize_t n = read(fd, buf, count);

read copies up to count bytes from the file (starting at the current offset) into buf, advances the offset, and returns how many bytes it actually copied. The three outcomes:

• > 0 — that many bytes were placed in buf (may be fewer than count).
• 0 — end of file. There is nothing more to read.
• -1 — an error occurred; check errno.

A return smaller than count is a short read and is completely normal — especially from pipes, sockets, and terminals, but possible from regular files near EOF too. It is not an error.

When NOT to use a single read: never assume one read gives you a whole "message" or the whole file. If you need an exact number of bytes, loop.

Pitfall: read does not add a '\0' terminator. If you plan to treat buf as a C string, you must place the terminator yourself at offset n, and your buffer must be at least count + 1 bytes.

Knowledge check (find-the-bug): char buf[64]; ssize_t n = read(fd, buf, 64); printf("%s\n", buf); — what is wrong, and what is the smallest fix?

3. write — pushing bytes out

ssize_t n = write(fd, buf, count);

write copies up to count bytes from buf to the file and returns how many it actually wrote. A return less than count is a short write. This is the failure mode the existing quiz asks about: a pipe can be full, a signal can interrupt mid-write, the disk can fill, or the kernel may simply choose to accept fewer bytes. The remaining bytes are your responsibility — you must call write again for the rest.

This is why every robust codebase has a write_all helper that loops until all bytes are sent or a real error occurs.

write_all(fd, buf, 1000)
  buf: [################################] 1000 bytes

  write -> returns 600   (short write)
  p += 600; left = 400
  buf:                  [############] 400 left

  write -> returns 400
  left = 0  -> done

When NOT to use a single write: any time correctness depends on all the bytes landing — which is almost always.

Pitfall: treating the short-write return as an error and aborting. It is not an error; it is a request to continue.

Knowledge check (predict-the-output): A program does write(STDOUT_FILENO, "hi", 2) and ignores the return value. On a regular terminal this usually prints hi. Name one situation where ignoring the return value would silently lose part of the output.

4. close — releasing the descriptor

int rc = close(fd);

close tells the kernel you are done with the descriptor. The number becomes free for reuse, and when the last descriptor referring to a file is closed, kernel resources for it are released. close returns 0 on success and -1 on error.

Why check close's return value? On some filesystems the final flush of data happens at close time, so a deferred I/O error (like a full disk) is reported here, not at write. Ignoring it can mean believing data was saved when it was not.

Pitfall: "double close." Closing the same fd twice is a bug; in a threaded program the number may have been reused by another thread in between, and your second close then closes someone else's file. Set the variable to -1 after closing if there is any chance of reuse.

Lifetime of a descriptor:

  open ----> [ fd in use: read / write / read ... ] ----> close
   3                                                        3 freed

Knowledge check (explain-in-your-own-words): Why might a disk-full error show up at close() rather than at the write() that produced the data?

Syntax notes

All four calls live in standard headers. You need <fcntl.h> for open and the O_* flags, <unistd.h> for read/write/close, and <errno.h> to read errno. The return types ssize_t (signed size, can be -1) and size_t come from <sys/types.h> (pulled in by the others).

#include <fcntl.h>      // open, O_RDONLY, O_WRONLY, O_CREAT, O_TRUNC, ...
#include <unistd.h>     // read, write, close, ssize_t
#include <errno.h>      // errno, EINTR
#include <string.h>     // strerror (for messages)

// Access mode is REQUIRED; behavior flags are OR-ed on:
int in  = open("in.txt",  O_RDONLY);                       // no mode: not creating
int out = open("out.txt", O_WRONLY | O_CREAT | O_TRUNC,    // creating -> mode needed
               0644);                                      // rw-r--r-- before umask

char buf[4096];
ssize_t n = read(in, buf, sizeof buf);   // up to 4096 bytes; n may be < 4096
if (n < 0) { /* error: inspect errno */ }

ssize_t w = write(out, buf, (size_t)n);  // returns bytes actually written

if (close(in)  == -1) { /* report */ }
if (close(out) == -1) { /* report */ }   // close can surface a deferred error

Key rules encoded above: pick exactly one of O_RDONLY/O_WRONLY/O_RDWR; add mode only with O_CREAT; store results in a signed type because -1 is a valid return; and always pass a count, never assume the call honors it fully.

Lesson

The four core syscalls

These four functions are the foundation of file I/O on Linux. Each works with a file descriptor (fd) — a small integer the kernel uses to identify an open file.

• open(path, flags, [mode]) — opens a file. Returns a file descriptor on success, or -1 on failure.
• read(fd, buf, n) — reads up to n bytes into buf. Returns the number of bytes actually read.
• write(fd, buf, n) — writes up to n bytes from buf. Returns the number of bytes actually written.
• close(fd) — releases the file descriptor when you are done.

Short reads and writes

read and write may transfer fewer bytes than you asked for. This is called a short read or short write, and it is normal — not an error.

For example, write(fd, buf, 1000) might return 600, meaning only 600 bytes were written. The remaining 400 still need to be sent.

Three rules to always follow

• Loop short writes. Keep calling write until every byte is sent.
• Check return values. A return of -1 signals an error.
• Handle EINTR. This error code means a signal interrupted the syscall before it finished. The correct response is simply to retry the call.

Code examples

#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>

/* Write exactly n bytes, looping over short writes and retrying on EINTR.
   Returns 0 on success, -1 on a real error (errno set). */
static int write_all(int fd, const void *buf, size_t n) {
    const char *p = buf;
    size_t left = n;
    while (left > 0) {
        ssize_t w = write(fd, p, left);
        if (w < 0) {
            if (errno == EINTR) continue;   /* interrupted by a signal: retry */
            return -1;                      /* a genuine error */
        }
        p    += w;                          /* advance past what was written */
        left -= (size_t)w;
    }
    return 0;
}

/* Copy src -> dst using only raw syscalls. Returns 0 on success, -1 on error. */
static int copy_file(const char *src, const char *dst) {
    int in = open(src, O_RDONLY);
    if (in < 0) { perror("open src"); return -1; }

    /* Create dst (or truncate it), owner rw, group/other read. */
    int out = open(dst, O_WRONLY | O_CREAT | O_TRUNC, 0644);
    if (out < 0) { perror("open dst"); close(in); return -1; }

    char buf[4096];
    ssize_t r;
    int status = 0;

    /* read returns 0 at EOF, which ends the loop cleanly. */
    while ((r = read(in, buf, sizeof buf)) != 0) {
        if (r < 0) {
            if (errno == EINTR) continue;   /* retry the read */
            perror("read");
            status = -1;
            break;
        }
        if (write_all(out, buf, (size_t)r) < 0) {
            perror("write");
            status = -1;
            break;
        }
    }

    /* Always close both. close can report a deferred write error on dst. */
    if (close(in)  < 0) { perror("close src"); status = -1; }
    if (close(out) < 0) { perror("close dst"); status = -1; }
    return status;
}

int main(int argc, char **argv) {
    if (argc != 3) {
        fprintf(stderr, "usage: %s <src> <dst>\n", argv[0]);
        return 1;
    }
    if (copy_file(argv[1], argv[2]) < 0) return 1;
    printf("copied %s -> %s\n", argv[1], argv[2]);
    return 0;
}

What it does: This is a complete, minimal cp built only from open, read, write, and close. It opens the source read-only, creates/truncates the destination, then repeatedly reads a 4 KB block and writes it out with write_all. The read loop ends naturally when read returns 0 (EOF). Both descriptors are closed at the end, and close errors are reported.

Expected output (after building and running on a file that exists):

$ ./mycp notes.txt copy.txt
copied notes.txt -> copy.txt

If the source does not exist, you get a diagnostic from perror such as open src: No such file or directory and exit status 1.

Key edge cases: an empty source file produces an empty destination (the loop body never runs because read returns 0 immediately); a read interrupted by a signal is retried via EINTR; a short write inside write_all is handled by looping; and a disk-full condition may surface either at write or at close(out) — both are checked.

Line by line

We trace copy_file("notes.txt", "copy.txt") where notes.txt contains the 5 bytes Hello.

1. open(src, O_RDONLY) resolves notes.txt, checks read permission, and returns the smallest free descriptor — say 3. in == 3.
2. open(dst, O_WRONLY | O_CREAT | O_TRUNC, 0644) creates copy.txt (length 0) with permission bits 0644 masked by your umask, and returns the next free descriptor — say 4. out == 4.
3. We enter the loop. read(3, buf, 4096) copies all 5 available bytes into buf and returns 5. The file offset for fd 3 is now 5.
4. r == 5, which is > 0, so we call write_all(4, buf, 5).
5. Inside write_all: left = 5. write(4, p, 5) returns, say, 5. p += 5, left = 0, loop ends, return 0.
6. Back in the loop: read(3, buf, 4096) is called again. The offset is already at end of file, so it returns 0 (EOF). The while condition r != 0 is false, so the loop exits.
7. close(3) releases the source descriptor and returns 0. close(4) flushes and releases the destination, returns 0. status stays 0.
8. copy_file returns 0; main prints copied notes.txt -> copy.txt and exits 0.

The crucial detail: the program never assumes read returned the file size or that write wrote everything. It reacts to the actual returned counts. Swap in a 10 MB file and the same code runs the read/write block thousands of times with no changes — that is the payoff of looping correctly.

Common mistakes

Mistake 1 — Assuming write writes everything

// WRONG: ignores the return value
write(fd, buf, n);   // what if only half went out?

Why it is wrong: write may perform a short write (full pipe, signal, disk pressure). The unwritten bytes are silently dropped from the file. The bug rarely shows on small writes to a regular file, so it survives testing and corrupts data later.

// CORRECT: loop until all bytes are written
if (write_all(fd, buf, n) < 0) { perror("write"); /* handle */ }

Recognize it: any bare write(...) whose return value is unused is suspect. Prevent it by always routing writes through a write_all helper.

Mistake 2 — Treating a short read or EOF as an error

// WRONG: aborts on a perfectly normal partial read
ssize_t r = read(fd, buf, count);
if (r != count) { fprintf(stderr, "read failed\n"); exit(1); }

Why it is wrong: r < count is normal, and r == 0 means clean EOF, not failure. This code rejects valid data.

// CORRECT: only -1 is an error; 0 is EOF; >0 is data
ssize_t r = read(fd, buf, count);
if (r < 0)        { /* real error */ }
else if (r == 0)  { /* EOF: stop */ }
else              { /* use r bytes */ }

Mistake 3 — Forgetting the mode with O_CREAT

// WRONG: O_CREAT but no mode argument -> file gets unpredictable permissions
int fd = open("out.txt", O_WRONLY | O_CREAT | O_TRUNC);

Why it is wrong: when O_CREAT is present the kernel reads a third argument for the permission bits; omitting it leaves whatever garbage was on the stack/register as the mode. The file may end up unreadable or world-writable.

// CORRECT: supply mode whenever O_CREAT is used
int fd = open("out.txt", O_WRONLY | O_CREAT | O_TRUNC, 0644);

Mistake 4 — Leaking descriptors on an early return

// WRONG: returns without closing 'in' when the second open fails
int in = open(src, O_RDONLY);
int out = open(dst, O_WRONLY | O_CREAT | O_TRUNC, 0644);
if (out < 0) return -1;        // 'in' is now leaked

Why it is wrong: each leaked descriptor consumes a slot; a long-running program eventually hits its per-process limit and every later open fails with EMFILE.

// CORRECT: clean up what you already acquired
int in = open(src, O_RDONLY);
if (in < 0) return -1;
int out = open(dst, O_WRONLY | O_CREAT | O_TRUNC, 0644);
if (out < 0) { close(in); return -1; }

Mistake 5 — Using a read result as a C string without terminating

// WRONG: buf is not NUL-terminated
ssize_t n = read(fd, buf, sizeof buf);
printf("%s\n", buf);    // reads past the data

// CORRECT: leave room for '\0' and terminate at n
char buf[4097];
ssize_t n = read(fd, buf, 4096);
if (n >= 0) { buf[n] = '\0'; printf("%s\n", buf); }

Debugging tips

Compiler errors

• implicit declaration of function 'open' / 'read': you forgot #include <fcntl.h> (for open) or #include <unistd.h> (for read/write/close). Add the headers.
• 'O_RDONLY' undeclared: same cause — <fcntl.h> defines the O_* flags.
• comparison is always true/false or sign warnings: you stored the result of read/write in an int or unsigned type. Use ssize_t so -1 is representable and comparisons work.

Runtime errors (check errno)

Always print the reason with perror("context") or strerror(errno). Common values:

Logic errors

• Output truncated or doubled: you ignored a short write, or you wrote the whole buffer (sizeof buf) instead of just the r bytes that read returned. Always write exactly r bytes.
• Program hangs: you are reading from a pipe/terminal that has no more data but has not signaled EOF. A single blocking read waits; that is expected for interactive input.
• Garbage at the end of printed text: missing NUL terminator after read (see Mistake 5).

Questions to ask when it does not work

1. Did I check every return value for -1?
2. Am I writing exactly the bytes read returned, or am I assuming a count?
3. Did I forget mode with O_CREAT?
4. Did I close every descriptor on every path, including error paths?
5. What does errno/perror actually say? (Read the message before guessing.)

Tools: strace ./prog shows every syscall with its arguments and return value — invaluable for seeing the exact open/read/write/close sequence and where it failed.

Memory safety

These syscalls move raw bytes, so the safety burden is on you:

• Buffer bounds. Never pass a count to read/write larger than the buffer. Use sizeof buf for stack arrays (it counts bytes for char arrays) and track the allocated size for heap buffers. A count bigger than the buffer is a classic overflow.
• No automatic NUL termination. read fills only data bytes; it does not append '\0'. Treating the buffer as a C string without terminating it at the returned length reads out of bounds. Size the buffer as N + 1 and set buf[n] = '\0' when you need a string.
• Initialization. Only the first n bytes of buf are valid after read returns n. Reading or printing beyond index n-1 accesses uninitialized memory — undefined behavior.
• Signed vs unsigned overflow. read/write return ssize_t (signed). Storing the result in size_t (unsigned) turns the -1 error into a huge positive number, and your loop arithmetic (left -= w) can underflow into a giant value. Keep the return in ssize_t and only cast to size_t after confirming it is >= 0.
• Descriptor lifetime. Using an fd after close (use-after-close) or closing it twice (double-close) is undefined and, in threaded code, can corrupt unrelated files because the number may have been reused. Set fd = -1 after closing if reuse is possible.
• Robustness habit. Validate the path/size inputs you receive, loop short transfers, and check close so a deferred ENOSPC is not mistaken for success.

Real-world uses

Where these calls actually run:

• Command-line tools. cp, cat, dd, and tee are essentially the read/write loop you built above, plus options.
• Servers and networking. On Linux, sockets are file descriptors too, so a web server reads requests and writes responses with the very same read/write calls — and short reads/writes are the norm on sockets, making read_all/write_all mandatory.
• Databases and logs. Engines like SQLite and append-only log writers use raw descriptors (often with O_APPEND) to control exactly when bytes are durable.
• Embedded and drivers. Device files under /dev are read and written with these same four calls; talking to a sensor or serial port is open("/dev/...") then read/write.

Professional best practices

Beginner rules:

• Check every return value; -1 means stop and inspect errno.
• Loop short reads and short writes; never assume the full count.
• Pair O_CREAT with an explicit mode.
• close everything you open, on every code path, and check the result.

Advanced habits:

• Centralize I/O in read_all/write_all helpers so the looping and EINTR logic lives in exactly one place.
• Choose buffer sizes deliberately (a multiple of the filesystem block size, e.g. 4 KB–64 KB) to balance syscall overhead against memory.
• When durability matters, call fsync(fd) before close so data reaches the physical device, and handle the error it can report.
• Open with the least access you need (O_RDONLY if you only read), and use O_EXCL with O_CREAT when you must not clobber an existing file — both reduce the blast radius of bugs.

Practice tasks

Beginner 1 — Print a file with raw I/O

Objective: Re-implement a tiny cat. Requirements: Open the file given as argv[1] read-only; in a loop, read into a 1024-byte buffer and write exactly the returned number of bytes to STDOUT_FILENO; stop at EOF; close the file. I/O example: ./mycat hello.txt prints the file's contents. Constraints: raw syscalls only — no fopen/printf for the file data. Hints: the read loop ends when read returns 0; write r bytes, not sizeof buf. Concepts: open, read, write, EOF handling.

Beginner 2 — Append a line to a log

Objective: Append a timestamped-style message to a log file. Requirements: open the path with O_WRONLY | O_CREAT | O_APPEND and mode 0644; build a line in a buffer ending with '\n'; write all of it; close and check the result. Constraints: must not truncate the file on each run (no O_TRUNC). Hints: verify by running twice and confirming both lines are present. Concepts: flags (O_APPEND vs O_TRUNC), mode with O_CREAT, checking close.

Intermediate 1 — A correct write_all

Objective: Implement int write_all(int fd, const void *buf, size_t n) returning 0 on success, -1 on error. Requirements: loop until all n bytes are written; on -1 with errno == EINTR, retry; on any other -1, return -1. Constraints: must work even if write returns a small count each time. Hints: advance a const char * pointer and decrement a size_t left; never lose track of how many bytes remain. Concepts: short writes, EINTR, pointer arithmetic.

Intermediate 2 — read_exactly

Objective: Implement ssize_t read_exactly(int fd, void *buf, size_t n) that reads exactly n bytes, returning n, or fewer than n only if EOF is hit first, or -1 on error. Requirements: loop calling read; on 0 (EOF) stop and return how many you got; retry EINTR. I/O example: asking for 100 bytes from a 40-byte file returns 40. Constraints: no over-read past the buffer. Hints: distinguish the three read outcomes carefully. Concepts: short reads, EOF, loop termination.

Challenge — Safe atomic-ish file replace

Objective: Write int safe_replace(const char *path, const void *data, size_t n) that writes data to path without leaving a half-written file if something fails midway. Requirements: open a temporary file in the same directory with O_WRONLY | O_CREAT | O_EXCL; write_all the data; fsync; close; then rename(tmp, path) to swap it into place atomically. On any error, close and unlink the temp file and return -1. Constraints: handle every error path with full cleanup; no descriptor or temp-file leaks. Hints: rename within the same filesystem is atomic, which is why the temp file must be in the same directory; O_EXCL prevents clobbering an unexpected leftover. Concepts: O_CREAT|O_EXCL, fsync, error-path cleanup, atomic replace pattern.

Summary

Key takeaways

• open, read, write, and close are the four core POSIX I/O system calls. open turns a path into a file descriptor; read/write move bytes using that descriptor; close releases it.
• open(path, flags, mode): pick exactly one access mode (O_RDONLY/O_WRONLY/O_RDWR), OR on behavior flags (O_CREAT, O_TRUNC, O_APPEND, O_EXCL), and supply mode (e.g. 0644) only when O_CREAT is present.
• read and write are requests, not guarantees. read returns >0 (bytes read), 0 (EOF), or -1 (error); write may write fewer bytes than asked (short write). Always loop until the work is done.
• Handle EINTR by retrying the call. Store results in ssize_t so -1 is representable. read does not NUL-terminate.
• The recurring mistakes: ignoring write's return value, treating a short read or EOF as an error, forgetting mode with O_CREAT, leaking descriptors on error paths, and printing an unterminated buffer.
• Remember the pattern: check every return, loop short transfers, retry EINTR, and close (and check close) everything you open. Centralize it in read_all/write_all helpers and the rest of your I/O code stays simple and correct.

Practice with these exercises

• Read with raw descriptors
• Write with raw descriptors