Networking in C · intermediate · ~20 min

Non-blocking sockets — O_NONBLOCK and EAGAIN

Read and write without blocking, and survive the EAGAIN cycle.

Overview

Setting O_NONBLOCK on a socket changes how read and write behave. Instead of waiting (blocking) when an operation cannot complete right away, they return immediately with the error EAGAIN (or EWOULDBLOCK).

This flag pairs naturally with epoll in edge-triggered mode:

Drain all data that is currently available.
Return to the event loop.
Wait until the kernel signals that more data has arrived.

Why it matters

A single thread can serve thousands of slow clients only if no operation ever blocks.

Non-blocking sockets are the precondition. Epoll is the dispatcher that tells you which socket is ready.

Core concepts

Making a socket non-blocking

Two ways:

Call fcntl(fd, F_SETFL, O_NONBLOCK) on an existing socket.
Pass SOCK_NONBLOCK when you create the socket.

EAGAIN and EWOULDBLOCK

These errors mean "no data right now, try again later" — not a real failure.

POSIX allows them to be two different values, so always check both. On Linux they happen to be identical.

Partial writes

A non-blocking write may write fewer bytes than you asked for.

Loop to send the rest.
If write returns -1 with EAGAIN, the kernel buffer is full. Queue the leftover bytes and ask epoll to notify you with EPOLLOUT (socket writable again).

Pentester mindset

Slowloris-style attacks send one byte every 30 seconds to tie up server resources.

A blocking server wastes one worker per attacker.
A non-blocking server with read-timeouts shrugs the attack off.

Defensive coding habits

ALWAYS handle EAGAIN before treating a -1 return as a real error.
ALWAYS drain a socket until you hit EAGAIN when using edge-triggered mode.

Syntax notes

Read the current flags, then add O_NONBLOCK without clearing the others:

int flags = fcntl(sock, F_GETFL);
fcntl(sock, F_SETFL, flags | O_NONBLOCK);

Lesson

A non-blocking socket returns from read or write immediately. You get one of two results:

Data (or bytes written).
EAGAIN / EWOULDBLOCK, meaning "try again later."

Combined with epoll, this is how you serve thousands of slow clients on a single thread.

Code examples

fcntl(sock, F_SETFL, O_NONBLOCK);
ssize_t n = read(sock, buf, sizeof buf);
if (n < 0) {
    if (errno == EAGAIN || errno == EWOULDBLOCK) {
        /* No data; come back when epoll says so. */
    }
}

Line by line

for (;;) {
    ssize_t n = read(fd, buf, sizeof buf);
    if (n > 0) { handle(buf, n); continue; }
    if (n == 0) { close(fd); break; }              /* EOF */
    if (errno == EINTR) continue;                  /* signal — retry */
    if (errno == EAGAIN || errno == EWOULDBLOCK)   /* drained — wait for epoll */
        break;
    perror("read"); close(fd); break;              /* real error */
}

Common mistakes

Treating EAGAIN as a real error instead of "try again later."
Forgetting EWOULDBLOCK — on some systems it is a distinct errno value.

Debugging tips

Use strace to watch the actual system calls:

strace -e read,write ./prog

This shows every short read and every EAGAIN.

If your server hangs, look for a blocking call you forgot to mark as non-blocking.

Memory safety

Always buffer the unsent tail of a partial write.

A common bug: you call write for 100 bytes, get a return value of 70, and forget the remaining 30. Those bytes are lost unless you keep them and send them later.

Real-world uses

Every high-performance server uses this pattern. The Linux kernel's networking stack is designed around it and expects user-space code to follow it.