select / poll for multiplexing

Overview

A network server usually has to keep an eye on several sockets at once: a listening socket that produces new connections, plus one socket per already-connected client. You cannot predict which will have data next, and you must not block on one socket while the others sit idle with unread bytes.

I/O multiplexing is the answer. It means asking the kernel a single question — "of all these file descriptors, which ones are ready right now?" — and being woken only when at least one is. select and poll are the two classic POSIX system calls that answer that question. Both let one thread watch a whole set of descriptors and react to whichever becomes ready, so you can serve dozens or hundreds of clients from a simple single-threaded loop.

Why it matters

The naive alternatives scale badly. Blocking read on one socket stalls every other client. fork-per-connection or thread-per-connection works but costs memory and context-switching, and shared state gets harder to reason about. Multiplexing keeps everything in one thread with one clear event loop, which is easier to debug and free of data races.

This is also the foundation every higher-level async runtime is built on. libevent, libuv (the engine under Node.js), nginx, Redis, and Python's asyncio all sit on top of select/poll/epoll/kqueue. Understanding the raw system calls demystifies what those frameworks do for you, and it is exactly the layer where subtle bugs — a descriptor you forgot to re-arm, a leaked fd, a partial write — actually bite.

Core concepts

The readiness model

Multiplexing is built on a simple idea: instead of doing I/O and blocking, you first ask which descriptors could be operated on without blocking, then act only on those.

            +-------------------+
  listen fd |                   |--> new connection ready --> accept()
  client 3  |   select/poll     |--> data to read        --> read()
  client 5  |  (ask the kernel) |--> nothing              --> skip
  client 8  |                   |--> peer closed          --> read()==0, close
            +-------------------+
                     ^
                     |  blocks until >=1 fd is ready (or timeout)

The call blocks until at least one watched descriptor is ready, then hands you back which ones. You loop over the ready set, service each, and call again.

select vs poll

Knowledge check 1: You have a client connected on file descriptor 2000 and you are using select on a system where FD_SETSIZE is 1024. What happens when you FD_SET(2000, &readfds)? Answer: undefined behavior — you write past the end of the fd_set bitmap. select cannot watch descriptors at or above FD_SETSIZE; this is a core reason to prefer poll.

The rebuild trap

select treats its fd_set arguments as both input and output. On return, a set contains only the fds that are ready — every fd that was not ready has been cleared. So a working select loop keeps a master copy and copies it into a working set before every call:

fd_set master, work;
FD_ZERO(&master);
FD_SET(listen_fd, &master);
for (;;) {
    work = master;                 /* fresh copy every iteration */
    select(maxfd + 1, &work, NULL, NULL, NULL);
    /* inspect work with FD_ISSET; master is still intact */
}

Knowledge check 2: A student's select server works for the very first event on each socket, then the socket seems to "go silent." What did they most likely forget? Answer: they reused the same fd_set across iterations instead of restoring it from a master set, so after the first return every not-ready fd was cleared and never watched again.

Readiness is not completion

"Ready to read" only promises that one read will not block — it does not promise how many bytes arrive, and a ready-for-write socket may still accept fewer bytes than you offer. TCP is a byte stream, so you must be prepared for short reads and short writes and loop accordingly. A read returning 0 means the peer performed an orderly close.

Knowledge check 3: poll says a client fd has POLLIN set, but your single read(fd, buf, 4096) returns only 12 bytes even though the client sent 5000. Is this a bug? Answer: no. Readiness guarantees a non-blocking read, not a complete message. Treat the connection as a stream and keep reading (across future POLLIN events) until you have a full application-level message.

Scaling further

select and poll both cost O(n) per call: the kernel scans every descriptor you passed, every time. That is fine for tens or low hundreds of connections. For thousands, use epoll (Linux) or kqueue (BSD/macOS), which register interest once and report only the fds that changed, giving roughly O(ready) behavior.

Syntax notes

select

#include <sys/select.h>
int select(int nfds, fd_set *readfds, fd_set *writefds,
           fd_set *exceptfds, struct timeval *timeout);

• nfds: the highest fd number in any set, plus one — not a count. A frequent off-by-one source.
• readfds/writefds/exceptfds: sets to watch for readability, writability, and exceptional conditions. Any may be NULL.
• timeout: NULL waits forever; a zeroed struct timeval polls and returns immediately; otherwise it is the maximum wait.
• Return value: number of ready fds, 0 on timeout, -1 on error (check errno).
• Manipulate sets only through the macros: FD_ZERO, FD_SET, FD_CLR, FD_ISSET.

poll

#include <poll.h>
int poll(struct pollfd *fds, nfds_t nfds, int timeout);

struct pollfd {
    int   fd;        /* descriptor to watch; negative to ignore this slot */
    short events;    /* requested events, e.g. POLLIN | POLLOUT */
    short revents;   /* filled in by the kernel on return */
};

• nfds: the number of entries in the array (a real count, unlike select).
• timeout: milliseconds; -1 waits forever, 0 returns immediately.
• Return value: number of entries with a non-zero revents, 0 on timeout, -1 on error.
• Request bits (events): POLLIN (readable), POLLOUT (writable), POLLPRI (urgent).
• Result-only bits the kernel may set in revents even if you did not request them: POLLHUP (peer hung up), POLLERR (error), POLLNVAL (fd not open). Always mask revents against what you handle.

Lesson

The problem

A server often needs to watch several sockets at once. You do not know which one will have data first. You also do not want to block on a single socket while others sit idle.

Multiplexing means watching many file descriptors (fds) at the same time and reacting to whichever one becomes ready.

select

select(int nfds, fd_set *r, fd_set *w, fd_set *e, struct timeval *to)

select waits until any of several fds becomes ready, then returns.

• r, w, e: sets of fds to watch for readability, writability, and errors.
• to: an optional timeout. Pass NULL to wait forever.

poll

poll is the modern, cleaner variant. Instead of fd sets, you pass an array of struct pollfd, one entry per fd you want to watch.

Why this matters

Both select and poll let a single thread serve many connections. You do not need to fork a new process for each client.

Scaling further

For thousands of fds, the next step up is epoll (Linux) or kqueue (BSD and macOS). These scale much better than select or poll when the number of connections is large.

Code examples

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <poll.h>
#include <sys/socket.h>
#include <netinet/in.h>

#define MAX_CLIENTS 64

int main(void) {
    int listen_fd = socket(AF_INET, SOCK_STREAM, 0);
    if (listen_fd < 0) {
        perror("socket");
        return 1;
    }

    int yes = 1;
    setsockopt(listen_fd, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes));

    struct sockaddr_in addr;
    memset(&addr, 0, sizeof(addr));
    addr.sin_family = AF_INET;
    addr.sin_addr.s_addr = htonl(INADDR_ANY);
    addr.sin_port = htons(8080);

    if (bind(listen_fd, (struct sockaddr *)&addr, sizeof(addr)) < 0) {
        perror("bind");
        return 1;
    }
    if (listen(listen_fd, 16) < 0) {
        perror("listen");
        return 1;
    }

    /* fds[0] is always the listening socket; the rest are clients. */
    struct pollfd fds[MAX_CLIENTS + 1];
    memset(fds, 0, sizeof(fds));
    fds[0].fd = listen_fd;
    fds[0].events = POLLIN;
    int nfds = 1;

    for (;;) {
        int ready = poll(fds, nfds, -1);   /* -1 = wait forever */
        if (ready < 0) {
            if (errno == EINTR)
                continue;
            perror("poll");
            break;
        }

        /* New connection on the listening socket? */
        if (fds[0].revents & POLLIN) {
            int client_fd = accept(listen_fd, NULL, NULL);
            if (client_fd >= 0) {
                if (nfds <= MAX_CLIENTS) {
                    fds[nfds].fd = client_fd;
                    fds[nfds].events = POLLIN;
                    nfds++;
                } else {
                    close(client_fd);   /* table full, refuse */
                }
            }
        }

        /* Walk the client entries; index 0 is the listener. */
        for (int i = 1; i < nfds; i++) {
            if (fds[i].revents & (POLLIN | POLLHUP | POLLERR)) {
                char buf[512];
                ssize_t n = read(fds[i].fd, buf, sizeof(buf));
                if (n <= 0) {
                    close(fds[i].fd);
                    fds[i] = fds[nfds - 1];   /* swap-remove */
                    nfds--;
                    i--;                      /* re-check swapped entry */
                } else {
                    write(fds[i].fd, buf, (size_t)n);   /* echo back */
                }
            }
        }
    }

    close(listen_fd);
    return 0;
}

A single-threaded poll-based echo server. It listens on TCP port 8080, accepts many clients at once, and echoes back whatever each client sends — all in one thread, no fork, no threads. Compile with cc -std=c11 -o echo echo.c, run it, and connect from another terminal with nc localhost 8080; anything you type is echoed straight back.

Line by line

• socket(AF_INET, SOCK_STREAM, 0) creates the TCP listening socket; a negative return means failure, so we check it.
• setsockopt(..., SO_REUSEADDR, ...) lets the server rebind to port 8080 immediately after a restart instead of waiting out the kernel's TIME_WAIT period.
• memset(&addr, 0, ...) then filling sin_family/sin_addr/sin_port builds the bind address. htonl/htons convert host byte order to network byte order. INADDR_ANY binds all interfaces.
• bind then listen attach the socket to the port and mark it passive with a backlog of 16 pending connections.
• struct pollfd fds[MAX_CLIENTS + 1] is our watch table. Slot 0 is reserved for the listening socket; slots 1..nfds-1 are connected clients. memset zeroes it so unused slots are inert.
• fds[0].fd = listen_fd; fds[0].events = POLLIN; registers the listener; readability on a listening socket means a new connection is waiting to be accepted.
• poll(fds, nfds, -1) is the heart of the loop: it blocks until at least one watched fd is ready, then fills each entry's revents. -1 means wait indefinitely.
• if (ready < 0) handles errors. EINTR (interrupted by a signal) is not a real failure, so we simply retry; any other error we report and exit.
• if (fds[0].revents & POLLIN) checks the listener first. We accept the new client, and if the table has room we add it as a fresh POLLIN slot; if it is full we close the new fd rather than leak it.
• The for (int i = 1; i < nfds; i++) loop services clients. We start at 1 because slot 0 is the listener.
• fds[i].revents & (POLLIN | POLLHUP | POLLERR) catches both data to read and a disconnect/error; in all three cases the right move is to attempt a read.
• read returning <= 0 means EOF (0, orderly close) or error (-1); we close the fd and remove its slot with a swap-remove (copy the last entry into this slot, shrink nfds), then i-- so the swapped-in entry is examined this pass.
• read returning > 0 echoes the bytes back with write. (A production server would loop on write to handle short writes; here the payloads are tiny.)
• close(listen_fd) runs only if the loop breaks on a fatal poll error, tidying up before exit.

Common mistakes

• Forgetting to rebuild the fd_set on every select iteration. select overwrites the sets in place, leaving only the ready fds. Keep a master set and copy it into a working set before each call, or your sockets go silent after the first event.
• Passing a count instead of maxfd + 1 as select's first argument. nfds is the highest descriptor number plus one, not the number of descriptors. Get it wrong and some fds are never watched.
• Exceeding FD_SETSIZE. FD_SET on an fd at or above FD_SETSIZE (often 1024) writes out of bounds — undefined behavior. High fd numbers happen sooner than you think once other fds are open.
• Ignoring POLLHUP/POLLERR in poll. These appear in revents even if you never asked for them. If you only test POLLIN, a hung-up peer can spin your loop; always handle the error/hangup bits.
• Assuming readiness means a whole message arrived. A ready fd guarantees one non-blocking read, nothing more. Handle short reads (and short writes) by looping.
• Not handling EINTR. A signal can interrupt select/poll with errno == EINTR; treat that as "try again," not as a fatal error.
• Leaking descriptors. Every accepted fd you cannot track (table full) or every closed client whose slot you forget to remove is a leak that eventually hits the process fd limit.

Debugging tips

• Trace the system calls. strace -e trace=network,poll,select ./server on Linux (or dtruss/ktrace on macOS) shows exactly which fds you pass in and what comes back — the fastest way to catch a stale set or a wrong nfds.
• Print the ready set each iteration. Log poll's return value and, for each entry, fd and revents in hex. A client that never appears ready points at a registration bug; a client that is always ready usually means an unhandled POLLHUP on a closed peer.
• Watch your fd count. ls /proc/<pid>/fd | wc -l (Linux) or lsof -p <pid> reveals a descriptor leak as a number that only ever grows.
• Reproduce short reads deliberately. Send a large payload in small pieces (e.g. nc piping a slow stream) to confirm your loop reassembles messages instead of assuming one read per message.
• Check errno immediately. select/poll returning -1 is meaningless without errno; distinguish EINTR (retry) from EBADF/EINVAL (a real bug, often a closed fd still left in the set).
• Test the disconnect path. Kill a client abruptly (Ctrl-C on nc) and confirm the server removes the slot and keeps serving others.

Memory safety

• Bound your pollfd array and check before adding. In the example, nfds <= MAX_CLIENTS gates every insertion into a fixed-size array; without that guard a flood of connections writes past the end. Never index a watch table with an unchecked counter.
• Size read buffers explicitly and pass the real size. read(fd, buf, sizeof(buf)) — never a hard-coded length larger than the buffer. A ready socket can deliver up to whatever you ask for.
• Do not use bytes you did not receive. read returns a count; only buf[0 .. n-1] are valid. Echoing sizeof(buf) instead of n would leak uninitialized stack memory back to the client.
• Close every descriptor exactly once. After close(fds[i].fd) and the swap-remove, the old slot no longer refers to that fd, so it cannot be closed again — double-close can hit an unrelated fd reused by the kernel.
• select's FD_SETSIZE limit is a memory-safety issue, not just a scaling one. FD_SET beyond the bitmap size corrupts adjacent memory; prefer poll when fd values may be large.
• Beware signedness with read's return. It is a signed ssize_t; check <= 0 before casting to an unsigned size for write, or a -1 error turns into a huge length.

Real-world uses

• Single-threaded network servers. Redis famously serves a huge request rate from essentially one event-loop thread built on this model (it uses epoll/kqueue, the same idea at scale).
• Reverse proxies and web servers. nginx multiplexes thousands of connections per worker with epoll/kqueue; select/poll are the portable fallbacks.
• Async runtimes. libuv (Node.js), libevent, and Python's asyncio selector loop all wrap these calls so higher-level code can await sockets without threads.
• Watching heterogeneous descriptors together. A daemon can poll a listening socket, a Unix control socket, a signalfd/self-pipe, and a timer fd in one loop, reacting to whichever fires.
• Interactive tools. A terminal client that must react to both keyboard input (stdin) and incoming network data uses select/poll to avoid blocking on one while the other is active.
• Implementing timeouts. Because select/poll take a timeout, they double as a portable way to bound how long you wait for I/O — the basis for connection idle timeouts.

Practice tasks

Beginner

1. Add a client counter. Extend the example so that after each accept and each disconnect it prints the current number of connected clients (nfds - 1). Verify the count rises and falls as you open and close nc sessions.
2. Convert the timeout. Change the poll call from an infinite wait (-1) to a 5000 ms timeout, and print "idle" whenever poll returns 0. Confirm the message appears only when no client sends anything for five seconds.

Intermediate

3. Port the loop to select. Rewrite the same echo server using select instead of poll: maintain a master fd_set, copy it into a working set each iteration, track maxfd, and use FD_ISSET to find ready sockets. Note where the select version is more error-prone.
4. Handle short writes. Replace the single write(fds[i].fd, buf, n) with a helper that loops until all n bytes are sent, retrying on partial writes and on EINTR. Describe why a single write is not guaranteed to send everything.

Challenge

5. Add per-connection write buffering. Give each client its own outbound buffer and register POLLOUT only when that buffer is non-empty. When a client is a slow reader, queue bytes instead of blocking, and drain the queue when the socket reports writable. Explain how this prevents one slow client from stalling the whole single-threaded server. (Do not block anywhere in the loop.)

Summary

• I/O multiplexing lets one thread watch many descriptors and act on whichever becomes ready, avoiding a process or thread per client.
• select uses three fd_set bitmaps, overwrites them in place (rebuild from a master every call), takes maxfd + 1 as its first arg, and is capped by FD_SETSIZE.
• poll uses an array of struct pollfd, leaves your events intact, reports per-fd revents, and has no fixed descriptor cap — generally the better default.
• Readiness means one non-blocking operation is possible, not that a whole message arrived: handle short reads/writes, POLLHUP/POLLERR, and EINTR.
• Bound your watch table, pass real buffer sizes, and close each fd exactly once to stay memory-safe.
• Both calls are O(n) per wait; for thousands of connections move to epoll (Linux) or kqueue (BSD/macOS).

Practice with these exercises

• Count ready descriptors
• Any readable fd?