Welcome back to Raw Ptr Insights. In the previous installment we hardened the blocking relay with proper half-close semantics, socket options that make the tool usable over real networks, and a typed error hierarchy that surfaces failures rather than swallowing them. The relay now exits cleanly, behaves correctly under interactive conditions, and composes into pipelines without deadlocking. What it cannot do is accept a second connection while the first is still active — the blocking model’s structural ceiling.
That ceiling is not an arbitrary implementation detail. It is the direct consequence of how OS threads interact with blocking I/O. Every thread::spawn we issue reserves an 8 MB virtual address space region for the new thread’s stack — the Linux default set by RLIMIT_STACK. A relay handling a hundred concurrent sessions holds 800 MB of virtual address space in stack allocations alone, before a single byte of application data is considered. The threads themselves are OS scheduling entities: each context switch requires saving and restoring register state, flushing TLB entries, and potentially invalidating L1 cache lines that the previous thread was warming. Beyond a few hundred threads, context-switch pressure dominates and throughput collapses.
The standard answer to this problem — Go’s answer — is a runtime goroutine scheduler: M goroutines multiplexed onto N OS threads, with goroutines starting at roughly 8 KB of stack and growing via stack-copying when they overflow. Rust deliberately omits a runtime scheduler. Instead, async/.await compiles each async function into a state machine that implements the Future trait, storing only the local variables needed across await points rather than a full stack frame. The runtime schedules these state machines cooperatively, parking them on I/O readiness rather than blocking threads. The cost is structural: async code must never issue a blocking syscall directly, because blocking one thread blocks all tasks that thread is running.
Most async Rust introductions present async/.await through small examples — fetching URLs, running timers — that never confront the specific problem we have: two concurrent I/O streams that need to be driven simultaneously in a single task. The duplex relay is precisely the case where tokio::select! is the obvious tool and exactly where it introduces a class of bugs — cancellation unsafety — that silently drops data rather than reporting an error. Understanding why select! is the wrong choice here, and what the right choice is, is the substance of this installment.
The operational context: the async port is not a performance optimization for single-session use. Over a single connection, the blocking relay from Article 2 and the async relay here are indistinguishable. The port is the structural prerequisite for Article 4’s multi-session server. A blocking accept() that parks the calling thread cannot be composed with concurrent session handling; an async accept().await that parks a task can be called from inside a loop that spawns new tasks. We are building the machinery that Article 4 will use.
This installment covers three sections:
- Why Threads Don’t Scale: The M:N Problem — traces the concrete cost of OS thread stack allocation and context switching, compares Rust’s compiled
Futurestate machine against Go’s goroutine runtime, and establishes why the blocking model cannot survive multi-session demands. - The Tokio Runtime and Reactor — dissects what
#[tokio::main]expands to, how Tokio’sepoll-based reactor manages I/O readiness without blocking threads, and what the running system looks like from/proc. - Concurrent Duplex Without Threads:
tokio::select!and Task Splitting — presents the async duplex relay, explains why the obviousselect!approach introduces cancellation-safety failures, and delivers the correct pattern usinginto_split()andtokio::spawn.
Why Threads Don’t Scale: The M:N Problem
The Mechanism
The cost of a Rust OS thread has two components: the stack allocation and the scheduling overhead.
Stack allocation is paid at spawn time. Linux’s default thread stack size is RLIMIT_STACK — 8 MB on most distributions, configurable via ulimit -s. Rust’s thread::spawn passes this limit to pthread_create(3), which reserves a virtual address space region of that size via mmap(2). The pages are not immediately faulted into physical memory — Linux uses demand paging, so only the stack pages actually written are backed by physical frames. But the virtual address space reservation is immediate and permanent for the thread’s lifetime. On a 64-bit system with a 128 TB user virtual address space, 8 MB per thread means roughly 16 million threads before virtual address exhaustion — not a realistic limit. The practical limits are physical memory (each active stack page consumes 4 KB) and TLB pressure (the CPU’s translation lookaside buffer caches virtual-to-physical mappings; many active stack regions fragment the working set and cause TLB misses).
Scheduling overhead compounds at high concurrency. The Linux kernel uses a preemptive CFS (Completely Fair Scheduler) with O(log n) scheduling decisions. Each context switch between OS threads requires: saving the outgoing thread’s register file to its kernel stack, selecting the next thread via the scheduler, restoring the new thread’s registers, and flushing TLB entries if the threads are in different processes (unnecessary within one process, but inter-process scheduling is common). On modern hardware a context switch costs roughly 1–10 microseconds. At 1,000 threads each receiving 1,000 requests per second, the scheduler processes 1,000,000 context switches per second — 1–10 ms of CPU time per second spent purely on scheduling, before a single byte of application data is processed.
Go’s answer is M:N threading: M goroutines mapped onto N OS threads, where N equals the number of CPU cores (set by GOMAXPROCS). Goroutines start with a ~8 KB stack that grows dynamically via stack-copying when they overflow. The Go runtime’s scheduler is cooperative: goroutines yield at function calls, channel operations, and system calls. The scheduler maintains per-thread run queues and work-steals across them. This keeps OS thread count at N regardless of goroutine count.
Rust’s answer is different. There is no runtime goroutine scheduler. Instead, async/.await is a compile-time transformation. An async fn is rewritten by the compiler into a state machine that implements Future:
// NOTE: Illustrative — the actual compiler output is more complex.
// This shows the conceptual state machine for a simple async function.
// Original async function:
// async fn read_then_write(stream: &mut TcpStream) -> io::Result<()> {
// let mut buf = [0u8; 1024];
// let n = stream.read(&mut buf).await?;
// stream.write_all(&buf[..n]).await?;
// Ok(())
// }
// Compiled state machine (simplified):
enum ReadThenWriteState {
// State 0: about to call read, holding the stream and buffer
BeforeRead { stream: *mut TcpStream, buf: [u8; 1024] },
// State 1: read returned, holding result and write data
AfterRead { stream: *mut TcpStream, data: Vec<u8> },
// State 2: completed
Done,
}The state machine stores only the variables that must survive across await points. A function that awaits one read and one write stores the stream reference and the buffer — not a full 8 MB stack. The typical async task state machine is a few hundred bytes to a few kilobytes, depending on how many locals survive across awaits. A relay handling 10,000 concurrent connections might consume 10 MB for task state; the equivalent blocking implementation with 8 MB stacks would consume 80 GB.
The Security Correlation
Thread count is an observable characteristic of a running process. /proc/<pid>/status reports Threads: and /proc/<pid>/task/ lists each thread’s directory. A relay binary that spawns one thread per connection advertising its concurrency level to anyone with read access to /proc. An async relay with a fixed thread pool (Tokio’s default: one thread per CPU core) has a constant, small Threads: count regardless of session count. Predictable, bounded resource consumption is not just an efficiency property — it is a detectability property.
The Tokio Runtime and Reactor
The Mechanism
The #[tokio::main] attribute macro rewrites the main function. The expansion for the default multi-threaded runtime is equivalent to:
// NOTE: Illustrative — shows what #[tokio::main] expands to.
fn main() {
tokio::runtime::Builder::new_multi_thread()
.enable_all() // enables both I/O and time drivers
.build()
.unwrap()
.block_on(async_main()) // runs the top-level future to completion
}The Runtime has two components: the task scheduler and the I/O reactor.
The task scheduler is a work-stealing thread pool with one OS thread per CPU core by default. Each worker thread runs a loop: pull a task from the local queue, poll it until it either completes or parks on an await point, then pull the next task. If a thread’s local queue is empty, it steals tasks from other threads’ queues. This is the same strategy as Go’s goroutine scheduler, but implemented in userspace and driven by compiled Future state machines rather than a runtime.
The I/O reactor wraps epoll(7). At startup, Tokio calls epoll_create1(EPOLL_CLOEXEC) to create an epoll instance — a kernel object tracked by a file descriptor. When a Tokio TcpStream is read or written, Tokio calls epoll_ctl(2) to register the socket fd with the epoll instance, specifying interest in EPOLLIN (readable) or EPOLLOUT (writable). One worker thread runs epoll_wait(2) in a loop. When a fd becomes ready, epoll_wait returns the ready events; Tokio looks up the waiting task for that fd and places it back in the run queue to be polled.
The difference from a blocking read(2): a blocking read parks the calling OS thread in the kernel until data arrives. An async read registers the fd with epoll, parks the task (a userspace structure), and returns the thread to the scheduler immediately. The thread can then execute other tasks while waiting for the fd to become ready.
The epoll fd itself is visible from the running process:
# List file descriptors of a running relay process.
# The epoll fd appears alongside the socket fds.
ls -la /proc/$(pgrep relay)/fd/
# fdinfo for the epoll fd shows its registered fds and their events.
cat /proc/$(pgrep relay)/fdinfo/<epoll_fd_number>The output of fdinfo includes each registered fd and its epoll event mask — a live view of which fds the reactor is watching.
The async versions of the standard network types are in tokio::net. tokio::net::TcpStream and tokio::net::TcpListener wrap the underlying OS fds and register them with the reactor automatically. Their APIs mirror std::net but return Future values rather than blocking.
The Security Correlation
The epoll fd and its registered fds are visible via /proc to any process with read access to the target process’s fd directory — which on Linux requires the same UID or CAP_SYS_PTRACE. An attacker who has obtained local code execution under the same user can read /proc/<pid>/fdinfo/<epoll_fd> and enumerate every socket fd the relay is watching, along with their peer addresses from /proc/<pid>/net/tcp. The relay’s session count and peer endpoints are fully enumerable without any network interaction. A relay running as a dedicated low-privilege user with no /proc access for other users of that user limits this exposure. This is one reason containerized relay deployments restrict /proc visibility through namespacing.
Concurrent Duplex Without Threads: tokio::select! and Task Splitting
The Mechanism
The async duplex relay faces the same structural problem as the blocking version: two concurrent I/O streams that must be driven simultaneously. In the blocking version, we solved this with try_clone() and thread::spawn. In the async version, the instinctive solution is tokio::select!.
tokio::select! polls multiple futures concurrently in a single task, returning when the first branch completes and dropping the remaining futures. The drop is the operative word:
// NOTE: Illustrative — this approach has a correctness problem explained below.
use tokio::io;
use tokio::net::TcpStream;
async fn duplex_relay_naive(relay_conn: TcpStream) -> io::Result<()> {
let (mut socket_rx, mut socket_tx) = relay_conn.into_split();
let inbound = io::copy(&mut socket_rx, &mut io::stdout());
let outbound = io::copy(&mut io::stdin(), &mut socket_tx);
// select! polls both futures and returns when the first completes,
// dropping the other. The losing future and all its state are freed.
tokio::select! {
r = inbound => r?,
r = outbound => r?,
}
Ok(())
}This compiles and runs — but it silently drops data. The failure mode is subtle. io::copy internally allocates an 8 KiB buffer and uses it to shuttle bytes from source to destination. Its poll loop is: fill the buffer from the source, flush the buffer to the destination, repeat. When select! drops the losing future at an await point, any bytes that have been read into the internal buffer but not yet written to the destination are freed with the buffer. They are gone. No error is raised. The destination receives fewer bytes than the source sent. Data loss without a diagnostic is the worst class of failure — the program appears to work but produces wrong output.
This is the definition of cancellation unsafety: a future is not cancellation-safe if dropping it at any await point can cause observable data loss. io::copy is not cancellation-safe because it holds user-space buffered data across await points.
TcpStream::into_split() splits the stream into an OwnedReadHalf and an OwnedWriteHalf backed by an Arc<TcpStream> — no dup(2) is issued. Dropping OwnedWriteHalf without calling .forget() sends FIN to the peer synchronously, exactly as shutdown(Shutdown::Write) does in the blocking implementation, but without requiring an explicit call.
The correct pattern mirrors what the blocking relay did: run each direction in an independent execution context. In the async world, that context is a tokio::spawn‘d task rather than an OS thread. Spawned tasks are not cancelled when the spawning task completes — they run independently to their natural termination:
use tokio::io;
use tokio::net::{TcpListener, TcpStream};
use tokio::task;
/// Runs a bidirectional relay between `relay_conn` and process stdio.
///
/// Spawns one task for the inbound direction (socket → stdout) and drives
/// the outbound direction (stdin → socket) on the calling task. Dropping
/// the OwnedWriteHalf after the outbound copy sends FIN to the peer,
/// signaling EOF to the inbound task.
async fn duplex_relay(relay_conn: TcpStream) -> io::Result<()> {
relay_conn.set_nodelay(true)?;
// into_split() shares the underlying TcpStream via Arc — no dup(2).
// socket_tx's Drop impl sends FIN when it goes out of scope,
// equivalent to shutdown(SHUT_WR) in the blocking implementation.
let (mut socket_rx, socket_tx) = relay_conn.into_split();
// Spawn the inbound relay as an independent Tokio task.
// Unlike select!, spawn does not cancel this task when the outbound
// direction finishes — both directions run to their natural EOF.
let inbound_task = task::spawn(async move {
io::copy(&mut socket_rx, &mut io::stdout()).await
});
// Outbound relay on the current task: stdin → socket.
let outbound_result = io::copy(&mut io::stdin(), &mut socket_tx).await;
// Dropping socket_tx here sends FIN to the peer. The inbound task's
// io::copy() will then receive Ok(0) on its next read and return.
drop(socket_tx);
// The ?? desugars to: propagate a JoinError (task panicked or was
// cancelled externally) and then propagate the inner io::Result.
inbound_task.await??;
outbound_result?;
Ok(())
}
async fn connect_mode(peer_addr: &str) -> io::Result<()> {
eprintln!("[*] Connecting to {}", peer_addr);
let relay_conn = TcpStream::connect(peer_addr).await?;
eprintln!("[+] Relay established: {}", peer_addr);
duplex_relay(relay_conn).await
}
async fn listen_mode(bind_addr: &str) -> io::Result<()> {
let listener = TcpListener::bind(bind_addr).await?;
eprintln!("[*] Waiting for session on {}", bind_addr);
let (session_stream, peer_addr) = listener.accept().await?;
eprintln!("[+] Session from {}", peer_addr);
duplex_relay(session_stream).await
}
#[tokio::main]
async fn main() {
// Mode dispatch follows the same RelayMode enum from Article 2.
// Omitted here to focus on the async relay mechanism.
todo!("wire RelayMode from Article 2")
}The task::spawn call allocates a new Tokio task on the heap and places it in the runtime’s task queue. The spawned task and the calling task now run concurrently on the thread pool. The thread count remains constant (number of CPU cores); only the task count grows with session count. At 10,000 concurrent sessions, there are 10,000 active tasks and — on an 8-core machine — 8 OS threads. The task state for each duplex relay is a few hundred bytes. The total overhead: roughly 3–5 MB for 10,000 sessions, compared to 80 GB for the blocking equivalent.
Technical Reality Check
The ?? on inbound_task.await is a double-unwrap: the outer ? propagates a JoinError (the task panicked or was cancelled via task.abort()), and the inner ? propagates the io::Result the task returned. If the inbound task panicked, await?? returns the JoinError as an io::Error. If the inbound io::copy returned Err(e), the inner ? propagates e. This composition is idiomatic but requires knowing which error is which when debugging. Naming the awaited result makes the source explicit:
// More explicit unwrapping for debugging:
let inbound_join = inbound_task.await;
let inbound_io = inbound_join.map_err(|e| io::Error::new(io::ErrorKind::Other, e))?;
inbound_io?;The Security Correlation
Cancellation-safety bugs are not merely theoretical. A relay that drops data in transit produces truncated output for the operator — a command whose output is silently cut off partway through, an exfiltrated file that arrives 7 KB short, a credential that arrives without its trailing newline. These failures do not raise errors. The tool appears to succeed. The data loss is discovered later, if at all. In an adversarial context where operators are making decisions based on relay output, silent truncation is an operational failure with downstream consequences. A tool that claims to relay bytes must relay all of them. The cancellation-safe spawn approach guarantees this; the select! approach does not.
Looking Ahead
The relay is now async and executes within Tokio’s task scheduler. The duplex relay runs without OS thread overhead, and the listen_mode function is one accept().await call away from becoming an accept loop. That loop — accepting connections indefinitely, spawning a new task per session, and managing their collective state — is Article 4’s subject. We will build the accept-loop-as-task-factory, examine two approaches to shared session state (Arc<Mutex<HashMap>> versus an actor-model mpsc coordinator), implement tokio::sync::broadcast-based relay between sessions, and clean up abandoned sessions with JoinSet.
Further Reading & Deep Dives:
- For Rust’s
Futuretrait, the poll model, and howasync/.awaitdesugars to state machines, see Programming Rust (Blandy et al., Chapter 20).- For the
epoll(7)interface, readiness notification semantics, and the contrast withselect(2)andpoll(2), see The Linux Programming Interface (Kerrisk, Chapter 63).- For Tokio’s architecture — the work-stealing scheduler, the I/O driver, and the timer wheel — see the Tokio internals documentation.
- For cancellation safety in async Rust — which futures are safe to drop at any await point and which are not — see the Tokio tutorial: Select chapter.
Leave a Reply