How to pipe data between processes

Short answer: A pipeline is two child processes and one kernel pipe. Give the first child .stdout(Stdio::piped()), take the read end with .stdout.take().unwrap(), and hand it to the second child as .stdin(...) before spawning it. The pipe has a finite buffer — 65536 bytes on this machine — so a child writing more than that blocks until someone drains the pipe. That is where the classic deadlock comes from: call wait() before you have read the pipe, and the child blocks in write() while you block in wait(), forever.

Everything below runs in one terminal, in order, on macOS or Linux: the shell blocks first, then the same moves in Rust, compiled with rustc --edition 2024 (this machine pins 1.99.0). Each Rust program is one file, built and run where it sits.

What the shell builds when you type a pipe

mkdir demo && cd demo
touch a b c
ls | wc

       3       3       6

Three names, three words, six bytes. The | is not string glue. The shell asks the kernel for a pipe — a one-way byte channel with a fixed-size buffer — then starts two children and rewires their file descriptors, so everything ls writes to stdout arrives on wc’s stdin. No bytes pass through the shell itself, which is why a pipeline streams: ls can still be writing while wc is already counting.

The exit status hides the same two-process structure:

false | true; echo $?

$? is the last child’s status. false failed, and nothing reported it. In Rust you hold both children as separate values, so you can check each status yourself — and choose which child to wait for first. That choice is what the second half of this page is about.

The same pipeline in Rust

cat > pipeline.rs <<'EOF'
use std::process::{Command, Stdio};

fn main() {
    let mut ls = Command::new("ls")
        .stdout(Stdio::piped())
        .spawn()
        .expect("failed to spawn ls");

    let mut wc = Command::new("wc")
        .stdin(ls.stdout.take().expect("no piped stdout"))
        .spawn()
        .expect("failed to spawn wc");

    let wc_status = wc.wait().expect("failed to wait for wc");
    let ls_status = ls.wait().expect("failed to wait for ls");
    println!("wc: {}", wc_status);
    println!("ls: {}", ls_status);
}
EOF
rustc --edition 2024 pipeline.rs && ./pipeline

       5       5      27
wc: exit status: 0
ls: exit status: 0

Five entries this time — the directory has gained pipeline.rs and the pipeline binary since the first block — and both statuses come back, where the shell’s $? would have shown only wc’s. The wiring has three moves:

.stdout(Stdio::piped()) on the upstream child. Without it, ls writes straight to your terminal and there is no read end to hand anyone.
ls.stdout.take() on the spawned child. The field is an Option<ChildStdout>: None unless you asked for a pipe. take() empties the option, because a read end belongs to exactly one consumer — after the move, ls.stdout is None.
.stdin(that) on the downstream child, before it spawns. The child’s stdin is now the kernel pipe instead of your keyboard.

Nothing here can deadlock yet, even though wc writes nothing and ls waits for no one: wc is a live process draining the pipe as fast as ls fills it. The deadlock needs a pipe that nobody drains — which is exactly what .wait() without reading builds.

The deadlock: wait before you read

One change makes the hang: pipe the child’s stdout, then wait for the child without ever reading the pipe. This program takes the number of bytes the child writes as its argument — the code is identical for both runs:

cat > maybe-deadlock.rs <<'EOF'
use std::env;
use std::process::{Command, Stdio};

fn main() {
    let bytes = env::args().nth(1).expect("usage: maybe-deadlock BYTES");
    let mut child = Command::new("sh")
        .args(["-c", &format!("dd if=/dev/zero bs={bytes} count=1 2>/dev/null")])
        .stdout(Stdio::piped())
        .spawn()
        .expect("failed to spawn dd");

    let status = child.wait().expect("failed to wait");
    println!("child exited: {}", status);
}
EOF
rustc --edition 2024 maybe-deadlock.rs
perl -e 'alarm 3; exec @ARGV' ./maybe-deadlock 65536 || echo "killed after 3 seconds"
perl -e 'alarm 3; exec @ARGV' ./maybe-deadlock 65537 || echo "killed after 3 seconds"

child exited: exit status: 0
killed after 3 seconds

The parent holds the read end of the pipe and never reads it. With 65536 bytes, the child’s writes all fit in the pipe buffer — 65536 bytes on this machine; Linux’s default pipe buffer is the same 64KiB — so the child writes everything, exits, and wait() returns. With 65537, the write does not fit: the kernel accepts what the buffer holds and blocks the child inside write() until space frees up. No space ever frees, because the only reader is the parent, and the parent is stuck in wait(). Neither process can move, so after three seconds the watchdog killed the second run — perl -e 'alarm 3; exec @ARGV' is just a way to say “run this, and send it SIGALRM if it is still alive after 3 seconds”, which is the default timeout shape on a Mac, where the timeout command does not ship.

This is also why the bug survives testing so easily. Every test fixture fits in the buffer — even the “works with 65536” run above is the trap, not the exception — and the first real input, one byte past whatever the buffer happens to be, hangs the program.

The fix: drain the pipe, then wait

Read the pipe to EOF first, wait second. EOF is the signal that the write end is closed: the child has produced everything it ever will, so the wait() that follows cannot block on data.

cat > drain.rs <<'EOF'
use std::io::Read;
use std::process::{Command, Stdio};

fn main() {
    let mut child = Command::new("dd")
        .args(["if=/dev/zero", "bs=65537", "count=1"])
        .stdout(Stdio::piped())
        .stderr(Stdio::null())
        .spawn()
        .expect("failed to spawn dd");

    let mut stdout = child.stdout.take().expect("no piped stdout");
    let mut buf = Vec::new();
    stdout.read_to_end(&mut buf).expect("failed to read");

    let status = child.wait().expect("failed to wait");
    println!("read {} bytes, then: {}", buf.len(), status);
}
EOF
rustc --edition 2024 drain.rs && ./drain

read 65537 bytes, then: exit status: 0

The same 65537 that deadlocked a paragraph ago, now finishing normally. The order is the whole fix: read_to_end runs while the child writes and returns at EOF, and only then does wait() run — by which time there is nothing left to wait on but the child’s exit.

Two related traps, same mechanism:

Two pipes read in order deadlock the same way. Pipe both stdout and stderr, read stdout to EOF, then start on stderr — if the child filled the stderr buffer along the way, it blocked in write() before it ever got to the stdout bytes you were waiting to see. Drain concurrent streams concurrently: one thread per pipe, or let output() do it.
output() never has this problem. The output article’s one-call API drains both streams while the child runs, which is why its examples all terminate. The deadlock on this page needs Stdio::piped() and a read order you wrote yourself.

How to run an external command and read its output — .output() drains both streams for you, so the deadlock above cannot happen there
How to check whether a command exists — the error a missing command gives before any pipe exists to deadlock
Processes, environment and signals — the rest of the chapter

How to pipe data between processes

What the shell builds when you type a pipe

The same pipeline in Rust

The deadlock: wait before you read

The fix: drain the pipe, then wait

Related