The command line looks almost honest. You type ls | wc -l, press Enter, and a moment later get a number. It seems that the shell simply started two programs and passed the output of one into the input of the other.
But the kernel has no operation called "execute a pipeline."
There are three processes, two ends of a pipe, and several file descriptors that must reach the correct programs. First, the shell creates a pipe and then copies itself twice. In one copy, standard output becomes the write end; in the other, standard input becomes the read end. After that, both copies stop being the shell and turn into ls and wc.
The parent process does not read the directory or count lines. It only assembles this structure, closes unnecessary descriptors, and waits.
Closing them is not ceremonial cleanup. Leaving just one extra end of the pipe open in the parent is enough for wc to read all the data but never see the end of its input. Both programs will do their work correctly, yet the pipeline will still hang.
The same mechanism explains other command-line behavior. cd cannot be moved into a separate program because a child process cannot change its parent's working directory. Stream redirection survives the launch of a new program because exec preserves open descriptors. Ctrl-C reaches the entire pipeline because the terminal sends a signal to a process group rather than one selected PID.
To see how this works without abstract diagrams, we will write a small shell and inspect its system calls with strace. We will first run one command, then connect two programs with a pipe, and finally omit one close deliberately. The complete shell source is included in the appendix.
One Call Does Not Start a Program
Let us begin without a pipeline:
echo helloA minimal external command launch looks like this:
fn run_simple(argv: &[String]) -> Result<i32> {
let pid = fork()?;
if pid == 0 {
// The child: becomes the program. exec only returns on error.
let error = exec(argv).unwrap_err();
eprintln!("rsh: {error}");
std::process::exit(127);
}
wait(pid)
}The important part is the separation between creating a process and starting a program.
fork creates a child process. After it returns, the same function continues in two processes.
The parent receives the child's PID, while the child receives zero:
if pid == 0 {
// This branch runs only in the child.
}This is where the two copies of the shell diverge.
The parent proceeds to wait. The child calls exec and replaces the program it is running.
After
fork, the parent and child have the same initial memory contents and the same set of open descriptors. The physical memory does not have to be copied immediately: the kernel normally uses copy-on-write and creates a separate copy of a page only after the first write.
Before exec, the child is still executing the shell's code. After a successful exec, it becomes another program, such as /usr/bin/echo.
exec Replaces the Program, Not the Process
exec does not create a new PID.
It loads a new program into an existing process. The code, stack, heap, and the rest of the address space are replaced, but the process itself remains.
Its PID, parent, and open file descriptors do not change, except for descriptors marked close-on-exec in advance.
Therefore, after:
exec(argv)the child does not "start echo next to itself." It becomes echo.
A process is a container for executing state: a PID, permissions, descriptors, and an address space. A program is the code and data loaded into that container.
execchanges the program while preserving the process.
A successful exec does not return. There is nowhere to return to because the Rust code that called it has disappeared from the address space.
The following lines therefore run only on error:
let error = exec(argv).unwrap_err();
eprintln!("rsh: {error}");
std::process::exit(127);If the executable cannot be found or launched, the child reports the error and exits with code 127.
The parent remains the shell throughout and waits for the child to finish.
Why cd Is Built into the Shell
This also explains why cd cannot be implemented as an ordinary external program.
Suppose the shell launches it in the same way:
shell
└─ fork → cdThe child changes its working directory and exits.
But the parent's working directory does not change. When the shell displays its prompt again, it remains in the previous directory.
The child process's state is not copied back into the parent.
That is why cd must run inside the shell itself:
if command == "cd" {
std::env::set_current_dir(path)?;
}For the same reason, export, unset, and exit are built-ins. They change the state of the current session and must therefore run in the process that owns that session.
An external program can receive a copy of the shell's environment, but it cannot change the environment of a parent that is already running.
Watching Two Processes with strace
Let us verify this model with an actual kernel trace.
The -f option makes strace follow child processes, while the filter keeps only process creation, program replacement, and waiting:
$ strace -f -e trace=clone,execve,wait4 -o simple.trace \
./rsh/target/release/rsh -c "echo hello"
$ grep -E "clone\(|execve\(.*= 0|wait4" simple.trace
2566099 execve("./rsh/target/release/rsh", ["./rsh/target/release/rsh", "-c", "echo hello"], 0x7ffe2a3a1178 /* 90 vars */) = 0
2566099 clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f4379a03a90) = 2566100
2566099 wait4(2566100 <unfinished ...>
2566100 execve("/usr/bin/echo", ["echo", "hello"], 0x7ffea7492678 /* 90 vars */) = 0
2566099 <... wait4 resumed>, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 2566100The first column is the PID of the process that made the system call.
Process 2566099 is our shell. Something also started the shell itself through execve, so the first line of the trace shows rsh being loaded.
The shell then calls clone and receives the child's PID, 2566100.
On Linux, the library implementation of fork uses the more general clone system call internally, which is why strace shows that name.
After the child is created, the trace splits between two PIDs.
The parent, 2566099, enters wait4:
2566099 wait4(2566100 <unfinished ...>The child, 2566100, loads echo:
2566100 execve("/usr/bin/echo", ["echo", "hello"], ...) = 0When echo exits, the parent returns from wait4 and receives its status:
WIFEXITED(s) && WEXITSTATUS(s) == 0The program exited normally with code 0.
cloneis the general Linux system call for creating processes and threads. Flags determine its exact behavior.SIGCHLDmeans that the parent should receive the corresponding signal when the child exits; it is one sign of classicforkbehavior.
One command already required two processes. A pipeline will require three: the parent shell and two children.
The | Character Becomes a Kernel Object
Return to this command:
echo one two | wc -wThe | character does not transfer bytes.
It tells the shell to create a pipe and connect two programs to it.
The pipe system call creates the pipe:
let mut ends = [0; 2];
ensure!(
unsafe { libc::pipe(ends.as_mut_ptr()) } == 0,
"pipe failed"
);
let (read_end, write_end) = (ends[0], ends[1]);The kernel returns two file descriptors.
Data is written into the pipe through write_end and read from it through read_end.
At this point, both ends belong to the shell. It then creates two children. Because fork inherits open descriptors, copies of both ends appear in every participant after the processes are created.
The result looks roughly like this:
shell: read_end, write_end
echo: read_end, write_end
wc: read_end, write_endBut echo should only write, while wc should only read.
The shell must leave the correct end in each process and make it the appropriate standard stream.
dup2 Connects a Program to the Pipe
The left child will become echo. Its standard output must lead to the pipe rather than the terminal.
Standard output always has descriptor number 1, so the child runs:
unsafe {
libc::dup2(write_end, 1);
libc::close(write_end);
libc::close(read_end);
}dup2(write_end, 1) makes descriptor 1 another reference to the same open end of the pipe.
The original write_end is no longer needed because stdout now points to the same place.
The child then calls exec and becomes echo.
The new program knows nothing about this substitution. It writes to stdout exactly as it would during an ordinary launch:
echo → write(1, ...)But descriptor 1 now leads to the pipe rather than the terminal.
The right child performs the mirror operation:
unsafe {
libc::dup2(read_end, 0);
libc::close(read_end);
libc::close(write_end);
}Descriptor 0 is standard input. After dup2, it points to the read end of the pipe.
The child then becomes wc.
It reads from stdin without knowing that the data comes from another process.
This is why almost any Unix program can participate in a pipeline. It needs no special support for |. It continues to use ordinary stdin and stdout, while the shell connects those descriptors to the correct kernel objects in advance.
The Substitution Survives exec
Let us inspect the pipeline with strace:
$ grep -E "pipe2\(|dup2\(|execve\(.*= 0" pipe.trace
2566105 execve("./rsh/target/release/rsh", ["./rsh/target/release/rsh", "-c", "echo one two | wc -w"], 0x7ffd640e1038 /* 90 vars */) = 0
2566105 pipe2([5, 6], 0) = 0
2566106 dup2(6, 1 <unfinished ...>
2566107 dup2(5, 0) = 0
2566107 execve("/usr/bin/wc", ["wc", "-w"], 0x7fff2c2c88c8 /* 90 vars */) = 0The shell, 2566105, receives descriptors [5, 6] from the kernel.
Descriptor 5 is the read end, and 6 is the write end.
The left child, 2566106, runs:
dup2(6, 1)Its stdout now leads into the pipe.
The right child, 2566107, runs:
dup2(5, 0)Its stdin now reads from the same pipe.
Only after replacing the streams do the children call execve and become echo and wc.
If open descriptors did not survive exec, the entire design would collapse: the new program would receive the terminal's standard streams again.
But exec changes the program while preserving the process's file descriptors. The prepared connection therefore survives the code replacement.
The Parent Also Received Both Ends of the Pipe
After launching the children, the shell does not participate in transferring data.
It neither writes to the pipe nor reads from it, so both parent copies must be closed:
unsafe {
libc::close(read_end);
libc::close(write_end);
}These lines are easy to mistake for optional resource cleanup. The read end can indeed almost be viewed that way.
But an unclosed write end changes the behavior of the entire pipeline.
wc reads until read returns zero. A zero result means EOF, the end of the input stream.
The kernel can return EOF only when two conditions are true: the pipe's buffer is empty, and not one open write end remains anywhere in the system.
The kernel counts any process with an open descriptor for the write end as a writer. It does not know whether that process actually intends to send anything. While at least one such descriptor exists, new data is still considered possible.
The termination of echo is therefore not enough.
If the parent shell keeps its copy of write_end, the kernel still sees a potential writer in the system.
wc reads the data already written, empties the pipe's buffer, and calls read again. Instead of EOF, the operation blocks: no data is currently available, but more can formally still arrive.
One Missing Line Hangs the Pipeline
Let us add a mode in which the shell deliberately does not close the write end:
unsafe {
libc::close(read_end);
if !leak_write_end {
libc::close(write_end);
}
}With the normal behavior, the pipeline finishes:
$ ./rsh/target/release/rsh -c "ls rsh/src | wc -l"
1Now run the same code while leaving write_end open in the parent:
$ timeout 3 ./rsh/target/release/rsh \
--leak-write-end \
-c "ls rsh/src | wc -l"
hung, killed by timeout: exit 124ls wrote its output and exited.
wc read every byte but did not receive EOF, so it kept waiting.
The shell itself was waiting for wc to finish.
This creates a closed loop. wc cannot finish while an open writer remains in the system. The parent shell is counted as that writer, and it is waiting for wc to finish.
After three seconds, the external timeout stops execution and returns code 124.
The programs did not change. The data did not change. We removed only one close, yet that was enough to change the protocol's meaning completely.
An open write end is not merely an occupied resource. It is a promise to the kernel that more data may still arrive.
Why the Parent Needs wait
After launching the pipeline, the shell closes its copies of the pipe and waits for the child processes.
Waiting is necessary not only to avoid displaying a new prompt too early.
When a process exits, the kernel retains its PID and exit code until the parent collects that status through one of the wait family of functions.
Until then, the child remains in the process table as a zombie.
A zombie no longer executes code or retains an ordinary address space. The kernel keeps only a small record with its exit status so that the parent can read it.
If the parent never calls wait, these records accumulate.
For a simple command, the shell waits for one child. For a pipeline, it must collect the statuses of every participant, although the pipeline's final exit code is normally the status of the last command.
For:
ls | wc -lthat is the status of wc.
Why Ctrl-C Reaches the Entire Pipeline
Parent-child relationships explain process creation and waiting, but they do not fully describe interactive control.
When the user presses Ctrl-C, the terminal sends SIGINT not to one PID but to the active process group.
The shell places every program in one pipeline into a shared process group and gives that group control of the terminal.
As a result, ls, wc, and every command in a longer pipeline receive the signal together.
A process group combines related processes for signal and terminal management. The foreground process group is the group to which the terminal currently sends signals generated by Ctrl-C, Ctrl-Z, and other control keys.
The interactive shell itself belongs to a different group. Otherwise, Ctrl-C would terminate not only the command but the shell as well.
When the pipeline finishes, the shell makes its own group active again and displays the prompt.
Job control is built on this mechanism: background commands, suspension with Ctrl-Z, and the fg and bg commands.
But the pipeline itself is understandable without implementing all of job control.
What Ultimately Happened After Enter
Return to the original line:
ls | wc -lThe shell parses it and sees two commands joined by a pipe.
It asks the kernel to create a pipe and then creates one child process for each program.
In the left child, the write end becomes standard output. In the right child, the read end becomes standard input.
The children then call exec and turn into ls and wc.
The parent closes its own copies of the pipe and waits for both processes to finish.
ls writes the file list to stdout without knowing that stdout leads to a pipe. wc reads stdin without knowing that ls is running at the other end.
When ls exits and the last open write end disappears, wc receives EOF, prints the number, and exits too.
The shell collects the children's statuses and displays its prompt again.
Between Enter and the output, there is no hidden interpreter moving data between commands. There are several processes and a set of descriptors prepared before exec.
And one line that looks like ordinary cleanup:
libc::close(write_end);is as much a part of the pipeline as pipe, fork, or dup2.
Without it, the number never appears on the screen.
Appendix: Full Source Files
rsh/Cargo.toml — 14 lines
[package]
name = "rsh"
version = "0.1.0"
edition = "2024"
[dependencies]
anyhow = "1"
clap = { version = "4", features = ["derive"] }
libc = "0.2"
[dev-dependencies]
[profile.release]
opt-level = 3rsh/src/main.rs — 200 lines
//! Scene 01: the core of a shell — everything that happens between
//! pressing Enter and seeing output.
//!
//! The loop is four verbs: read a line, split it into words, fork a
//! child that execs the program, wait for the child to finish. A `|`
//! adds one kernel object — a pipe — and one discipline: every end you
//! do not use must be closed, in every process that holds it. The
//! `--leak-write-end` flag deliberately breaks that discipline so the
//! resulting hang can be observed instead of believed.
use anyhow::{Context as _, Result, bail, ensure};
use clap::Parser;
use std::ffi::CString;
use std::io::{self, BufRead, Write};
/// A minimal shell: run one command line, or read lines from stdin.
#[derive(Parser)]
struct Args {
/// Execute this command line and exit (like sh -c)
#[arg(short = 'c')]
command: Option<String>,
/// Deliberately keep the pipe's write end open in the parent —
/// scene 03 uses this to demonstrate the classic hang
#[arg(long)]
leak_write_end: bool,
}
/// One command line: either a single program or `left | right`.
#[derive(Debug, PartialEq)]
enum Line {
Simple(Vec<String>),
Pipe(Vec<String>, Vec<String>),
}
fn parse(line: &str) -> Result<Line> {
let words =
|part: &str| -> Vec<String> { part.split_whitespace().map(str::to_owned).collect() };
match line.split_once('|') {
None => {
let simple = words(line);
ensure!(!simple.is_empty(), "empty command");
Ok(Line::Simple(simple))
}
Some((left, right)) => {
let (left, right) = (words(left), words(right));
ensure!(!left.is_empty(), "empty left side of the pipe");
ensure!(!right.is_empty(), "empty right side of the pipe");
Ok(Line::Pipe(left, right))
}
}
}
/// fork(2): after this call there are two processes running this very
/// function; the return value is the only way to tell them apart.
fn fork() -> Result<libc::pid_t> {
// SAFETY: fork has no preconditions; a single-threaded process may
// continue normally in both parent and child.
let pid = unsafe { libc::fork() };
ensure!(pid >= 0, "fork failed");
Ok(pid)
}
/// execvp(3): replace this process's program with another one. On
/// success it does not return — the code after it belongs to a program
/// that no longer exists in this process.
fn exec(argv: &[String]) -> Result<()> {
let c_args: Vec<CString> = argv
.iter()
.map(|arg| CString::new(arg.as_str()).context("argument contains NUL"))
.collect::<Result<_>>()?;
let mut pointers: Vec<*const libc::c_char> = c_args.iter().map(|arg| arg.as_ptr()).collect();
pointers.push(std::ptr::null());
// SAFETY: pointers reference live CStrings and end with NULL.
unsafe { libc::execvp(pointers[0], pointers.as_ptr()) };
bail!("exec {:?} failed: {}", argv[0], io::Error::last_os_error());
}
fn wait(pid: libc::pid_t) -> Result<i32> {
let mut status = 0;
// SAFETY: waiting for our own direct child.
let waited = unsafe { libc::waitpid(pid, &mut status, 0) };
ensure!(waited == pid, "waitpid failed");
Ok(libc::WEXITSTATUS(status))
}
fn run_simple(argv: &[String]) -> Result<i32> {
let pid = fork()?;
if pid == 0 {
// The child: becomes the program. exec only returns on error.
let error = exec(argv).unwrap_err();
eprintln!("rsh: {error}");
std::process::exit(127);
}
wait(pid)
}
fn run_pipe(left: &[String], right: &[String], leak_write_end: bool) -> Result<i32> {
let mut ends = [0; 2];
// SAFETY: pipe writes two valid descriptors into the array.
ensure!(unsafe { libc::pipe(ends.as_mut_ptr()) } == 0, "pipe failed");
let (read_end, write_end) = (ends[0], ends[1]);
let left_pid = fork()?;
if left_pid == 0 {
// Left child: stdout becomes the pipe's write end.
// SAFETY: descriptors are live; dup2/close have no other effects.
unsafe {
libc::dup2(write_end, 1);
libc::close(write_end);
libc::close(read_end);
}
let error = exec(left).unwrap_err();
eprintln!("rsh: {error}");
std::process::exit(127);
}
let right_pid = fork()?;
if right_pid == 0 {
// Right child: stdin becomes the pipe's read end.
// SAFETY: as above.
unsafe {
libc::dup2(read_end, 0);
libc::close(read_end);
libc::close(write_end);
}
let error = exec(right).unwrap_err();
eprintln!("rsh: {error}");
std::process::exit(127);
}
// The parent used neither end — both must be closed HERE, otherwise
// the reader never sees EOF: the kernel counts the parent's open
// write end as a potential writer.
// SAFETY: closing descriptors this process owns.
unsafe {
libc::close(read_end);
if !leak_write_end {
libc::close(write_end);
}
}
wait(left_pid)?;
wait(right_pid)
}
fn run_line(line: &str, leak_write_end: bool) -> Result<i32> {
match parse(line)? {
Line::Simple(argv) => run_simple(&argv),
Line::Pipe(left, right) => run_pipe(&left, &right, leak_write_end),
}
}
fn main() -> Result<()> {
let args = Args::parse();
if let Some(command) = args.command {
std::process::exit(run_line(&command, args.leak_write_end)?);
}
let stdin = io::stdin();
loop {
print!("rsh$ ");
io::stdout().flush()?;
let mut line = String::new();
if stdin.lock().read_line(&mut line)? == 0 {
return Ok(());
}
if line.trim().is_empty() {
continue;
}
if let Err(error) = run_line(line.trim(), args.leak_write_end) {
eprintln!("rsh: {error}");
}
}
}
#[cfg(test)]
mod tests {
use super::{Line, parse};
use anyhow::{Result, ensure};
#[test]
fn simple_line_splits_into_words() -> Result<()> {
ensure!(
parse("ls -l /tmp")? == Line::Simple(vec!["ls".into(), "-l".into(), "/tmp".into()])
);
Ok(())
}
#[test]
fn pipe_splits_into_two_commands() -> Result<()> {
let parsed = parse("ls | wc -l")?;
ensure!(parsed == Line::Pipe(vec!["ls".into()], vec!["wc".into(), "-l".into()]));
Ok(())
}
#[test]
fn empty_side_is_rejected() -> Result<()> {
ensure!(parse("ls |").is_err());
Ok(())
}
}