How to test a CLI end to end

Short answer: Drive the compiled binary with assert_cmd, match its stdout, stderr and exit status with predicates, and when the program talks to the network, answer it with a wiremock mock server that starts inside the test. The test crates are dev-dependencies, so the release binary never carries them.

The program under test

A CLI worth testing end to end touches the whole process surface: flags in, exit code out, output in between. cargo new fetch, then the dependencies. The program is a small HTTP client; how to build a good one is the next book’s problem, and this one exists to be tested.

[dependencies]
assert_cmd = "2.2.2"
clap = { version = "4.6.7", features = ["derive"] }
predicates = "3.1.4"
reqwest = { version = "0.13.5", features = ["blocking"] }
tokio = { version = "1.53.3", features = ["macros", "rt-multi-thread"] }
wiremock = "0.6.5"

In a project of your own, split this fence: clap and reqwest are what the binary links, and the four test crates belong under [dev-dependencies], which cargo compiles for tests, examples and benchmarks only. They sit under [dependencies] here so that every block below compiles as printed.

use clap::Parser;

#[derive(Parser)]
struct Args {
    /// URL to fetch
    #[arg(long)]
    url: String,
}

fn main() {
    let args = Args::parse();
    match reqwest::blocking::get(&args.url) {
        Ok(response) => {
            println!("{}", response.status());
            match response.text() {
                Ok(body) => println!("{body}"),
                Err(e) => eprintln!("body: {e}"),
            }
        }
        Err(e) => {
            eprintln!("{}: {e}", args.url);
            std::process::exit(1);
        }
    }
}

The program has three exits, and a test can check all three: no URL is a usage error, a URL that cannot be fetched is a failure, and a good URL prints the status line and the body. A bare run shows the first:

$ cargo run -q
error: the following required arguments were not provided:
  --url <URL>

Usage: fetch --url <URL>

For more information, try '--help'.

assert_cmd drives the binary

To see assert_cmd work on its own, here is a program that drives any binary on PATH:

use assert_cmd::Command;
use predicates::prelude::predicate;

fn main() {
    Command::new("git")
        .arg("--version")
        .assert()
        .success()
        .stdout(predicate::str::contains("git version"));

    println!("the child ran and every assertion held");
}

$ cargo run -q
the child ran and every assertion held

.assert() runs the child process and returns an Assert whose methods check one thing each: .success() for exit 0, .failure() for anything else, .stdout(...) and .stderr(...) against the output, .code(...) when the exact status matters. A failing assertion panics with a diff of what was expected against what arrived. Command::new drives anything on PATH; Command::cargo_bin("fetch") resolves the binary of the crate under test, because cargo sets CARGO_BIN_EXE_fetch when it builds the test crate, so no test depends on where the binary was installed.

The tests

use assert_cmd::Command;
use predicates::prelude::predicate;
use wiremock::matchers::{method, path};
use wiremock::{Mock, MockServer, ResponseTemplate};

fn fetch() -> Command {
    Command::cargo_bin("fetch").unwrap()
}

#[test]
fn no_url_is_a_usage_error() {
    fetch()
        .assert()
        .failure()
        .code(predicate::eq(2))
        .stderr(predicate::str::contains("--url"));
}

#[tokio::test]
async fn the_mock_answers_the_request() {
    let server = MockServer::start().await;
    Mock::given(method("GET"))
        .and(path("/weather"))
        .respond_with(ResponseTemplate::new(200).set_body_string("18C, overcast"))
        .mount(&server)
        .await;

    fetch()
        .args(["--url", &format!("{}/weather", server.uri())])
        .assert()
        .success()
        .stdout(predicate::str::contains("200 OK"))
        .stdout(predicate::str::contains("18C, overcast"));
}

#[tokio::test]
async fn an_unmatched_request_gets_404() {
    let server = MockServer::start().await;

    fetch()
        .args(["--url", &format!("{}/nothing-here", server.uri())])
        .assert()
        .success()
        .stdout(predicate::str::contains("404 Not Found"));
}

$ cargo test
   Compiling fetch v0.1.0 (/private/tmp/e2e-fetch)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.62s
     Running unittests src/main.rs (target/debug/deps/fetch-ac8f7f3213eba661)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/cli.rs (target/debug/deps/cli-ab76575344f33d3c)

running 3 tests
test no_url_is_a_usage_error ... ok
test the_mock_answers_the_request ... ok
test an_unmatched_request_gets_404 ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.32s

The tests live in tests/ because that is the only place that reaches the binary the way a user does: cargo compiles each file there as its own crate and builds the binary first, which is where CARGO_BIN_EXE_fetch comes from. The running 0 tests section is the binary crate itself — fetch has no unit tests, so all of its coverage is the integration file. The order the tests report in varies from run to run, and a cold build compiles the dependencies first; those lines are left out above, and the path and the hash belong to the machine the run happened on.

The first test needs no server; clap rejects the missing argument with exit code 2 before fetch runs. The other two start a MockServer, mount one canned response, and point the CLI at server.uri(), whose port is whatever the operating system handed out. Nothing leaves the machine, so the suite passes with the network unplugged and CI needs nothing beyond a toolchain.

The 404 test is the gotcha worth remembering: wiremock answers 404 Not Found to any request no mounted Mock matches, so a failing 200 OK assertion usually means a path() or method() matcher stopped matching, not that the program broke.

How to let a CLI read a config file — the previous step
Testing a CLI without a network — the same pattern inside a real CLI, wired into CI
Back to Building a command-line tool

The next step in this chapter builds a release binary you can hand to someone.

How to test a CLI end to end

The program under test

assert_cmd drives the binary

The tests

Next