← All posts

Why Cargo Keeps Rerunning build.rs: Declaring the Generator's Real Inputs

Consider a small crate named protogen that models code generation for a telemetry system. Such a system receives many kinds of messages from devices: sensor readings, limit settings, and service records. Instead of writing a Rust struct and parser for every format by hand, the project stores message descriptions in text schemas.

During the build, build.rs reads those schemas and generates Rust types from them. The crate's ordinary code includes the generated file and uses SensorReading, LimitConfig, and hundreds of other structs as normal types.

Now I change src/main.rs, a file the generator never reads, and the next build takes 2.399 s.

After two lines are added to build.rs, the same edit builds in 0.118 s. The generator's logic and output remain unchanged, but more than 20 times the unnecessary work disappears.

A build.rs file, or build script, is compiled and run by Cargo before the crate itself. It can generate code, locate system libraries, and send instructions to Cargo by printing lines beginning with cargo:... to standard output.

The generator itself is not slow. Cargo simply does not know which files it reads, so it has to treat the entire package as an input. But the obvious way to narrow that set is dangerous: forgetting even one real input makes the build fast while silently reusing stale code.

We will first catch the unnecessary rerun in Cargo's logs. Then we will declare the exact dependencies, verify the improvement, and deliberately provide an incomplete declaration to expose its more serious consequence. The complete source and scripts are collected in the appendix.

build.rs Turns Schemas into Rust Code

The schemas here are ordinary text files inside the project, not database table definitions. We use them to describe telemetry message formats: each record's name, fields, and types. These files are the source of truth, avoiding hundreds of manually synchronized Rust structs and parsing functions.

build.rs reads the schemas during the build and generates the corresponding Rust code. The primary records are written by hand; additional repetitive schemas only enlarge the example enough to make unnecessary recompilation visible.

The source format is deliberately simple. record starts a record, and indented lines declare its fields:

record SensorReading
  device_id: u32
  channel: u16
  temperature_mc: i32
  humidity_ppm: u32
  captured_at_ms: u64
  healthy: bool

SensorReading is not a file name. It is the name of a record in the schema. The generator turns it into pub struct SensorReading and writes that struct, along with the other generated types, into one generated.rs file.

The generator sorts the discovered files, parses the records, checks that names are unique, and creates structs with parse functions:

fn main() -> Result<()> {
    let manifest_dir = env::var("CARGO_MANIFEST_DIR")?;
    let out_dir = env::var("OUT_DIR")?;
    let schema_dir = Path::new(&manifest_dir).join("schema");

    let files = schema_files(&schema_dir)?;
    let mut records = Vec::new();
    for file in &files {
        records.append(&mut parse_schema_file(file)?);
    }
    check_unique_names(&records)?;

    let strict = env::var("PROTOGEN_STRICT").is_ok_and(|value| value == "1");
    let code = generate(&records, &files, strict)?;
    fs::write(PathBuf::from(out_dir).join("generated.rs"), code)?;
    Ok(())
}

Besides the schema directory, the generator reads the PROTOGEN_STRICT environment variable.

Why PROTOGEN_STRICT exists

PROTOGEN_STRICT is provided only while building:

$ PROTOGEN_STRICT=1 cargo build

Suppose the schema permits only device_id, but a device also sends firmware:

device_id=42;firmware=1.8.0

Without the variable, the generated parser reads device_id=42 and ignores the unfamiliar field. With PROTOGEN_STRICT=1, the same parser returns an error:

unknown key: firmware

build.rs chooses this behavior while creating generated.rs. The generated result therefore depends on both the files under schema/ and the value of PROTOGEN_STRICT.

The result is written to OUT_DIR and included in the crate:

include!(concat!(env!("OUT_DIR"), "/generated.rs"));

OUT_DIR points to a separate directory under target that Cargo assigns to a particular build script. Generated files belong there so the build does not modify the project's source files.

The working binary confirms that the schemas became ordinary Rust types:

$ ./target/debug/protogen
records generated: 643 from 18 schema files
strict mode: false
LimitConfig fields: ["max_batch", "flush_interval_ms", "compression"]
parsed: SensorReading { device_id: 42, channel: 3, temperature_mc: 21500, humidity_ppm: 440000, captured_at_ms: 1754300000000, healthy: true }

The generated module is much larger than the generator itself:

$ wc -l target/debug/build/protogen-*/out/generated.rs
60310 target/debug/build/protogen-a9f2125ad04fcb9f/out/generated.rs

Whenever build.rs runs again and rewrites this file, rustc may have to process 60310 generated lines again.

Without Directives, Cargo Watches the Entire Package

Change only src/main.rs. The schemas, generator mode, and build.rs itself remain untouched:

$ touch src/main.rs
$ time cargo build
   Compiling protogen v0.1.0 (…/protogen)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 2.39s

real	0m2.399s

touch does not change the file's contents, yet the build still takes 2.399 s. Cargo's verbose -vv mode shows that the generator ran again alongside the binary:

$ touch src/main.rs
$ cargo build -vv 2>&1 | grep -E 'Dirty|protogen:'
       Dirty protogen v0.1.0 (…/protogen): the precalculated components changed
[protogen 0.1.0] protogen: 643 records from 18 schema files in 29.579198ms

The second line was printed by build.rs itself. Cargo therefore did rerun the generator after an edit to a file that does not participate in generation.

The fingerprint log identifies the reason:

$ touch src/main.rs
$ CARGO_LOG=cargo::core::compiler::fingerprint=trace cargo build 2>&1 \
    | grep -oE 'PrecalculatedComponentsChanged \{[^}]*\}'
PrecalculatedComponentsChanged { old: "1785795587.570593534s (src/main.rs)", new: "1785795590.093588143s (src/main.rs)" }

A fingerprint stores information about the inputs to the previous build. Cargo compares the new fingerprint with the old one and reruns a step when they differ. The mtime inside it is a file's modification timestamp; touch changes exactly that value.

When a build script prints no rerun-if-* directives, Cargo uses a conservative default: it watches the files in the entire package. Cargo does not analyze read_dir and read_to_string calls inside an arbitrary Rust program, so it cannot infer those dependencies automatically.

Changing src/main.rs changes the package-wide fingerprint. Cargo runs build.rs, which writes generated.rs again, after which the library containing the generated types and the binary are recompiled.

The generator itself takes only 29.6 ms out of 2.399 s. Most of the cost comes later in the cascade, when rustc receives tens of thousands of generated lines again.

Two Directives Narrow the Fingerprint to the Real Inputs

The build script reads the schema directory and one user-controlled environment variable. Declare exactly those inputs:

println!("cargo:rerun-if-changed=schema/");
println!("cargo:rerun-if-env-changed=PROTOGEN_STRICT");

rerun-if-changed adds a path to the build script's input set, while rerun-if-env-changed adds an environment variable. Cargo remembers their states and runs the script again only after the corresponding input changes.

Now repeat the original experiment:

$ touch src/main.rs
$ time cargo build
   Compiling protogen v0.1.0 (…/protogen)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.10s

real	0m0.117s

Three runs took 0.117, 0.118, and 0.121 s. The median fell from 2.399 to 0.118 s.

The fingerprint log now marks only the build target containing src/main.rs as stale:

$ touch src/main.rs
$ CARGO_LOG=cargo::core::compiler::fingerprint=trace cargo build 2>&1 \
    | grep -oE 'stale: changed "[^"]*"' | sort -u | sed "s|$PWD|.|g"
stale: changed "./src/main.rs"

build-script-build no longer appears in the verbose output. Cargo preserves generated.rs and does not force the library to recompile its generated types.

The opposite direction matters too: changing a real input must still rerun the generator.

$ touch schema/core.schema
$ time cargo build
   Compiling protogen v0.1.0 (…/protogen)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 2.41s

real	0m2.419s

A schema edit still costs about 2.4 s because the generator must run and its output must pass through rustc. The medians of three runs separate eliminated work from necessary work:

Change before cargo buildNo directivesExact inputs
touch src/main.rs2.399 s0.118 s
touch schema/core.schema2.361 s2.348 s
cargo clean20.033 s19.528 s

The directives did not accelerate schema changes or clean builds. They removed only the false dependency on src/main.rs, which cost about 2.3 s after every unrelated edit.

An Incomplete Input List Produces Stale Code

The exact declaration looks simple, which makes it tempting to narrow it even further and name one file:

println!("cargo:rerun-if-changed=schema/core.schema");

An unrelated edit remains fast: touch src/main.rs builds in 0.130 s. By build time alone, this version looks almost identical to the correct one.

But the generator reads the entire directory. Add a field to a different existing file, schema/limits.schema:

$ printf '  retry_budget: u16\n' >> schema/limits.schema
$ cargo build
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.01s
$ ./target/debug/protogen | grep 'LimitConfig'
LimitConfig fields: ["max_batch", "flush_interval_ms", "compression"]

The schema already contains retry_budget, but the compiled LimitConfig does not. Cargo considers everything fresh because the only declared file did not change. No error or warning appears.

Adding a new file falls through the same gap:

$ printf '# Probe records for the v2 wire.\n\nrecord WireProbe\n  probe_id: u32\n' \
    > schema/wire_v2.schema
$ cargo build
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.01s
$ ./target/debug/protogen | grep 'records generated'
records generated: 643 from 18 schema files

The directory now contains one more file and one more record, but the binary continues reporting the previous values.

The environment variable was omitted as well:

$ PROTOGEN_STRICT=1 cargo build
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.01s
$ ./target/debug/protogen | grep 'strict'
strict mode: false

The new value will be read only the next time build.rs actually runs. Until then, the compiled code stays in the old mode.

As soon as a build script prints at least one rerun-if-* directive, Cargo stops using its package-wide fallback and trusts the declared list. It does not compare that list with the script's actual file reads. Declaring one input makes every forgotten input invisible.

If the only declared file now changes, the generator finally sees all the accumulated edits:

$ touch schema/core.schema
$ time cargo build
   Compiling protogen v0.1.0 (…/protogen)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 9.77s

real	0m9.787s
$ ./target/debug/protogen | grep -E 'records generated|LimitConfig'
records generated: 644 from 19 schema files
LimitConfig fields: ["max_batch", "flush_interval_ms", "compression", "retry_budget"]

All missed changes appear at once. This build is more expensive than a normal rerun after touch because generated.rs actually changes, leaving rustc with less previous work to reuse.

An incomplete declaration is worse than no declaration. Without directives, Cargo performs unnecessary work but remains correct. With an incomplete list, it quickly confirms a build whose generated code no longer matches its inputs.

Reading a Directory Requires a Dependency on the Directory

Restore the correct form:

println!("cargo:rerun-if-changed=schema/");
println!("cargo:rerun-if-env-changed=PROTOGEN_STRICT");

The directory path covers three events: editing an existing schema, adding a new one, and removing an old one. The final case can be tested with the wire_v2.schema file created above:

$ rm schema/wire_v2.schema
$ cargo build
   Compiling protogen v0.1.0 (…/protogen)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 10.18s
$ ./target/debug/protogen | grep 'records generated'
records generated: 643 from 18 schema files

A list of individual files cannot notice a new name that is not yet on the list. A dependency on the directory notices changes to its contents. A call to read_dir("schema") should therefore correspond to rerun-if-changed=schema/, not to a list of whatever files existed when the script was first written.

The environment variable behaves symmetrically. Once declared with rerun-if-env-changed, both setting and removing it rerun the generator:

$ PROTOGEN_STRICT=1 cargo build
   Compiling protogen v0.1.0 (…/protogen)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 17.24s
$ ./target/debug/protogen | grep 'strict'
strict mode: true
$ cargo build
   Compiling protogen v0.1.0 (…/protogen)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 16.59s
$ ./target/debug/protogen | grep 'strict'
strict mode: false

These reruns are expensive because the variable changes the generated code, not because checking an environment variable is costly. Strict mode changes all 643 parse functions, so rustc really does receive a different large module. The important part is that this cost appears only when a real input changes, not after an unrelated edit nearby.

build.rs itself does not need to be declared separately. Cargo always tracks the build script's own sources; changing them recompiles and reruns the script.

An Exact Fingerprint Provides Both Speed and Correctness

From the build system's perspective, the generator is a function:

generated.rs = generate(schema/, PROTOGEN_STRICT)

The rerun-if-* directives describe that function's arguments to Cargo. Without directives, Cargo substitutes the entire package. The result stays correct, but unrelated files create false reruns.

With an incomplete list, the fingerprint stops changing with some of the function's real arguments. Cargo faithfully reuses the old result even though the generator would produce a different file from the new inputs.

An exact declaration mirrors the script's actual reads: a directory for read_dir, a path for every individually read file, and rerun-if-env-changed for every environment variable that affects the result. Then an unrelated src/main.rs does not start the generator, while a new schema or generation mode cannot go unnoticed.

The purpose of these directives is not to run build.rs as rarely as possible. It is to run it exactly when its result could have changed.

Appendix: Full Source Files

protogen-naive/build.rs — 291 lines

//! Schema-driven code generator, the way build scripts are usually written.
//!
//! Reads every `.schema` file under `schema/`, parses the record
//! definitions and writes Rust structs with parse functions into
//! `$OUT_DIR/generated.rs`. The crate pulls the result in via `include!`.
//!
//! Deliberately missing: any `cargo:rerun-if-changed` directive. Without
//! it cargo falls back to "rerun when anything in the package changes",
//! so this script runs again after touching a file the schemas never see.

use std::collections::HashSet;
use std::env;
use std::fmt::Write as _;
use std::fs;
use std::path::{Path, PathBuf};
use std::time::Instant;

use anyhow::{Context, Result, ensure};

/// A single `name: type` line inside a record.
struct Field {
    name: String,
    ty: String,
}

/// One `record Name` block parsed from a schema file.
struct Record {
    name: String,
    fields: Vec<Field>,
    source: String,
}

/// Types the generator accepts on the right-hand side of a field.
const ALLOWED_TYPES: &[&str] = &[
    "u8", "u16", "u32", "u64", "i8", "i16", "i32", "i64", "f32", "f64", "bool", "String",
];

fn main() -> Result<()> {
    let started = Instant::now();
    let manifest_dir = env::var("CARGO_MANIFEST_DIR").context("CARGO_MANIFEST_DIR not set")?;
    let out_dir = env::var("OUT_DIR").context("OUT_DIR not set")?;
    let schema_dir = Path::new(&manifest_dir).join("schema");

    let files = schema_files(&schema_dir)?;
    ensure!(!files.is_empty(), "no .schema files in {schema_dir:?}");

    let mut records = Vec::new();
    for file in &files {
        let mut parsed = parse_schema_file(file)?;
        records.append(&mut parsed);
    }
    check_unique_names(&records)?;

    // Strict mode makes generated parsers reject unknown keys. The env
    // var is read here, at generation time, and baked into the output.
    let strict = env::var("PROTOGEN_STRICT").is_ok_and(|value| value == "1");

    let code = generate(&records, &files, strict)?;
    let target = PathBuf::from(&out_dir).join("generated.rs");
    fs::write(&target, code).with_context(|| format!("writing {target:?}"))?;

    // Visible in `cargo build -vv` output: proof of every rerun.
    println!(
        "protogen: {} records from {} schema files in {:?}",
        records.len(),
        files.len(),
        started.elapsed(),
    );
    Ok(())
}

/// Lists `.schema` files in the schema directory, sorted by name so the
/// generated output is deterministic.
fn schema_files(dir: &Path) -> Result<Vec<PathBuf>> {
    let entries = fs::read_dir(dir).with_context(|| format!("reading {dir:?}"))?;
    let mut files = Vec::new();
    for entry in entries {
        let path = entry.context("reading directory entry")?.path();
        let is_schema = path.extension().is_some_and(|ext| ext == "schema");
        if is_schema {
            files.push(path);
        }
    }
    files.sort();
    Ok(files)
}

/// Parses one schema file into records.
fn parse_schema_file(path: &Path) -> Result<Vec<Record>> {
    let text = fs::read_to_string(path).with_context(|| format!("reading {path:?}"))?;
    let source = path
        .file_name()
        .context("schema path has no file name")?
        .to_string_lossy()
        .into_owned();

    let mut records: Vec<Record> = Vec::new();
    for (index, raw_line) in text.lines().enumerate() {
        let line = raw_line.trim();
        let location = format!("{source}:{}", index + 1);
        if line.is_empty() {
            continue;
        }
        if line.starts_with('#') {
            continue;
        }
        if let Some(name) = line.strip_prefix("record ") {
            let name = name.trim();
            ensure!(is_type_name(name), "{location}: bad record name {name:?}");
            records.push(Record {
                name: name.to_string(),
                fields: Vec::new(),
                source: source.clone(),
            });
            continue;
        }
        let (name, ty) = line
            .split_once(':')
            .with_context(|| format!("{location}: expected `name: type`, got {line:?}"))?;
        let field = Field {
            name: name.trim().to_string(),
            ty: ty.trim().to_string(),
        };
        ensure!(
            is_field_name(&field.name),
            "{location}: bad field name {:?}",
            field.name
        );
        ensure!(
            ALLOWED_TYPES.contains(&field.ty.as_str()),
            "{location}: unsupported type {:?}",
            field.ty
        );
        let record = records
            .last_mut()
            .with_context(|| format!("{location}: field before any `record`"))?;
        record.fields.push(field);
    }
    for record in &records {
        ensure!(
            !record.fields.is_empty(),
            "{source}: record {} has no fields",
            record.name
        );
    }
    Ok(records)
}

/// Rejects duplicate record names across all schema files.
fn check_unique_names(records: &[Record]) -> Result<()> {
    let mut seen = HashSet::new();
    for record in records {
        let fresh = seen.insert(record.name.as_str());
        ensure!(
            fresh,
            "duplicate record name {} in {}",
            record.name,
            record.source
        );
    }
    Ok(())
}

/// `true` for `UpperCamel` identifiers.
fn is_type_name(name: &str) -> bool {
    let mut chars = name.chars();
    let head_ok = chars.next().is_some_and(|first| first.is_ascii_uppercase());
    head_ok && chars.all(|ch| ch.is_ascii_alphanumeric())
}

/// `true` for `lower_snake` identifiers.
fn is_field_name(name: &str) -> bool {
    let mut chars = name.chars();
    let head_ok = chars.next().is_some_and(|first| first.is_ascii_lowercase());
    head_ok && chars.all(|ch| ch.is_ascii_lowercase() || ch.is_ascii_digit() || ch == '_')
}

/// Renders the whole generated module as one string.
fn generate(records: &[Record], files: &[PathBuf], strict: bool) -> Result<String> {
    let mut out = String::new();
    writeln!(out, "// Generated by build.rs — do not edit by hand.")?;
    writeln!(out)?;
    writeln!(out, "/// Number of records across all schema files.")?;
    writeln!(out, "pub const RECORD_COUNT: usize = {};", records.len())?;
    writeln!(out, "/// Schema files the build script consumed, sorted.")?;
    write!(out, "pub const SCHEMA_FILES: &[&str] = &[")?;
    for file in files {
        let name = file
            .file_name()
            .context("schema path has no file name")?
            .to_string_lossy();
        write!(out, "{name:?}, ")?;
    }
    writeln!(out, "];")?;
    writeln!(out, "/// Whether generated parsers reject unknown keys.")?;
    writeln!(out, "pub const STRICT_MODE: bool = {strict};")?;
    for record in records {
        render_record(&mut out, record, strict)?;
    }
    Ok(out)
}

/// Renders one struct plus its `parse` / `field_names` impl.
fn render_record(out: &mut String, record: &Record, strict: bool) -> Result<()> {
    writeln!(out)?;
    writeln!(out, "/// Generated from `{}`.", record.source)?;
    writeln!(out, "#[derive(Debug, Clone, PartialEq)]")?;
    writeln!(out, "pub struct {} {{", record.name)?;
    for field in &record.fields {
        writeln!(out, "    pub {}: {},", field.name, field.ty)?;
    }
    writeln!(out, "}}")?;
    writeln!(out)?;
    writeln!(out, "impl {} {{", record.name)?;
    writeln!(out, "    /// Field names in schema order.")?;
    writeln!(
        out,
        "    pub fn field_names() -> &'static [&'static str] {{"
    )?;
    write!(out, "        &[")?;
    for field in &record.fields {
        write!(out, "{:?}, ", field.name)?;
    }
    writeln!(out, "]")?;
    writeln!(out, "    }}")?;
    writeln!(out)?;
    writeln!(
        out,
        "    /// Parses a `key=value;key=value` line into the record."
    )?;
    writeln!(
        out,
        "    pub fn parse(input: &str) -> Result<Self, String> {{"
    )?;
    for field in &record.fields {
        writeln!(
            out,
            "        let mut {}: Option<{}> = None;",
            field.name, field.ty
        )?;
    }
    writeln!(out, "        for pair in input.split(';') {{")?;
    writeln!(out, "            let pair = pair.trim();")?;
    writeln!(out, "            if pair.is_empty() {{")?;
    writeln!(out, "                continue;")?;
    writeln!(out, "            }}")?;
    writeln!(out, "            let (key, value) = pair")?;
    writeln!(out, "                .split_once('=')")?;
    writeln!(
        out,
        "                .ok_or_else(|| format!(\"bad pair: {{pair}}\"))?;"
    )?;
    writeln!(out, "            match key.trim() {{")?;
    for field in &record.fields {
        let convert = if field.ty == "String" {
            "value.trim().to_string()".to_string()
        } else {
            format!(
                "value.trim().parse::<{}>().map_err(|error| format!(\"{}: {{error}}\"))?",
                field.ty, field.name
            )
        };
        writeln!(
            out,
            "                {:?} => {} = Some({convert}),",
            field.name, field.name
        )?;
    }
    if strict {
        writeln!(
            out,
            "                other => return Err(format!(\"unknown key: {{other}}\")),"
        )?;
    } else {
        writeln!(out, "                _other => {{}}")?;
    }
    writeln!(out, "            }}")?;
    writeln!(out, "        }}")?;
    writeln!(out, "        Ok(Self {{")?;
    for field in &record.fields {
        writeln!(
            out,
            "            {}: {}.ok_or(\"missing field: {}\")?,",
            field.name, field.name, field.name
        )?;
    }
    writeln!(out, "        }})")?;
    writeln!(out, "    }}")?;
    writeln!(out, "}}")?;
    Ok(())
}

protogen-naive/Cargo.toml — 11 lines

[package]
name = "protogen-naive"
version = "0.1.0"
edition = "2024"
publish = false

[dependencies]
anyhow = "1"

[build-dependencies]
anyhow = "1"

protogen-naive/src/lib.rs — 6 lines

//! Types generated at build time from `schema/*.schema`.
//!
//! The whole module body lives in `$OUT_DIR/generated.rs`; the build
//! script writes it, this file only pulls it in.

include!(concat!(env!("OUT_DIR"), "/generated.rs"));

protogen-naive/src/main.rs — 21 lines

//! Smoke check for the generated types: parse one wire line per record
//! kind and report what the build script produced.

use anyhow::{Error, Result};

use protogen_naive::{LimitConfig, RECORD_COUNT, SCHEMA_FILES, STRICT_MODE, SensorReading};

fn main() -> Result<()> {
    println!(
        "records generated: {RECORD_COUNT} from {} schema files",
        SCHEMA_FILES.len()
    );
    println!("strict mode: {STRICT_MODE}");
    println!("LimitConfig fields: {:?}", LimitConfig::field_names());

    let wire = "device_id=42;channel=3;temperature_mc=21500;\
                humidity_ppm=440000;captured_at_ms=1754300000000;healthy=true";
    let reading = SensorReading::parse(wire).map_err(Error::msg)?;
    println!("parsed: {reading:?}");
    Ok(())
}

protogen-naive/schema/core.schema — 22 lines

# Core telemetry records the ingest node accepts.
# Format: `record Name` opens a record, indented `name: type` lines
# declare fields, `#` starts a comment.

record SensorReading
  device_id: u32
  channel: u16
  temperature_mc: i32
  humidity_ppm: u32
  captured_at_ms: u64
  healthy: bool

record DeviceEvent
  device_id: u32
  kind: String
  payload: String
  emitted_at_ms: u64

record LimitConfig
  max_batch: u32
  flush_interval_ms: u64
  compression: String

protogen-precise/build.rs — 293 lines

//! Schema-driven code generator, incremental edition.
//!
//! Same generator as `01-rerun-everything`, one difference: it declares
//! its real inputs. `cargo:rerun-if-changed=schema/` narrows the rerun
//! trigger from "anything in the package" down to the schema directory,
//! and `cargo:rerun-if-env-changed` covers the env var the script reads.

use std::collections::HashSet;
use std::env;
use std::fmt::Write as _;
use std::fs;
use std::path::{Path, PathBuf};
use std::time::Instant;

use anyhow::{Context, Result, ensure};

/// A single `name: type` line inside a record.
struct Field {
    name: String,
    ty: String,
}

/// One `record Name` block parsed from a schema file.
struct Record {
    name: String,
    fields: Vec<Field>,
    source: String,
}

/// Types the generator accepts on the right-hand side of a field.
const ALLOWED_TYPES: &[&str] = &[
    "u8", "u16", "u32", "u64", "i8", "i16", "i32", "i64", "f32", "f64", "bool", "String",
];

fn main() -> Result<()> {
    let started = Instant::now();
    let manifest_dir = env::var("CARGO_MANIFEST_DIR").context("CARGO_MANIFEST_DIR not set")?;
    let out_dir = env::var("OUT_DIR").context("OUT_DIR not set")?;
    let schema_dir = Path::new(&manifest_dir).join("schema");

    let files = schema_files(&schema_dir)?;
    ensure!(!files.is_empty(), "no .schema files in {schema_dir:?}");

    let mut records = Vec::new();
    for file in &files {
        let mut parsed = parse_schema_file(file)?;
        records.append(&mut parsed);
    }
    check_unique_names(&records)?;

    // Strict mode makes generated parsers reject unknown keys. The env
    // var is read here, at generation time, and baked into the output.
    let strict = env::var("PROTOGEN_STRICT").is_ok_and(|value| value == "1");

    let code = generate(&records, &files, strict)?;
    let target = PathBuf::from(&out_dir).join("generated.rs");
    fs::write(&target, code).with_context(|| format!("writing {target:?}"))?;

    // The fix: declare the real inputs. Cargo now reruns this script
    // only when the schema directory or the declared env var changes.
    println!("cargo:rerun-if-changed=schema/");
    println!("cargo:rerun-if-env-changed=PROTOGEN_STRICT");

    // Visible in `cargo build -vv` output: proof of every rerun.
    println!(
        "protogen: {} records from {} schema files in {:?}",
        records.len(),
        files.len(),
        started.elapsed(),
    );
    Ok(())
}

/// Lists `.schema` files in the schema directory, sorted by name so the
/// generated output is deterministic.
fn schema_files(dir: &Path) -> Result<Vec<PathBuf>> {
    let entries = fs::read_dir(dir).with_context(|| format!("reading {dir:?}"))?;
    let mut files = Vec::new();
    for entry in entries {
        let path = entry.context("reading directory entry")?.path();
        let is_schema = path.extension().is_some_and(|ext| ext == "schema");
        if is_schema {
            files.push(path);
        }
    }
    files.sort();
    Ok(files)
}

/// Parses one schema file into records.
fn parse_schema_file(path: &Path) -> Result<Vec<Record>> {
    let text = fs::read_to_string(path).with_context(|| format!("reading {path:?}"))?;
    let source = path
        .file_name()
        .context("schema path has no file name")?
        .to_string_lossy()
        .into_owned();

    let mut records: Vec<Record> = Vec::new();
    for (index, raw_line) in text.lines().enumerate() {
        let line = raw_line.trim();
        let location = format!("{source}:{}", index + 1);
        if line.is_empty() {
            continue;
        }
        if line.starts_with('#') {
            continue;
        }
        if let Some(name) = line.strip_prefix("record ") {
            let name = name.trim();
            ensure!(is_type_name(name), "{location}: bad record name {name:?}");
            records.push(Record {
                name: name.to_string(),
                fields: Vec::new(),
                source: source.clone(),
            });
            continue;
        }
        let (name, ty) = line
            .split_once(':')
            .with_context(|| format!("{location}: expected `name: type`, got {line:?}"))?;
        let field = Field {
            name: name.trim().to_string(),
            ty: ty.trim().to_string(),
        };
        ensure!(
            is_field_name(&field.name),
            "{location}: bad field name {:?}",
            field.name
        );
        ensure!(
            ALLOWED_TYPES.contains(&field.ty.as_str()),
            "{location}: unsupported type {:?}",
            field.ty
        );
        let record = records
            .last_mut()
            .with_context(|| format!("{location}: field before any `record`"))?;
        record.fields.push(field);
    }
    for record in &records {
        ensure!(
            !record.fields.is_empty(),
            "{source}: record {} has no fields",
            record.name
        );
    }
    Ok(records)
}

/// Rejects duplicate record names across all schema files.
fn check_unique_names(records: &[Record]) -> Result<()> {
    let mut seen = HashSet::new();
    for record in records {
        let fresh = seen.insert(record.name.as_str());
        ensure!(
            fresh,
            "duplicate record name {} in {}",
            record.name,
            record.source
        );
    }
    Ok(())
}

/// `true` for `UpperCamel` identifiers.
fn is_type_name(name: &str) -> bool {
    let mut chars = name.chars();
    let head_ok = chars.next().is_some_and(|first| first.is_ascii_uppercase());
    head_ok && chars.all(|ch| ch.is_ascii_alphanumeric())
}

/// `true` for `lower_snake` identifiers.
fn is_field_name(name: &str) -> bool {
    let mut chars = name.chars();
    let head_ok = chars.next().is_some_and(|first| first.is_ascii_lowercase());
    head_ok && chars.all(|ch| ch.is_ascii_lowercase() || ch.is_ascii_digit() || ch == '_')
}

/// Renders the whole generated module as one string.
fn generate(records: &[Record], files: &[PathBuf], strict: bool) -> Result<String> {
    let mut out = String::new();
    writeln!(out, "// Generated by build.rs — do not edit by hand.")?;
    writeln!(out)?;
    writeln!(out, "/// Number of records across all schema files.")?;
    writeln!(out, "pub const RECORD_COUNT: usize = {};", records.len())?;
    writeln!(out, "/// Schema files the build script consumed, sorted.")?;
    write!(out, "pub const SCHEMA_FILES: &[&str] = &[")?;
    for file in files {
        let name = file
            .file_name()
            .context("schema path has no file name")?
            .to_string_lossy();
        write!(out, "{name:?}, ")?;
    }
    writeln!(out, "];")?;
    writeln!(out, "/// Whether generated parsers reject unknown keys.")?;
    writeln!(out, "pub const STRICT_MODE: bool = {strict};")?;
    for record in records {
        render_record(&mut out, record, strict)?;
    }
    Ok(out)
}

/// Renders one struct plus its `parse` / `field_names` impl.
fn render_record(out: &mut String, record: &Record, strict: bool) -> Result<()> {
    writeln!(out)?;
    writeln!(out, "/// Generated from `{}`.", record.source)?;
    writeln!(out, "#[derive(Debug, Clone, PartialEq)]")?;
    writeln!(out, "pub struct {} {{", record.name)?;
    for field in &record.fields {
        writeln!(out, "    pub {}: {},", field.name, field.ty)?;
    }
    writeln!(out, "}}")?;
    writeln!(out)?;
    writeln!(out, "impl {} {{", record.name)?;
    writeln!(out, "    /// Field names in schema order.")?;
    writeln!(
        out,
        "    pub fn field_names() -> &'static [&'static str] {{"
    )?;
    write!(out, "        &[")?;
    for field in &record.fields {
        write!(out, "{:?}, ", field.name)?;
    }
    writeln!(out, "]")?;
    writeln!(out, "    }}")?;
    writeln!(out)?;
    writeln!(
        out,
        "    /// Parses a `key=value;key=value` line into the record."
    )?;
    writeln!(
        out,
        "    pub fn parse(input: &str) -> Result<Self, String> {{"
    )?;
    for field in &record.fields {
        writeln!(
            out,
            "        let mut {}: Option<{}> = None;",
            field.name, field.ty
        )?;
    }
    writeln!(out, "        for pair in input.split(';') {{")?;
    writeln!(out, "            let pair = pair.trim();")?;
    writeln!(out, "            if pair.is_empty() {{")?;
    writeln!(out, "                continue;")?;
    writeln!(out, "            }}")?;
    writeln!(out, "            let (key, value) = pair")?;
    writeln!(out, "                .split_once('=')")?;
    writeln!(
        out,
        "                .ok_or_else(|| format!(\"bad pair: {{pair}}\"))?;"
    )?;
    writeln!(out, "            match key.trim() {{")?;
    for field in &record.fields {
        let convert = if field.ty == "String" {
            "value.trim().to_string()".to_string()
        } else {
            format!(
                "value.trim().parse::<{}>().map_err(|error| format!(\"{}: {{error}}\"))?",
                field.ty, field.name
            )
        };
        writeln!(
            out,
            "                {:?} => {} = Some({convert}),",
            field.name, field.name
        )?;
    }
    if strict {
        writeln!(
            out,
            "                other => return Err(format!(\"unknown key: {{other}}\")),"
        )?;
    } else {
        writeln!(out, "                _other => {{}}")?;
    }
    writeln!(out, "            }}")?;
    writeln!(out, "        }}")?;
    writeln!(out, "        Ok(Self {{")?;
    for field in &record.fields {
        writeln!(
            out,
            "            {}: {}.ok_or(\"missing field: {}\")?,",
            field.name, field.name, field.name
        )?;
    }
    writeln!(out, "        }})")?;
    writeln!(out, "    }}")?;
    writeln!(out, "}}")?;
    Ok(())
}

protogen-incomplete/build.rs — 294 lines

//! Schema-driven code generator, the trap edition.
//!
//! Same generator as the other two crates, but the directives at the
//! bottom are wrong in the classic way: rerun-if-changed points at ONE
//! schema file, while the script reads the whole directory plus an env
//! var. Cargo believes the declaration, not the actual reads — so edits
//! to `extra.schema`, new schema files and `PROTOGEN_STRICT` changes all
//! produce silently stale generated types.

use std::collections::HashSet;
use std::env;
use std::fmt::Write as _;
use std::fs;
use std::path::{Path, PathBuf};
use std::time::Instant;

use anyhow::{Context, Result, ensure};

/// A single `name: type` line inside a record.
struct Field {
    name: String,
    ty: String,
}

/// One `record Name` block parsed from a schema file.
struct Record {
    name: String,
    fields: Vec<Field>,
    source: String,
}

/// Types the generator accepts on the right-hand side of a field.
const ALLOWED_TYPES: &[&str] = &[
    "u8", "u16", "u32", "u64", "i8", "i16", "i32", "i64", "f32", "f64", "bool", "String",
];

fn main() -> Result<()> {
    let started = Instant::now();
    let manifest_dir = env::var("CARGO_MANIFEST_DIR").context("CARGO_MANIFEST_DIR not set")?;
    let out_dir = env::var("OUT_DIR").context("OUT_DIR not set")?;
    let schema_dir = Path::new(&manifest_dir).join("schema");

    let files = schema_files(&schema_dir)?;
    ensure!(!files.is_empty(), "no .schema files in {schema_dir:?}");

    let mut records = Vec::new();
    for file in &files {
        let mut parsed = parse_schema_file(file)?;
        records.append(&mut parsed);
    }
    check_unique_names(&records)?;

    // Strict mode makes generated parsers reject unknown keys. The env
    // var is read here, at generation time, and baked into the output.
    let strict = env::var("PROTOGEN_STRICT").is_ok_and(|value| value == "1");

    let code = generate(&records, &files, strict)?;
    let target = PathBuf::from(&out_dir).join("generated.rs");
    fs::write(&target, code).with_context(|| format!("writing {target:?}"))?;

    // The trap: one file declared, the rest of the inputs forgotten.
    // From here on cargo trusts this line and ignores everything else.
    println!("cargo:rerun-if-changed=schema/base.schema");

    // Visible in `cargo build -vv` output: proof of every rerun.
    println!(
        "protogen: {} records from {} schema files in {:?}",
        records.len(),
        files.len(),
        started.elapsed(),
    );
    Ok(())
}

/// Lists `.schema` files in the schema directory, sorted by name so the
/// generated output is deterministic.
fn schema_files(dir: &Path) -> Result<Vec<PathBuf>> {
    let entries = fs::read_dir(dir).with_context(|| format!("reading {dir:?}"))?;
    let mut files = Vec::new();
    for entry in entries {
        let path = entry.context("reading directory entry")?.path();
        let is_schema = path.extension().is_some_and(|ext| ext == "schema");
        if is_schema {
            files.push(path);
        }
    }
    files.sort();
    Ok(files)
}

/// Parses one schema file into records.
fn parse_schema_file(path: &Path) -> Result<Vec<Record>> {
    let text = fs::read_to_string(path).with_context(|| format!("reading {path:?}"))?;
    let source = path
        .file_name()
        .context("schema path has no file name")?
        .to_string_lossy()
        .into_owned();

    let mut records: Vec<Record> = Vec::new();
    for (index, raw_line) in text.lines().enumerate() {
        let line = raw_line.trim();
        let location = format!("{source}:{}", index + 1);
        if line.is_empty() {
            continue;
        }
        if line.starts_with('#') {
            continue;
        }
        if let Some(name) = line.strip_prefix("record ") {
            let name = name.trim();
            ensure!(is_type_name(name), "{location}: bad record name {name:?}");
            records.push(Record {
                name: name.to_string(),
                fields: Vec::new(),
                source: source.clone(),
            });
            continue;
        }
        let (name, ty) = line
            .split_once(':')
            .with_context(|| format!("{location}: expected `name: type`, got {line:?}"))?;
        let field = Field {
            name: name.trim().to_string(),
            ty: ty.trim().to_string(),
        };
        ensure!(
            is_field_name(&field.name),
            "{location}: bad field name {:?}",
            field.name
        );
        ensure!(
            ALLOWED_TYPES.contains(&field.ty.as_str()),
            "{location}: unsupported type {:?}",
            field.ty
        );
        let record = records
            .last_mut()
            .with_context(|| format!("{location}: field before any `record`"))?;
        record.fields.push(field);
    }
    for record in &records {
        ensure!(
            !record.fields.is_empty(),
            "{source}: record {} has no fields",
            record.name
        );
    }
    Ok(records)
}

/// Rejects duplicate record names across all schema files.
fn check_unique_names(records: &[Record]) -> Result<()> {
    let mut seen = HashSet::new();
    for record in records {
        let fresh = seen.insert(record.name.as_str());
        ensure!(
            fresh,
            "duplicate record name {} in {}",
            record.name,
            record.source
        );
    }
    Ok(())
}

/// `true` for `UpperCamel` identifiers.
fn is_type_name(name: &str) -> bool {
    let mut chars = name.chars();
    let head_ok = chars.next().is_some_and(|first| first.is_ascii_uppercase());
    head_ok && chars.all(|ch| ch.is_ascii_alphanumeric())
}

/// `true` for `lower_snake` identifiers.
fn is_field_name(name: &str) -> bool {
    let mut chars = name.chars();
    let head_ok = chars.next().is_some_and(|first| first.is_ascii_lowercase());
    head_ok && chars.all(|ch| ch.is_ascii_lowercase() || ch.is_ascii_digit() || ch == '_')
}

/// Renders the whole generated module as one string.
fn generate(records: &[Record], files: &[PathBuf], strict: bool) -> Result<String> {
    let mut out = String::new();
    writeln!(out, "// Generated by build.rs — do not edit by hand.")?;
    writeln!(out)?;
    writeln!(out, "/// Number of records across all schema files.")?;
    writeln!(out, "pub const RECORD_COUNT: usize = {};", records.len())?;
    writeln!(out, "/// Schema files the build script consumed, sorted.")?;
    write!(out, "pub const SCHEMA_FILES: &[&str] = &[")?;
    for file in files {
        let name = file
            .file_name()
            .context("schema path has no file name")?
            .to_string_lossy();
        write!(out, "{name:?}, ")?;
    }
    writeln!(out, "];")?;
    writeln!(out, "/// Whether generated parsers reject unknown keys.")?;
    writeln!(out, "pub const STRICT_MODE: bool = {strict};")?;
    for record in records {
        render_record(&mut out, record, strict)?;
    }
    Ok(out)
}

/// Renders one struct plus its `parse` / `field_names` impl.
fn render_record(out: &mut String, record: &Record, strict: bool) -> Result<()> {
    writeln!(out)?;
    writeln!(out, "/// Generated from `{}`.", record.source)?;
    writeln!(out, "#[derive(Debug, Clone, PartialEq)]")?;
    writeln!(out, "pub struct {} {{", record.name)?;
    for field in &record.fields {
        writeln!(out, "    pub {}: {},", field.name, field.ty)?;
    }
    writeln!(out, "}}")?;
    writeln!(out)?;
    writeln!(out, "impl {} {{", record.name)?;
    writeln!(out, "    /// Field names in schema order.")?;
    writeln!(
        out,
        "    pub fn field_names() -> &'static [&'static str] {{"
    )?;
    write!(out, "        &[")?;
    for field in &record.fields {
        write!(out, "{:?}, ", field.name)?;
    }
    writeln!(out, "]")?;
    writeln!(out, "    }}")?;
    writeln!(out)?;
    writeln!(
        out,
        "    /// Parses a `key=value;key=value` line into the record."
    )?;
    writeln!(
        out,
        "    pub fn parse(input: &str) -> Result<Self, String> {{"
    )?;
    for field in &record.fields {
        writeln!(
            out,
            "        let mut {}: Option<{}> = None;",
            field.name, field.ty
        )?;
    }
    writeln!(out, "        for pair in input.split(';') {{")?;
    writeln!(out, "            let pair = pair.trim();")?;
    writeln!(out, "            if pair.is_empty() {{")?;
    writeln!(out, "                continue;")?;
    writeln!(out, "            }}")?;
    writeln!(out, "            let (key, value) = pair")?;
    writeln!(out, "                .split_once('=')")?;
    writeln!(
        out,
        "                .ok_or_else(|| format!(\"bad pair: {{pair}}\"))?;"
    )?;
    writeln!(out, "            match key.trim() {{")?;
    for field in &record.fields {
        let convert = if field.ty == "String" {
            "value.trim().to_string()".to_string()
        } else {
            format!(
                "value.trim().parse::<{}>().map_err(|error| format!(\"{}: {{error}}\"))?",
                field.ty, field.name
            )
        };
        writeln!(
            out,
            "                {:?} => {} = Some({convert}),",
            field.name, field.name
        )?;
    }
    if strict {
        writeln!(
            out,
            "                other => return Err(format!(\"unknown key: {{other}}\")),"
        )?;
    } else {
        writeln!(out, "                _other => {{}}")?;
    }
    writeln!(out, "            }}")?;
    writeln!(out, "        }}")?;
    writeln!(out, "        Ok(Self {{")?;
    for field in &record.fields {
        writeln!(
            out,
            "            {}: {}.ok_or(\"missing field: {}\")?,",
            field.name, field.name, field.name
        )?;
    }
    writeln!(out, "        }})")?;
    writeln!(out, "    }}")?;
    writeln!(out, "}}")?;
    Ok(())
}

protogen-incomplete/schema/base.schema — 15 lines

# Wire records, part one. The build script declares THIS file
# in rerun-if-changed — and only this file.

record SensorReading
  device_id: u32
  channel: u16
  temperature_mc: i32
  captured_at_ms: u64
  healthy: bool

record DeviceEvent
  device_id: u32
  kind: String
  payload: String
  emitted_at_ms: u64

protogen-incomplete/schema/extra.schema — 8 lines

# Wire records, part two. The build script reads this file as well —
# but forgot to declare it in rerun-if-changed. Edits here are invisible
# to cargo: the generated types silently go stale.

record LimitConfig
  max_batch: u32
  flush_interval_ms: u64
  compression: String
Newsletter

New playgrounds in your inbox

Get an email when a new playground drops, plus the occasional engineering deep-dive. No spam, unsubscribe anytime.