Consider a small crate named protogen that models code generation for a telemetry system. Such a system receives many kinds of messages from devices: sensor readings, limit settings, and service records. Instead of writing a Rust struct and parser for every format by hand, the project stores message descriptions in text schemas.
During the build, build.rs reads those schemas and generates Rust types from them. The crate's ordinary code includes the generated file and uses SensorReading, LimitConfig, and hundreds of other structs as normal types.
Now I change src/main.rs, a file the generator never reads, and the next build takes 2.399 s.
After two lines are added to build.rs, the same edit builds in 0.118 s. The generator's logic and output remain unchanged, but more than 20 times the unnecessary work disappears.
A
build.rsfile, or build script, is compiled and run by Cargo before the crate itself. It can generate code, locate system libraries, and send instructions to Cargo by printing lines beginning withcargo:...to standard output.
The generator itself is not slow. Cargo simply does not know which files it reads, so it has to treat the entire package as an input. But the obvious way to narrow that set is dangerous: forgetting even one real input makes the build fast while silently reusing stale code.
We will first catch the unnecessary rerun in Cargo's logs. Then we will declare the exact dependencies, verify the improvement, and deliberately provide an incomplete declaration to expose its more serious consequence. The complete source and scripts are collected in the appendix.
build.rs Turns Schemas into Rust Code
The schemas here are ordinary text files inside the project, not database table definitions. We use them to describe telemetry message formats: each record's name, fields, and types. These files are the source of truth, avoiding hundreds of manually synchronized Rust structs and parsing functions.
build.rs reads the schemas during the build and generates the corresponding Rust code. The primary records are written by hand; additional repetitive schemas only enlarge the example enough to make unnecessary recompilation visible.
The source format is deliberately simple. record starts a record, and indented lines declare its fields:
record SensorReading
device_id: u32
channel: u16
temperature_mc: i32
humidity_ppm: u32
captured_at_ms: u64
healthy: bool
SensorReadingis not a file name. It is the name of a record in the schema. The generator turns it intopub struct SensorReadingand writes that struct, along with the other generated types, into onegenerated.rsfile.
The generator sorts the discovered files, parses the records, checks that names are unique, and creates structs with parse functions:
fn main() -> Result<()> {
let manifest_dir = env::var("CARGO_MANIFEST_DIR")?;
let out_dir = env::var("OUT_DIR")?;
let schema_dir = Path::new(&manifest_dir).join("schema");
let files = schema_files(&schema_dir)?;
let mut records = Vec::new();
for file in &files {
records.append(&mut parse_schema_file(file)?);
}
check_unique_names(&records)?;
let strict = env::var("PROTOGEN_STRICT").is_ok_and(|value| value == "1");
let code = generate(&records, &files, strict)?;
fs::write(PathBuf::from(out_dir).join("generated.rs"), code)?;
Ok(())
}
Besides the schema directory, the generator reads the PROTOGEN_STRICT environment variable.
The result is written to OUT_DIR and included in the crate:
include!(concat!(env!("OUT_DIR"), "/generated.rs"));
OUT_DIRpoints to a separate directory undertargetthat Cargo assigns to a particular build script. Generated files belong there so the build does not modify the project's source files.
The working binary confirms that the schemas became ordinary Rust types:
$ ./target/debug/protogen
records generated: 643 from 18 schema files
strict mode: false
LimitConfig fields: ["max_batch", "flush_interval_ms", "compression"]
parsed: SensorReading { device_id: 42, channel: 3, temperature_mc: 21500, humidity_ppm: 440000, captured_at_ms: 1754300000000, healthy: true }
The generated module is much larger than the generator itself:
$ wc -l target/debug/build/protogen-*/out/generated.rs
60310 target/debug/build/protogen-a9f2125ad04fcb9f/out/generated.rs
Whenever build.rs runs again and rewrites this file, rustc may have to process 60310 generated lines again.
Without Directives, Cargo Watches the Entire Package
Change only src/main.rs. The schemas, generator mode, and build.rs itself remain untouched:
$ touch src/main.rs
$ time cargo build
Compiling protogen v0.1.0 (…/protogen)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 2.39s
real 0m2.399s
touch does not change the file's contents, yet the build still takes 2.399 s. Cargo's verbose -vv mode shows that the generator ran again alongside the binary:
$ touch src/main.rs
$ cargo build -vv 2>&1 | grep -E 'Dirty|protogen:'
Dirty protogen v0.1.0 (…/protogen): the precalculated components changed
[protogen 0.1.0] protogen: 643 records from 18 schema files in 29.579198ms
The second line was printed by build.rs itself. Cargo therefore did rerun the generator after an edit to a file that does not participate in generation.
The fingerprint log identifies the reason:
$ touch src/main.rs
$ CARGO_LOG=cargo::core::compiler::fingerprint=trace cargo build 2>&1 \
| grep -oE 'PrecalculatedComponentsChanged \{[^}]*\}'
PrecalculatedComponentsChanged { old: "1785795587.570593534s (src/main.rs)", new: "1785795590.093588143s (src/main.rs)" }
A fingerprint stores information about the inputs to the previous build. Cargo compares the new fingerprint with the old one and reruns a step when they differ. The
mtimeinside it is a file's modification timestamp;touchchanges exactly that value.
When a build script prints no rerun-if-* directives, Cargo uses a conservative default: it watches the files in the entire package. Cargo does not analyze read_dir and read_to_string calls inside an arbitrary Rust program, so it cannot infer those dependencies automatically.
Changing src/main.rs changes the package-wide fingerprint. Cargo runs build.rs, which writes generated.rs again, after which the library containing the generated types and the binary are recompiled.
The generator itself takes only 29.6 ms out of 2.399 s. Most of the cost comes later in the cascade, when rustc receives tens of thousands of generated lines again.
Two Directives Narrow the Fingerprint to the Real Inputs
The build script reads the schema directory and one user-controlled environment variable. Declare exactly those inputs:
println!("cargo:rerun-if-changed=schema/");
println!("cargo:rerun-if-env-changed=PROTOGEN_STRICT");
rerun-if-changedadds a path to the build script's input set, whilererun-if-env-changedadds an environment variable. Cargo remembers their states and runs the script again only after the corresponding input changes.
Now repeat the original experiment:
$ touch src/main.rs
$ time cargo build
Compiling protogen v0.1.0 (…/protogen)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.10s
real 0m0.117s
Three runs took 0.117, 0.118, and 0.121 s. The median fell from 2.399 to 0.118 s.
The fingerprint log now marks only the build target containing src/main.rs as stale:
$ touch src/main.rs
$ CARGO_LOG=cargo::core::compiler::fingerprint=trace cargo build 2>&1 \
| grep -oE 'stale: changed "[^"]*"' | sort -u | sed "s|$PWD|.|g"
stale: changed "./src/main.rs"
build-script-build no longer appears in the verbose output. Cargo preserves generated.rs and does not force the library to recompile its generated types.
The opposite direction matters too: changing a real input must still rerun the generator.
$ touch schema/core.schema
$ time cargo build
Compiling protogen v0.1.0 (…/protogen)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 2.41s
real 0m2.419s
A schema edit still costs about 2.4 s because the generator must run and its output must pass through rustc. The medians of three runs separate eliminated work from necessary work:
Change before cargo build | No directives | Exact inputs |
|---|---|---|
touch src/main.rs | 2.399 s | 0.118 s |
touch schema/core.schema | 2.361 s | 2.348 s |
cargo clean | 20.033 s | 19.528 s |
The directives did not accelerate schema changes or clean builds. They removed only the false dependency on src/main.rs, which cost about 2.3 s after every unrelated edit.
An Incomplete Input List Produces Stale Code
The exact declaration looks simple, which makes it tempting to narrow it even further and name one file:
println!("cargo:rerun-if-changed=schema/core.schema");
An unrelated edit remains fast: touch src/main.rs builds in 0.130 s. By build time alone, this version looks almost identical to the correct one.
But the generator reads the entire directory. Add a field to a different existing file, schema/limits.schema:
$ printf ' retry_budget: u16\n' >> schema/limits.schema
$ cargo build
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.01s
$ ./target/debug/protogen | grep 'LimitConfig'
LimitConfig fields: ["max_batch", "flush_interval_ms", "compression"]
The schema already contains retry_budget, but the compiled LimitConfig does not. Cargo considers everything fresh because the only declared file did not change. No error or warning appears.
Adding a new file falls through the same gap:
$ printf '# Probe records for the v2 wire.\n\nrecord WireProbe\n probe_id: u32\n' \
> schema/wire_v2.schema
$ cargo build
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.01s
$ ./target/debug/protogen | grep 'records generated'
records generated: 643 from 18 schema files
The directory now contains one more file and one more record, but the binary continues reporting the previous values.
The environment variable was omitted as well:
$ PROTOGEN_STRICT=1 cargo build
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.01s
$ ./target/debug/protogen | grep 'strict'
strict mode: false
The new value will be read only the next time build.rs actually runs. Until then, the compiled code stays in the old mode.
As soon as a build script prints at least one rerun-if-* directive, Cargo stops using its package-wide fallback and trusts the declared list. It does not compare that list with the script's actual file reads. Declaring one input makes every forgotten input invisible.
If the only declared file now changes, the generator finally sees all the accumulated edits:
$ touch schema/core.schema
$ time cargo build
Compiling protogen v0.1.0 (…/protogen)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 9.77s
real 0m9.787s
$ ./target/debug/protogen | grep -E 'records generated|LimitConfig'
records generated: 644 from 19 schema files
LimitConfig fields: ["max_batch", "flush_interval_ms", "compression", "retry_budget"]
All missed changes appear at once. This build is more expensive than a normal rerun after touch because generated.rs actually changes, leaving rustc with less previous work to reuse.
An incomplete declaration is worse than no declaration. Without directives, Cargo performs unnecessary work but remains correct. With an incomplete list, it quickly confirms a build whose generated code no longer matches its inputs.
Reading a Directory Requires a Dependency on the Directory
Restore the correct form:
println!("cargo:rerun-if-changed=schema/");
println!("cargo:rerun-if-env-changed=PROTOGEN_STRICT");
The directory path covers three events: editing an existing schema, adding a new one, and removing an old one. The final case can be tested with the wire_v2.schema file created above:
$ rm schema/wire_v2.schema
$ cargo build
Compiling protogen v0.1.0 (…/protogen)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 10.18s
$ ./target/debug/protogen | grep 'records generated'
records generated: 643 from 18 schema files
A list of individual files cannot notice a new name that is not yet on the list. A dependency on the directory notices changes to its contents. A call to read_dir("schema") should therefore correspond to rerun-if-changed=schema/, not to a list of whatever files existed when the script was first written.
The environment variable behaves symmetrically. Once declared with rerun-if-env-changed, both setting and removing it rerun the generator:
$ PROTOGEN_STRICT=1 cargo build
Compiling protogen v0.1.0 (…/protogen)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 17.24s
$ ./target/debug/protogen | grep 'strict'
strict mode: true
$ cargo build
Compiling protogen v0.1.0 (…/protogen)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 16.59s
$ ./target/debug/protogen | grep 'strict'
strict mode: false
These reruns are expensive because the variable changes the generated code, not because checking an environment variable is costly. Strict mode changes all 643 parse functions, so rustc really does receive a different large module. The important part is that this cost appears only when a real input changes, not after an unrelated edit nearby.
build.rs itself does not need to be declared separately. Cargo always tracks the build script's own sources; changing them recompiles and reruns the script.
An Exact Fingerprint Provides Both Speed and Correctness
From the build system's perspective, the generator is a function:
generated.rs = generate(schema/, PROTOGEN_STRICT)
The rerun-if-* directives describe that function's arguments to Cargo. Without directives, Cargo substitutes the entire package. The result stays correct, but unrelated files create false reruns.
With an incomplete list, the fingerprint stops changing with some of the function's real arguments. Cargo faithfully reuses the old result even though the generator would produce a different file from the new inputs.
An exact declaration mirrors the script's actual reads: a directory for read_dir, a path for every individually read file, and rerun-if-env-changed for every environment variable that affects the result. Then an unrelated src/main.rs does not start the generator, while a new schema or generation mode cannot go unnoticed.
The purpose of these directives is not to run build.rs as rarely as possible. It is to run it exactly when its result could have changed.
Appendix: Full Source Files
protogen-naive/build.rs — 291 lines
//! Schema-driven code generator, the way build scripts are usually written.
//!
//! Reads every `.schema` file under `schema/`, parses the record
//! definitions and writes Rust structs with parse functions into
//! `$OUT_DIR/generated.rs`. The crate pulls the result in via `include!`.
//!
//! Deliberately missing: any `cargo:rerun-if-changed` directive. Without
//! it cargo falls back to "rerun when anything in the package changes",
//! so this script runs again after touching a file the schemas never see.
use std::collections::HashSet;
use std::env;
use std::fmt::Write as _;
use std::fs;
use std::path::{Path, PathBuf};
use std::time::Instant;
use anyhow::{Context, Result, ensure};
/// A single `name: type` line inside a record.
struct Field {
name: String,
ty: String,
}
/// One `record Name` block parsed from a schema file.
struct Record {
name: String,
fields: Vec<Field>,
source: String,
}
/// Types the generator accepts on the right-hand side of a field.
const ALLOWED_TYPES: &[&str] = &[
"u8", "u16", "u32", "u64", "i8", "i16", "i32", "i64", "f32", "f64", "bool", "String",
];
fn main() -> Result<()> {
let started = Instant::now();
let manifest_dir = env::var("CARGO_MANIFEST_DIR").context("CARGO_MANIFEST_DIR not set")?;
let out_dir = env::var("OUT_DIR").context("OUT_DIR not set")?;
let schema_dir = Path::new(&manifest_dir).join("schema");
let files = schema_files(&schema_dir)?;
ensure!(!files.is_empty(), "no .schema files in {schema_dir:?}");
let mut records = Vec::new();
for file in &files {
let mut parsed = parse_schema_file(file)?;
records.append(&mut parsed);
}
check_unique_names(&records)?;
// Strict mode makes generated parsers reject unknown keys. The env
// var is read here, at generation time, and baked into the output.
let strict = env::var("PROTOGEN_STRICT").is_ok_and(|value| value == "1");
let code = generate(&records, &files, strict)?;
let target = PathBuf::from(&out_dir).join("generated.rs");
fs::write(&target, code).with_context(|| format!("writing {target:?}"))?;
// Visible in `cargo build -vv` output: proof of every rerun.
println!(
"protogen: {} records from {} schema files in {:?}",
records.len(),
files.len(),
started.elapsed(),
);
Ok(())
}
/// Lists `.schema` files in the schema directory, sorted by name so the
/// generated output is deterministic.
fn schema_files(dir: &Path) -> Result<Vec<PathBuf>> {
let entries = fs::read_dir(dir).with_context(|| format!("reading {dir:?}"))?;
let mut files = Vec::new();
for entry in entries {
let path = entry.context("reading directory entry")?.path();
let is_schema = path.extension().is_some_and(|ext| ext == "schema");
if is_schema {
files.push(path);
}
}
files.sort();
Ok(files)
}
/// Parses one schema file into records.
fn parse_schema_file(path: &Path) -> Result<Vec<Record>> {
let text = fs::read_to_string(path).with_context(|| format!("reading {path:?}"))?;
let source = path
.file_name()
.context("schema path has no file name")?
.to_string_lossy()
.into_owned();
let mut records: Vec<Record> = Vec::new();
for (index, raw_line) in text.lines().enumerate() {
let line = raw_line.trim();
let location = format!("{source}:{}", index + 1);
if line.is_empty() {
continue;
}
if line.starts_with('#') {
continue;
}
if let Some(name) = line.strip_prefix("record ") {
let name = name.trim();
ensure!(is_type_name(name), "{location}: bad record name {name:?}");
records.push(Record {
name: name.to_string(),
fields: Vec::new(),
source: source.clone(),
});
continue;
}
let (name, ty) = line
.split_once(':')
.with_context(|| format!("{location}: expected `name: type`, got {line:?}"))?;
let field = Field {
name: name.trim().to_string(),
ty: ty.trim().to_string(),
};
ensure!(
is_field_name(&field.name),
"{location}: bad field name {:?}",
field.name
);
ensure!(
ALLOWED_TYPES.contains(&field.ty.as_str()),
"{location}: unsupported type {:?}",
field.ty
);
let record = records
.last_mut()
.with_context(|| format!("{location}: field before any `record`"))?;
record.fields.push(field);
}
for record in &records {
ensure!(
!record.fields.is_empty(),
"{source}: record {} has no fields",
record.name
);
}
Ok(records)
}
/// Rejects duplicate record names across all schema files.
fn check_unique_names(records: &[Record]) -> Result<()> {
let mut seen = HashSet::new();
for record in records {
let fresh = seen.insert(record.name.as_str());
ensure!(
fresh,
"duplicate record name {} in {}",
record.name,
record.source
);
}
Ok(())
}
/// `true` for `UpperCamel` identifiers.
fn is_type_name(name: &str) -> bool {
let mut chars = name.chars();
let head_ok = chars.next().is_some_and(|first| first.is_ascii_uppercase());
head_ok && chars.all(|ch| ch.is_ascii_alphanumeric())
}
/// `true` for `lower_snake` identifiers.
fn is_field_name(name: &str) -> bool {
let mut chars = name.chars();
let head_ok = chars.next().is_some_and(|first| first.is_ascii_lowercase());
head_ok && chars.all(|ch| ch.is_ascii_lowercase() || ch.is_ascii_digit() || ch == '_')
}
/// Renders the whole generated module as one string.
fn generate(records: &[Record], files: &[PathBuf], strict: bool) -> Result<String> {
let mut out = String::new();
writeln!(out, "// Generated by build.rs — do not edit by hand.")?;
writeln!(out)?;
writeln!(out, "/// Number of records across all schema files.")?;
writeln!(out, "pub const RECORD_COUNT: usize = {};", records.len())?;
writeln!(out, "/// Schema files the build script consumed, sorted.")?;
write!(out, "pub const SCHEMA_FILES: &[&str] = &[")?;
for file in files {
let name = file
.file_name()
.context("schema path has no file name")?
.to_string_lossy();
write!(out, "{name:?}, ")?;
}
writeln!(out, "];")?;
writeln!(out, "/// Whether generated parsers reject unknown keys.")?;
writeln!(out, "pub const STRICT_MODE: bool = {strict};")?;
for record in records {
render_record(&mut out, record, strict)?;
}
Ok(out)
}
/// Renders one struct plus its `parse` / `field_names` impl.
fn render_record(out: &mut String, record: &Record, strict: bool) -> Result<()> {
writeln!(out)?;
writeln!(out, "/// Generated from `{}`.", record.source)?;
writeln!(out, "#[derive(Debug, Clone, PartialEq)]")?;
writeln!(out, "pub struct {} {{", record.name)?;
for field in &record.fields {
writeln!(out, " pub {}: {},", field.name, field.ty)?;
}
writeln!(out, "}}")?;
writeln!(out)?;
writeln!(out, "impl {} {{", record.name)?;
writeln!(out, " /// Field names in schema order.")?;
writeln!(
out,
" pub fn field_names() -> &'static [&'static str] {{"
)?;
write!(out, " &[")?;
for field in &record.fields {
write!(out, "{:?}, ", field.name)?;
}
writeln!(out, "]")?;
writeln!(out, " }}")?;
writeln!(out)?;
writeln!(
out,
" /// Parses a `key=value;key=value` line into the record."
)?;
writeln!(
out,
" pub fn parse(input: &str) -> Result<Self, String> {{"
)?;
for field in &record.fields {
writeln!(
out,
" let mut {}: Option<{}> = None;",
field.name, field.ty
)?;
}
writeln!(out, " for pair in input.split(';') {{")?;
writeln!(out, " let pair = pair.trim();")?;
writeln!(out, " if pair.is_empty() {{")?;
writeln!(out, " continue;")?;
writeln!(out, " }}")?;
writeln!(out, " let (key, value) = pair")?;
writeln!(out, " .split_once('=')")?;
writeln!(
out,
" .ok_or_else(|| format!(\"bad pair: {{pair}}\"))?;"
)?;
writeln!(out, " match key.trim() {{")?;
for field in &record.fields {
let convert = if field.ty == "String" {
"value.trim().to_string()".to_string()
} else {
format!(
"value.trim().parse::<{}>().map_err(|error| format!(\"{}: {{error}}\"))?",
field.ty, field.name
)
};
writeln!(
out,
" {:?} => {} = Some({convert}),",
field.name, field.name
)?;
}
if strict {
writeln!(
out,
" other => return Err(format!(\"unknown key: {{other}}\")),"
)?;
} else {
writeln!(out, " _other => {{}}")?;
}
writeln!(out, " }}")?;
writeln!(out, " }}")?;
writeln!(out, " Ok(Self {{")?;
for field in &record.fields {
writeln!(
out,
" {}: {}.ok_or(\"missing field: {}\")?,",
field.name, field.name, field.name
)?;
}
writeln!(out, " }})")?;
writeln!(out, " }}")?;
writeln!(out, "}}")?;
Ok(())
}protogen-naive/Cargo.toml — 11 lines
[package]
name = "protogen-naive"
version = "0.1.0"
edition = "2024"
publish = false
[dependencies]
anyhow = "1"
[build-dependencies]
anyhow = "1"protogen-naive/src/lib.rs — 6 lines
//! Types generated at build time from `schema/*.schema`.
//!
//! The whole module body lives in `$OUT_DIR/generated.rs`; the build
//! script writes it, this file only pulls it in.
include!(concat!(env!("OUT_DIR"), "/generated.rs"));protogen-naive/src/main.rs — 21 lines
//! Smoke check for the generated types: parse one wire line per record
//! kind and report what the build script produced.
use anyhow::{Error, Result};
use protogen_naive::{LimitConfig, RECORD_COUNT, SCHEMA_FILES, STRICT_MODE, SensorReading};
fn main() -> Result<()> {
println!(
"records generated: {RECORD_COUNT} from {} schema files",
SCHEMA_FILES.len()
);
println!("strict mode: {STRICT_MODE}");
println!("LimitConfig fields: {:?}", LimitConfig::field_names());
let wire = "device_id=42;channel=3;temperature_mc=21500;\
humidity_ppm=440000;captured_at_ms=1754300000000;healthy=true";
let reading = SensorReading::parse(wire).map_err(Error::msg)?;
println!("parsed: {reading:?}");
Ok(())
}protogen-naive/schema/core.schema — 22 lines
# Core telemetry records the ingest node accepts.
# Format: `record Name` opens a record, indented `name: type` lines
# declare fields, `#` starts a comment.
record SensorReading
device_id: u32
channel: u16
temperature_mc: i32
humidity_ppm: u32
captured_at_ms: u64
healthy: bool
record DeviceEvent
device_id: u32
kind: String
payload: String
emitted_at_ms: u64
record LimitConfig
max_batch: u32
flush_interval_ms: u64
compression: Stringprotogen-precise/build.rs — 293 lines
//! Schema-driven code generator, incremental edition.
//!
//! Same generator as `01-rerun-everything`, one difference: it declares
//! its real inputs. `cargo:rerun-if-changed=schema/` narrows the rerun
//! trigger from "anything in the package" down to the schema directory,
//! and `cargo:rerun-if-env-changed` covers the env var the script reads.
use std::collections::HashSet;
use std::env;
use std::fmt::Write as _;
use std::fs;
use std::path::{Path, PathBuf};
use std::time::Instant;
use anyhow::{Context, Result, ensure};
/// A single `name: type` line inside a record.
struct Field {
name: String,
ty: String,
}
/// One `record Name` block parsed from a schema file.
struct Record {
name: String,
fields: Vec<Field>,
source: String,
}
/// Types the generator accepts on the right-hand side of a field.
const ALLOWED_TYPES: &[&str] = &[
"u8", "u16", "u32", "u64", "i8", "i16", "i32", "i64", "f32", "f64", "bool", "String",
];
fn main() -> Result<()> {
let started = Instant::now();
let manifest_dir = env::var("CARGO_MANIFEST_DIR").context("CARGO_MANIFEST_DIR not set")?;
let out_dir = env::var("OUT_DIR").context("OUT_DIR not set")?;
let schema_dir = Path::new(&manifest_dir).join("schema");
let files = schema_files(&schema_dir)?;
ensure!(!files.is_empty(), "no .schema files in {schema_dir:?}");
let mut records = Vec::new();
for file in &files {
let mut parsed = parse_schema_file(file)?;
records.append(&mut parsed);
}
check_unique_names(&records)?;
// Strict mode makes generated parsers reject unknown keys. The env
// var is read here, at generation time, and baked into the output.
let strict = env::var("PROTOGEN_STRICT").is_ok_and(|value| value == "1");
let code = generate(&records, &files, strict)?;
let target = PathBuf::from(&out_dir).join("generated.rs");
fs::write(&target, code).with_context(|| format!("writing {target:?}"))?;
// The fix: declare the real inputs. Cargo now reruns this script
// only when the schema directory or the declared env var changes.
println!("cargo:rerun-if-changed=schema/");
println!("cargo:rerun-if-env-changed=PROTOGEN_STRICT");
// Visible in `cargo build -vv` output: proof of every rerun.
println!(
"protogen: {} records from {} schema files in {:?}",
records.len(),
files.len(),
started.elapsed(),
);
Ok(())
}
/// Lists `.schema` files in the schema directory, sorted by name so the
/// generated output is deterministic.
fn schema_files(dir: &Path) -> Result<Vec<PathBuf>> {
let entries = fs::read_dir(dir).with_context(|| format!("reading {dir:?}"))?;
let mut files = Vec::new();
for entry in entries {
let path = entry.context("reading directory entry")?.path();
let is_schema = path.extension().is_some_and(|ext| ext == "schema");
if is_schema {
files.push(path);
}
}
files.sort();
Ok(files)
}
/// Parses one schema file into records.
fn parse_schema_file(path: &Path) -> Result<Vec<Record>> {
let text = fs::read_to_string(path).with_context(|| format!("reading {path:?}"))?;
let source = path
.file_name()
.context("schema path has no file name")?
.to_string_lossy()
.into_owned();
let mut records: Vec<Record> = Vec::new();
for (index, raw_line) in text.lines().enumerate() {
let line = raw_line.trim();
let location = format!("{source}:{}", index + 1);
if line.is_empty() {
continue;
}
if line.starts_with('#') {
continue;
}
if let Some(name) = line.strip_prefix("record ") {
let name = name.trim();
ensure!(is_type_name(name), "{location}: bad record name {name:?}");
records.push(Record {
name: name.to_string(),
fields: Vec::new(),
source: source.clone(),
});
continue;
}
let (name, ty) = line
.split_once(':')
.with_context(|| format!("{location}: expected `name: type`, got {line:?}"))?;
let field = Field {
name: name.trim().to_string(),
ty: ty.trim().to_string(),
};
ensure!(
is_field_name(&field.name),
"{location}: bad field name {:?}",
field.name
);
ensure!(
ALLOWED_TYPES.contains(&field.ty.as_str()),
"{location}: unsupported type {:?}",
field.ty
);
let record = records
.last_mut()
.with_context(|| format!("{location}: field before any `record`"))?;
record.fields.push(field);
}
for record in &records {
ensure!(
!record.fields.is_empty(),
"{source}: record {} has no fields",
record.name
);
}
Ok(records)
}
/// Rejects duplicate record names across all schema files.
fn check_unique_names(records: &[Record]) -> Result<()> {
let mut seen = HashSet::new();
for record in records {
let fresh = seen.insert(record.name.as_str());
ensure!(
fresh,
"duplicate record name {} in {}",
record.name,
record.source
);
}
Ok(())
}
/// `true` for `UpperCamel` identifiers.
fn is_type_name(name: &str) -> bool {
let mut chars = name.chars();
let head_ok = chars.next().is_some_and(|first| first.is_ascii_uppercase());
head_ok && chars.all(|ch| ch.is_ascii_alphanumeric())
}
/// `true` for `lower_snake` identifiers.
fn is_field_name(name: &str) -> bool {
let mut chars = name.chars();
let head_ok = chars.next().is_some_and(|first| first.is_ascii_lowercase());
head_ok && chars.all(|ch| ch.is_ascii_lowercase() || ch.is_ascii_digit() || ch == '_')
}
/// Renders the whole generated module as one string.
fn generate(records: &[Record], files: &[PathBuf], strict: bool) -> Result<String> {
let mut out = String::new();
writeln!(out, "// Generated by build.rs — do not edit by hand.")?;
writeln!(out)?;
writeln!(out, "/// Number of records across all schema files.")?;
writeln!(out, "pub const RECORD_COUNT: usize = {};", records.len())?;
writeln!(out, "/// Schema files the build script consumed, sorted.")?;
write!(out, "pub const SCHEMA_FILES: &[&str] = &[")?;
for file in files {
let name = file
.file_name()
.context("schema path has no file name")?
.to_string_lossy();
write!(out, "{name:?}, ")?;
}
writeln!(out, "];")?;
writeln!(out, "/// Whether generated parsers reject unknown keys.")?;
writeln!(out, "pub const STRICT_MODE: bool = {strict};")?;
for record in records {
render_record(&mut out, record, strict)?;
}
Ok(out)
}
/// Renders one struct plus its `parse` / `field_names` impl.
fn render_record(out: &mut String, record: &Record, strict: bool) -> Result<()> {
writeln!(out)?;
writeln!(out, "/// Generated from `{}`.", record.source)?;
writeln!(out, "#[derive(Debug, Clone, PartialEq)]")?;
writeln!(out, "pub struct {} {{", record.name)?;
for field in &record.fields {
writeln!(out, " pub {}: {},", field.name, field.ty)?;
}
writeln!(out, "}}")?;
writeln!(out)?;
writeln!(out, "impl {} {{", record.name)?;
writeln!(out, " /// Field names in schema order.")?;
writeln!(
out,
" pub fn field_names() -> &'static [&'static str] {{"
)?;
write!(out, " &[")?;
for field in &record.fields {
write!(out, "{:?}, ", field.name)?;
}
writeln!(out, "]")?;
writeln!(out, " }}")?;
writeln!(out)?;
writeln!(
out,
" /// Parses a `key=value;key=value` line into the record."
)?;
writeln!(
out,
" pub fn parse(input: &str) -> Result<Self, String> {{"
)?;
for field in &record.fields {
writeln!(
out,
" let mut {}: Option<{}> = None;",
field.name, field.ty
)?;
}
writeln!(out, " for pair in input.split(';') {{")?;
writeln!(out, " let pair = pair.trim();")?;
writeln!(out, " if pair.is_empty() {{")?;
writeln!(out, " continue;")?;
writeln!(out, " }}")?;
writeln!(out, " let (key, value) = pair")?;
writeln!(out, " .split_once('=')")?;
writeln!(
out,
" .ok_or_else(|| format!(\"bad pair: {{pair}}\"))?;"
)?;
writeln!(out, " match key.trim() {{")?;
for field in &record.fields {
let convert = if field.ty == "String" {
"value.trim().to_string()".to_string()
} else {
format!(
"value.trim().parse::<{}>().map_err(|error| format!(\"{}: {{error}}\"))?",
field.ty, field.name
)
};
writeln!(
out,
" {:?} => {} = Some({convert}),",
field.name, field.name
)?;
}
if strict {
writeln!(
out,
" other => return Err(format!(\"unknown key: {{other}}\")),"
)?;
} else {
writeln!(out, " _other => {{}}")?;
}
writeln!(out, " }}")?;
writeln!(out, " }}")?;
writeln!(out, " Ok(Self {{")?;
for field in &record.fields {
writeln!(
out,
" {}: {}.ok_or(\"missing field: {}\")?,",
field.name, field.name, field.name
)?;
}
writeln!(out, " }})")?;
writeln!(out, " }}")?;
writeln!(out, "}}")?;
Ok(())
}protogen-incomplete/build.rs — 294 lines
//! Schema-driven code generator, the trap edition.
//!
//! Same generator as the other two crates, but the directives at the
//! bottom are wrong in the classic way: rerun-if-changed points at ONE
//! schema file, while the script reads the whole directory plus an env
//! var. Cargo believes the declaration, not the actual reads — so edits
//! to `extra.schema`, new schema files and `PROTOGEN_STRICT` changes all
//! produce silently stale generated types.
use std::collections::HashSet;
use std::env;
use std::fmt::Write as _;
use std::fs;
use std::path::{Path, PathBuf};
use std::time::Instant;
use anyhow::{Context, Result, ensure};
/// A single `name: type` line inside a record.
struct Field {
name: String,
ty: String,
}
/// One `record Name` block parsed from a schema file.
struct Record {
name: String,
fields: Vec<Field>,
source: String,
}
/// Types the generator accepts on the right-hand side of a field.
const ALLOWED_TYPES: &[&str] = &[
"u8", "u16", "u32", "u64", "i8", "i16", "i32", "i64", "f32", "f64", "bool", "String",
];
fn main() -> Result<()> {
let started = Instant::now();
let manifest_dir = env::var("CARGO_MANIFEST_DIR").context("CARGO_MANIFEST_DIR not set")?;
let out_dir = env::var("OUT_DIR").context("OUT_DIR not set")?;
let schema_dir = Path::new(&manifest_dir).join("schema");
let files = schema_files(&schema_dir)?;
ensure!(!files.is_empty(), "no .schema files in {schema_dir:?}");
let mut records = Vec::new();
for file in &files {
let mut parsed = parse_schema_file(file)?;
records.append(&mut parsed);
}
check_unique_names(&records)?;
// Strict mode makes generated parsers reject unknown keys. The env
// var is read here, at generation time, and baked into the output.
let strict = env::var("PROTOGEN_STRICT").is_ok_and(|value| value == "1");
let code = generate(&records, &files, strict)?;
let target = PathBuf::from(&out_dir).join("generated.rs");
fs::write(&target, code).with_context(|| format!("writing {target:?}"))?;
// The trap: one file declared, the rest of the inputs forgotten.
// From here on cargo trusts this line and ignores everything else.
println!("cargo:rerun-if-changed=schema/base.schema");
// Visible in `cargo build -vv` output: proof of every rerun.
println!(
"protogen: {} records from {} schema files in {:?}",
records.len(),
files.len(),
started.elapsed(),
);
Ok(())
}
/// Lists `.schema` files in the schema directory, sorted by name so the
/// generated output is deterministic.
fn schema_files(dir: &Path) -> Result<Vec<PathBuf>> {
let entries = fs::read_dir(dir).with_context(|| format!("reading {dir:?}"))?;
let mut files = Vec::new();
for entry in entries {
let path = entry.context("reading directory entry")?.path();
let is_schema = path.extension().is_some_and(|ext| ext == "schema");
if is_schema {
files.push(path);
}
}
files.sort();
Ok(files)
}
/// Parses one schema file into records.
fn parse_schema_file(path: &Path) -> Result<Vec<Record>> {
let text = fs::read_to_string(path).with_context(|| format!("reading {path:?}"))?;
let source = path
.file_name()
.context("schema path has no file name")?
.to_string_lossy()
.into_owned();
let mut records: Vec<Record> = Vec::new();
for (index, raw_line) in text.lines().enumerate() {
let line = raw_line.trim();
let location = format!("{source}:{}", index + 1);
if line.is_empty() {
continue;
}
if line.starts_with('#') {
continue;
}
if let Some(name) = line.strip_prefix("record ") {
let name = name.trim();
ensure!(is_type_name(name), "{location}: bad record name {name:?}");
records.push(Record {
name: name.to_string(),
fields: Vec::new(),
source: source.clone(),
});
continue;
}
let (name, ty) = line
.split_once(':')
.with_context(|| format!("{location}: expected `name: type`, got {line:?}"))?;
let field = Field {
name: name.trim().to_string(),
ty: ty.trim().to_string(),
};
ensure!(
is_field_name(&field.name),
"{location}: bad field name {:?}",
field.name
);
ensure!(
ALLOWED_TYPES.contains(&field.ty.as_str()),
"{location}: unsupported type {:?}",
field.ty
);
let record = records
.last_mut()
.with_context(|| format!("{location}: field before any `record`"))?;
record.fields.push(field);
}
for record in &records {
ensure!(
!record.fields.is_empty(),
"{source}: record {} has no fields",
record.name
);
}
Ok(records)
}
/// Rejects duplicate record names across all schema files.
fn check_unique_names(records: &[Record]) -> Result<()> {
let mut seen = HashSet::new();
for record in records {
let fresh = seen.insert(record.name.as_str());
ensure!(
fresh,
"duplicate record name {} in {}",
record.name,
record.source
);
}
Ok(())
}
/// `true` for `UpperCamel` identifiers.
fn is_type_name(name: &str) -> bool {
let mut chars = name.chars();
let head_ok = chars.next().is_some_and(|first| first.is_ascii_uppercase());
head_ok && chars.all(|ch| ch.is_ascii_alphanumeric())
}
/// `true` for `lower_snake` identifiers.
fn is_field_name(name: &str) -> bool {
let mut chars = name.chars();
let head_ok = chars.next().is_some_and(|first| first.is_ascii_lowercase());
head_ok && chars.all(|ch| ch.is_ascii_lowercase() || ch.is_ascii_digit() || ch == '_')
}
/// Renders the whole generated module as one string.
fn generate(records: &[Record], files: &[PathBuf], strict: bool) -> Result<String> {
let mut out = String::new();
writeln!(out, "// Generated by build.rs — do not edit by hand.")?;
writeln!(out)?;
writeln!(out, "/// Number of records across all schema files.")?;
writeln!(out, "pub const RECORD_COUNT: usize = {};", records.len())?;
writeln!(out, "/// Schema files the build script consumed, sorted.")?;
write!(out, "pub const SCHEMA_FILES: &[&str] = &[")?;
for file in files {
let name = file
.file_name()
.context("schema path has no file name")?
.to_string_lossy();
write!(out, "{name:?}, ")?;
}
writeln!(out, "];")?;
writeln!(out, "/// Whether generated parsers reject unknown keys.")?;
writeln!(out, "pub const STRICT_MODE: bool = {strict};")?;
for record in records {
render_record(&mut out, record, strict)?;
}
Ok(out)
}
/// Renders one struct plus its `parse` / `field_names` impl.
fn render_record(out: &mut String, record: &Record, strict: bool) -> Result<()> {
writeln!(out)?;
writeln!(out, "/// Generated from `{}`.", record.source)?;
writeln!(out, "#[derive(Debug, Clone, PartialEq)]")?;
writeln!(out, "pub struct {} {{", record.name)?;
for field in &record.fields {
writeln!(out, " pub {}: {},", field.name, field.ty)?;
}
writeln!(out, "}}")?;
writeln!(out)?;
writeln!(out, "impl {} {{", record.name)?;
writeln!(out, " /// Field names in schema order.")?;
writeln!(
out,
" pub fn field_names() -> &'static [&'static str] {{"
)?;
write!(out, " &[")?;
for field in &record.fields {
write!(out, "{:?}, ", field.name)?;
}
writeln!(out, "]")?;
writeln!(out, " }}")?;
writeln!(out)?;
writeln!(
out,
" /// Parses a `key=value;key=value` line into the record."
)?;
writeln!(
out,
" pub fn parse(input: &str) -> Result<Self, String> {{"
)?;
for field in &record.fields {
writeln!(
out,
" let mut {}: Option<{}> = None;",
field.name, field.ty
)?;
}
writeln!(out, " for pair in input.split(';') {{")?;
writeln!(out, " let pair = pair.trim();")?;
writeln!(out, " if pair.is_empty() {{")?;
writeln!(out, " continue;")?;
writeln!(out, " }}")?;
writeln!(out, " let (key, value) = pair")?;
writeln!(out, " .split_once('=')")?;
writeln!(
out,
" .ok_or_else(|| format!(\"bad pair: {{pair}}\"))?;"
)?;
writeln!(out, " match key.trim() {{")?;
for field in &record.fields {
let convert = if field.ty == "String" {
"value.trim().to_string()".to_string()
} else {
format!(
"value.trim().parse::<{}>().map_err(|error| format!(\"{}: {{error}}\"))?",
field.ty, field.name
)
};
writeln!(
out,
" {:?} => {} = Some({convert}),",
field.name, field.name
)?;
}
if strict {
writeln!(
out,
" other => return Err(format!(\"unknown key: {{other}}\")),"
)?;
} else {
writeln!(out, " _other => {{}}")?;
}
writeln!(out, " }}")?;
writeln!(out, " }}")?;
writeln!(out, " Ok(Self {{")?;
for field in &record.fields {
writeln!(
out,
" {}: {}.ok_or(\"missing field: {}\")?,",
field.name, field.name, field.name
)?;
}
writeln!(out, " }})")?;
writeln!(out, " }}")?;
writeln!(out, "}}")?;
Ok(())
}protogen-incomplete/schema/base.schema — 15 lines
# Wire records, part one. The build script declares THIS file
# in rerun-if-changed — and only this file.
record SensorReading
device_id: u32
channel: u16
temperature_mc: i32
captured_at_ms: u64
healthy: bool
record DeviceEvent
device_id: u32
kind: String
payload: String
emitted_at_ms: u64protogen-incomplete/schema/extra.schema — 8 lines
# Wire records, part two. The build script reads this file as well —
# but forgot to declare it in rerun-if-changed. Edits here are invisible
# to cargo: the generated types silently go stale.
record LimitConfig
max_batch: u32
flush_interval_ms: u64
compression: String