提前生成元数据可使 Rust 的构建和检查速度最高提升至两倍。
Emitting metadata early makes building/checking Rust up to twice as fast

原始链接: https://github.com/PowderworksCode/headstart

Headstart 是一种用于 rustc/Cargo 的原型优化,可让依赖 crate 在其依赖项完成函数体检查之前就开始类型检查。当 crate 的接口有效后,rustc 会提前生成 `.rmeta` 文件;Cargo 会使用这份元数据启动依赖 crate 的编译,同时继续并发检查原 crate 的函数体。 运行 `cargo check` 时,依赖 crate 可以直接使用提前生成的元数据完成检查。运行 `cargo build` 时,依赖 crate 会先完成分析,然后暂停并释放任务槽位,直到完整元数据可用后再进行代码生成。依赖项中的错误仍会导致构建失败,并产生等效的诊断信息和退出状态,但执行时机、进度消息及 JSON 输出顺序可能有所不同。其代价包括丢弃下游工作、延迟报告错误以及增加峰值内存占用。 在 16 核机器上,使用默认前端进行全新构建时,`check` 的加速最高可达 54%,`build` 最高可达 42%;并行前端可获得最高 25% 的加速。在核心数较少的机器上,收益会减小或没有明显提升。 该仓库提供 rustc 和 Cargo 的补丁系列、设计与基准测试文档、冒烟测试、错误测试、增量测试、元数据切换测试、rustc-perf 验证,以及用于真实项目基准测试的脚本。使用 `scripts/setup.sh` 构建打过补丁的工具后,可通过 `-Zheadstart`、`CARGO_UNSTABLE_HEADSTART=true` 或相应的 unstable 配置项来运行 Cargo。

Hacker News 最新 | 往期 | 评论 | 提问 | 展示 | 招聘 | 提交 登录 提前生成元数据可使 Rust 的构建和检查速度最高提升至两倍 ( github.com/powderworkscode ) 8 分 由 knuckleheads 发布 4 小时前 | 隐藏 | 往期 | 收藏 | 讨论 帮助 考虑申请 YC 2027 年冬季批次! 申请开放至 11 月 2 日。 指南 | 常见问题 | 列表 | API | 安全 | 法律 | 申请加入 YC | 联系我们 搜索:
相关文章

原文

Start dependent crates before their dependencies finish type-checking.

Every crate waits for the crates it depends on to be fully checked, function bodies included, before it starts. It doesn't need those bodies to type-check itself. It compiles against the dependency's interface, the metadata in its .rmeta file.

Headstart makes rustc write an early metadata file as soon as the interface is checked, and makes cargo start dependents on it. Each crate's bodies are then checked while the crates downstream are already compiling.

  • cargo check: dependents run to completion on early metadata.
  • cargo build: dependents do all their analysis on early metadata, then wait for the dependency's full metadata before generating code. While they wait, they give their job slot back.

If a body has an error, the build still fails with that error, with the same diagnostics and exit status as today; only progress lines and the cross-crate order of JSON messages can differ. Cargo reports a crate's output only once all its dependencies have finished cleanly, and drops it if one fails. The costs are work downstream that gets thrown away, errors reported slightly later, and more memory in use at once (see docs/design.md).

  • rustc, -Zearly-metadata (6 patches):
    • a new analysis_interfaces query splits analysis into item interfaces and function bodies;
    • the driver writes .early-rmeta between the two;
    • crate loading accepts early metadata, and swaps in full metadata before code generation, waiting for it if necessary on a lock its producer holds until it's written.
  • cargo, -Zheadstart (3 patches):
    • passes -Zearly-metadata to every compile;
    • starts dependents on the early-metadata notification, in both check and build;
    • gives a paused compilation's job slot to other work;
    • reports a crate's output only when its dependencies succeeded.

The patches are a commit series, each with a commit message and tests, meant to become upstream pull requests: see patches/README.md.

On rustc's default front end, headstart makes clean builds of 13 real projects (rust-analyzer, zed, bevy, lemmy, polars and others) up to 54% faster for cargo check, and up to 42% for cargo build. None is slower. With the parallel front end (-Zthreads=8), which covers some of the same ground, it adds up to 25%. Those are 16-core numbers. The gain comes from cores the build would leave idle, so it shrinks on smaller machines. On 4 cores, rust-analyzer's check is 24% faster and its build 13–15%, codex-rs's check 14%, and wide builds come out even.

How it works, what early metadata leaves out, and the risks: docs/design.md. Measurements: docs/results.md. Whether it's ready to bring to the compiler and cargo teams: docs/readiness.md.

scripts/setup.sh    # check out rustc + cargo, apply the patches, build both

Then, in any Rust project:

RUSTC=/path/to/headstart/rustc/build/host/stage1/bin/rustc \
  /path/to/headstart/cargo/target/release/cargo check -Zheadstart   # or build

CARGO_UNSTABLE_HEADSTART=true turns it on too, as does [unstable] headstart = true in .cargo/config.toml. Without it, the patched cargo behaves like upstream, so the same binaries give a fair baseline.

tests/smoke is a two-crate workspace that shows the effect. Its slow library takes several seconds to check, almost all of it in function bodies. With headstart on, app starts about 0.2 s in instead of waiting for slow to finish.

scripts/check-errors.sh checks the claim about errors on tests/errors. It runs three scenarios (a clean build, an error in a dependency, an error in the binary) with cargo check and cargo build, headstart off and on. It then compares the human-readable output, the JSON output, the exit status and what the built binary prints.

scripts/check-incremental.sh [check|build] does the same across a sequence of incremental edits. The steps include adding an impl Fn a dependent calls, and breaking and then fixing an interface. It also compares the final state against a clean build.

scripts/check-swap.sh makes a library start on its dependency's early metadata and swap in the full metadata while paused, at every optimization level. The program built from it must print the same as one built from full metadata.

scripts/sweep.sh builds all 53 rustc-perf compile benchmarks with headstart off and on, with -Zearly-metadata-verify. It passes when every build succeeds in both modes with the same diagnostics, and verify reports nothing. -c build sweeps cargo build, -r the release profile, and -t the parallel front end (-Zthreads=8).

scripts/bench.sh -n 5 [-c build] path/to/project ...

This times clean cargo check (or cargo build) builds, alternating headstart off and on, and prints the medians. A project can take cargo arguments after :: (path/to/vaultwarden::--features=sqlite). `scripts/real-projects.sh

` clones the 13 real projects from [docs/results.md](docs/results.md) at the commits measured, and prints them in that form:
scripts/bench.sh -n 3 -c build $(scripts/real-projects.sh ~/hs-real)

scripts/setup-codex.sh <dir> does the same for codex-rs, which needs a patched dependency and codex's prebuilt V8 (see the script); source <dir>/codex/headstart.env before timing it.

-w adds an untimed warm-up build per project, for build scripts that do one-time work outside target (helix compiles its grammars into its source tree).

scripts/bench-suite.sh <out-dir> runs the whole suite of docs/results.md on the current machine: rust-analyzer, the 21 rustc-perf benchmarks, the other real projects and codex-rs, check and build. It keeps each benchmark's results separately and skips finished ones, so it can be restarted, and writes a summary table at the end. It's how to get the numbers for a machine size not measured yet, such as 8 cores.

scripts/bench-mem.sh samples the total memory of all rustc processes during a build, in the project directory itself (run it on an otherwise idle machine). scripts/bench-incremental.sh times incremental rechecks after editing one function body. scripts/log-rustc records when each rustc run started and ended, so you can see the schedule. The rustc-perf benchmarks are under rustc/src/tools/rustc-perf/collector/compile-benchmarks (git -C rustc submodule update --init --depth 1 src/tools/rustc-perf).

联系我们 contact @ memedata.com