Skip to content

Switch pipelining metadata action to hollow rlib (-Zno-codegen)#3870

Open
Walter-Reactor wants to merge 1 commit intobazelbuild:mainfrom
Walter-Reactor:user/wgray/pipelining
Open

Switch pipelining metadata action to hollow rlib (-Zno-codegen)#3870
Walter-Reactor wants to merge 1 commit intobazelbuild:mainfrom
Walter-Reactor:user/wgray/pipelining

Conversation

@Walter-Reactor
Copy link

Replace the --rustc-quit-on-rmeta / .rmeta approach with Buck2-style hollow rlibs: the RustcMetadata action runs rustc to completion with -Zno-codegen, emitting a .rlib archive (named -hollow.rlib) that contains only metadata and optimized MIR.

Key fixes:

  • Main Rustc action uses full rlib --extern deps so the SVH embedded in the full rlib matches what downstream binaries expect (avoiding E0460 with non-deterministic proc macros).
  • RUSTC_BOOTSTRAP=1 is set on both actions: it changes the crate hash, so inconsistent use would cause SVH mismatch even for deterministic crates.
  • -Ldependency= points to the _hollow/ subdirectory so the full rlib and hollow rlib never appear in the same search path (avoids E0463).
  • transitive_metadata_outputs are always included in the sandbox so rustc can resolve transitive -Ldependency= references.

Adds an SVH mismatch test that demonstrates the problem with non-deterministic proc macros (uses a HashMap-based derive macro whose output varies by OS-seeded random seed).

I've been using this locally, and it seems to be a notable improvement - I've been able to remove all disable_pipelining=True attributes and enable pipelining globally while still having good performance

Replace the --rustc-quit-on-rmeta / .rmeta approach with Buck2-style
hollow rlibs: the RustcMetadata action runs rustc to completion with
-Zno-codegen, emitting a .rlib archive (named -hollow.rlib) that
contains only metadata and optimized MIR.

Key fixes:
- Main Rustc action uses full rlib --extern deps so the SVH embedded
  in the full rlib matches what downstream binaries expect (avoiding
  E0460 with non-deterministic proc macros).
- RUSTC_BOOTSTRAP=1 is set on both actions: it changes the crate hash,
  so inconsistent use would cause SVH mismatch even for deterministic
  crates.
- -Ldependency= points to the _hollow/ subdirectory so the full rlib
  and hollow rlib never appear in the same search path (avoids E0463).
- transitive_metadata_outputs are always included in the sandbox so
  rustc can resolve transitive -Ldependency= references.

Adds an SVH mismatch test that demonstrates the problem with
non-deterministic proc macros (uses a HashMap-based derive macro whose
output varies by OS-seeded random seed).
@google-cla
Copy link

google-cla bot commented Feb 26, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant