Skip to content

toolchain: ABI-pin host compiler to g++-15 under a sanitizer#949

Open
ChaoWao wants to merge 1 commit into
hw-native-sys:mainfrom
ChaoWao:fix/sanitizer-host-cxx-abi-pin
Open

toolchain: ABI-pin host compiler to g++-15 under a sanitizer#949
ChaoWao wants to merge 1 commit into
hw-native-sys:mainfrom
ChaoWao:fix/sanitizer-host-cxx-abi-pin

Conversation

@ChaoWao
Copy link
Copy Markdown
Collaborator

@ChaoWao ChaoWao commented May 31, 2026

Summary

A real build-correctness bug found while making the TSAN nightly actually run:
under a sanitizer the sim host artifacts were not reliably compiled with
g++-15, so they linked a different sanitizer runtime than the run-step
preloads — and TSAN failed to even start.

  • GxxToolchain(prefer_g15=True) exists to force g++-15 so every host .so
    shares the one sanitizer runtime that pytest/CI preloads
    (LD_PRELOAD=$(g++-15 -print-file-name=libtsan.so) → libtsan.so.2).
  • But _host_compiler_cmake_args read CC/CXX from the environment and let
    them override the prefer_g15 default (the conda-compat path).
    scikit-build-core exports CXX during pip install, so the sanitized
    .so were built with the env compiler (system g++ = gcc-11 → libtsan.so.0)
    while the run-step preloaded g++-15's libtsan.so.2.
  • That version split fails at dlopen with
    cannot allocate memory in static TLS block → TSAN never runs.

Fix

Add a pin_compiler mode to _host_compiler_cmake_args: under prefer_g15
the env CC/CXX binary is ignored (forced to gcc-15/g++-15), while
any env-injected flags (conda's -B …/compiler_compat) are still
preserved. Non-sanitizer builds are unchanged.

Validation (Linux/aarch64 docker)

  • Repro: with CXX=g++ exported + --sanitizer tsan, built .so linked
    libtsan.so.0 (gcc-11) → dlopen TLS failure.
  • After fix: all 14 sim .so link libtsan.so.2; the prepared_callable
    TSAN run starts and reports races instead of crashing at import.
  • pip install path verified end-to-end (scikit-build-core exports CXX).

Testing

  • New regression tests in tests/ut/py/test_toolchain.py:
    prefer_g15=True with CXX=g++ still emits -DCMAKE_CXX_COMPILER=g++-15,
    and conda -B compiler_compat flags survive the pin.
  • Unit tests pass (34/34 toolchain + sanitizer)
  • Simulation tests pass
  • Hardware tests (n/a — host-compiler build logic only)

Under a sanitizer the sim host artifacts (runtime, kernels, orchestration)
must all link the SAME sanitizer runtime that the pytest/CI run-step
preloads via LD_PRELOAD — g++-15's libtsan.so.2 / libasan.so. GxxToolchain
carries `prefer_g15` precisely to guarantee that.

But `_host_compiler_cmake_args` read CC/CXX from the environment and let them
OVERRIDE the prefer_g15 default (the conda-compat path). scikit-build-core
exports CXX during `pip install`, so the sanitized .so were built with the
env compiler (e.g. system g++ = gcc-11, libtsan.so.0) while the run-step
preloaded g++-15's libtsan.so.2 — a version split that fails at dlopen with
"cannot allocate memory in static TLS block", so TSAN never even runs.

Add a `pin_compiler` mode: under prefer_g15 the env CC/CXX *binary* is
ignored (forced to gcc-15/g++-15) while any env-injected flags (conda's
-B compiler_compat) are still preserved. Non-sanitizer builds are unchanged.

Regression test: GxxToolchain(prefer_g15=True) with CXX=g++ in the env still
emits -DCMAKE_CXX_COMPILER=g++-15, and conda flags survive the pin.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 31, 2026

Warning

Review limit reached

@ChaoWao, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 23 minutes and 42 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: e5f32081-a428-4564-8179-7b5ad94c4f76

📥 Commits

Reviewing files that changed from the base of the PR and between 0e2e7cd and 9ebf4b8.

📒 Files selected for processing (2)
  • simpler_setup/toolchain.py
  • tests/ut/py/test_toolchain.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a compiler pinning mechanism (pin_compiler) under sanitizer builds to ensure the host compiler is ABI-pinned to gcc-15/g++-15, preventing environment overrides while preserving injected flags. The review feedback suggests improving this by preserving custom absolute or relative paths to gcc-15/g++-15 (e.g., /usr/local/bin/gcc-15) instead of unconditionally overriding them with the default binary names, and adding a corresponding unit test to verify this behavior.

Comment on lines +75 to +76
if pin_compiler:
cc, cxx = default_cc, default_cxx
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

If pin_compiler is enabled, unconditionally overriding cc and cxx to default_cc and default_cxx (e.g., "gcc-15" and "g++-15") will discard any custom absolute or relative paths specified by the user in CC/CXX (for example, /usr/local/bin/gcc-15 or /opt/toolchains/bin/g++-15). If these custom paths are not in the system's default PATH, the build will fail.

We should preserve the user's custom path if it already points to a version 15 compiler by checking if "gcc-15" / "g++-15" is part of the executable's basename.

    if pin_compiler:
        if "gcc-15" not in os.path.basename(cc):
            cc = default_cc
        if "g++-15" not in os.path.basename(cxx):
            cxx = default_cxx

Comment on lines +115 to +116
assert "-DCMAKE_C_FLAGS=-pthread -B /data/envs/lyf/compiler_compat" in args
assert "-DCMAKE_CXX_FLAGS=-pthread -B /data/envs/lyf/compiler_compat" in args
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Add a unit test to verify that custom paths to gcc-15 and g++-15 are preserved when pin_compiler is enabled.

Suggested change
assert "-DCMAKE_C_FLAGS=-pthread -B /data/envs/lyf/compiler_compat" in args
assert "-DCMAKE_CXX_FLAGS=-pthread -B /data/envs/lyf/compiler_compat" in args
assert "-DCMAKE_C_FLAGS=-pthread -B /data/envs/lyf/compiler_compat" in args
assert "-DCMAKE_CXX_FLAGS=-pthread -B /data/envs/lyf/compiler_compat" in args
def test_custom_path_to_g15_is_preserved(self, toolchain, monkeypatch):
# Custom paths to gcc-15/g++-15 must be preserved to support non-standard installation paths.
monkeypatch.setenv("CC", "/opt/custom/bin/gcc-15")
monkeypatch.setenv("CXX", "/opt/custom/bin/g++-15")
args = toolchain.get_cmake_args()
assert "-DCMAKE_C_COMPILER=/opt/custom/bin/gcc-15" in args
assert "-DCMAKE_CXX_COMPILER=/opt/custom/bin/g++-15" in args

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant