Skip to content

fix: point pyvenv.cfg home= directly at PBS bin/ to fix sys.base_prefix on Python 3.11/3.12#1058

Open
xangcastle wants to merge 2 commits into
mainfrom
xangcastle/issue-1048
Open

fix: point pyvenv.cfg home= directly at PBS bin/ to fix sys.base_prefix on Python 3.11/3.12#1058
xangcastle wants to merge 2 commits into
mainfrom
xangcastle/issue-1048

Conversation

@xangcastle
Copy link
Copy Markdown
Member

@xangcastle xangcastle commented May 29, 2026

Fix #1048

Problem: #1048

Python 3.11/3.12 reimplemented prefix discovery as pure Python (Modules/getpath.py).
Its resolvedpath() correctly handles one-level local symlinks but has a bug with
multi-hop relative symlinks that traverse .. components across repository boundaries —
exactly the shape that venv/bin/python produces:

venv/bin/python → ../../../../interpreter_repo/bin/python3.11

When resolution fails, Python falls back to the PBS compile-time prefix /install,
which does not exist at runtime:

ModuleNotFoundError: No module named 'encodings'

Python 3.9/3.10 used C code (correct). Python 3.13 fixed the bug. Only 3.11 and 3.12
are affected.

Fix

Change pyvenv.cfg's home = key for hermetic (runfiles-based) interpreters from
./bin/ to a relative path that points directly at the PBS bin/ directory:

home = ../../../../aspect_rules_py++python_interpreters+.../bin

This means getpath.py only needs to resolve bin/python → python3.11 — a single
local hop within the PBS package — which works correctly in all Python versions.

System (non-hermetic) interpreters keep home = ./bin unchanged.

Changes

  • py/private/py_venv/venv.bzl — compute pyvenv_home from the interpreter's
    rlocation path for hermetic toolchains; fall back to ./bin for system interpreters
  • e2e/tools/verify_venv/verify_venv.py — add verify_base_prefix() asserting
    sys.base_prefix != "/install" and that the stdlib dir exists; call it from
    verify_all()
  • e2e/cases/pbs-symlink-prefix-1048/ — regression tests for Python 3.11 and 3.12
  • OCI layer snapshots updated to reflect the new pyvenv.cfg and verify_venv.py sizes

@aspect-workflows
Copy link
Copy Markdown

aspect-workflows Bot commented May 29, 2026

✨ Aspect Workflows Tasks

📅 Fri May 29 09:52:09 UTC 2026

✅ 7 successful tasks

  • ✅ buildifier · ⏱ 19.7s · 🐙 GitHub Actions · ☑️ Check
    💬 Format complete (clean)
  • ✅ gazelle · ⏱ 22s · 🐙 GitHub Actions · ☑️ Check
    💬 Gazelle complete (clean)
  • ✅ test (test-e2e-bazel-8) · ⏱ 1m 30s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (139/139 passed · 131 cached)
  • ✅ test (test-e2e-bazel-9) · ⏱ 1m 20s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (134/134 passed · 128 cached)
  • ✅ test (test-examples-uv_pip_compile-bazel-8) · ⏱ 19.4s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (1/1 passed · 1 cached)
  • ✅ test (test-root-bazel-8) · ⏱ 2m 37s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (185/185 passed · 185 cached)
  • ✅ test (test-root-bazel-9) · ⏱ 2m 9s · 🐙 GitHub Actions · ☑️ Check
    💬 Bazel test complete (184/184 passed · 184 cached)

⏱ Last updated Fri May 29 09:55:30 UTC 2026 · 📊 GitHub API quota 4,316/15,000 (29% used, resets in 14m)
🚀 Powered by Aspect CLI (v2026.22.17)  |  Aspect Build · X · LinkedIn · YouTube

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 29, 2026

py_binary startup benchmark

Version Mean (ms) Median (ms) ± stddev vs BCR vs main Build (s)
BCR 1.11.5 (baseline) 304.726 302.473 ±7.580 35.49
HEAD main 124.281 123.775 ±2.522 -59.2% 14.49
This PR 125.791 124.519 ±3.043 -58.7% +1.2% 11.64

Measured with hyperfine --warmup 5 --runs 50 on Linux
Gate: PR vs HEAD main (threshold: 10%). BCR is shown only as a historical baseline.
Build time: cold bazel build //:bench with isolated output base, no disk cache.

sys.path quality

Version sys.path entries distinct site-packages roots duplicate realpaths
BCR 1.11.5 (baseline) 6 1 0
HEAD main 7 2 0
This PR 7 2 0

sys.path quality measured by bench_syspath inside the assembled venv. Duplicate realpaths indicate symlink redundancy; many distinct site-packages roots suggest an inefficient venv layout.

@xangcastle xangcastle requested a review from jbedard May 29, 2026 10:02
@xangcastle xangcastle marked this pull request as ready for review May 29, 2026 10:02
@xangcastle
Copy link
Copy Markdown
Member Author

@ctcjab please take a look

@@ -0,0 +1,48 @@
"""Regression test for issue #1048: PBS symlink chain breaks sys.base_prefix.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this test (and associated BUILD) simple enough to just put into py/tests as a unit test?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: v2.0.0-alpha.1 breaks sys.base_prefix for Python 3.11/3.12 with native extensions

2 participants