Missing pip dependencies with venvs_site_packages=yes

[nikonikolov]

🐞 bug report

Affected Rule

The issue is caused by the rule: `py_test`

Is this a regression?

No

Description

Pip dependencies are missing for py_test targets.

1. Ex: pytest pip package depends on py pip package. Adding pytest to the deps for py_test doesn't include its transitive dependencies, e.g. py (and others). It does include some transitive dependencies though, e.g. pluggy
2. In the above example, adding py as an explicit dependency in deps fixes the problem as it gets included in the generated venv. However, this 'fix' doesn't work for iniconfig, another missing transitive dependency of pytest - it doesn't get included in the venv

🔬 Minimal Reproduction

🔥 Exception or Error

Traceback (most recent call last):                                                                                                                                                                                                          
  File "/scratch/nikolay_nikolov/.cache/bazel/_bazel_nikolay_nikolov/79bf5e678fbb2019f1e30944a206f079/sandbox/linux-sandbox/890/execroot/_main/bazel-out/k8-opt/bin/barrel/core/common/logger/logger_tests_py_test.runfiles/_main/barrel/cor
e/common/logger/_logger_tests_py_test_actual_stage2_bootstrap.py", line 537, in                                                                                                                                                     
    main()                                                                                                                                                                                                                                  
  File "/scratch/nikolay_nikolov/.cache/bazel/_bazel_nikolay_nikolov/79bf5e678fbb2019f1e30944a206f079/sandbox/linux-sandbox/890/execroot/_main/bazel-out/k8-opt/bin/barrel/core/common/logger/logger_tests_py_test.runfiles/_main/barrel/cor
e/common/logger/_logger_tests_py_test_actual_stage2_bootstrap.py", line 531, in main                                                                                                                                                        
    _run_py_path(main_filename, args=sys.argv[1:])                                                                                                                                                                                          
  File "/scratch/nikolay_nikolov/.cache/bazel/_bazel_nikolay_nikolov/79bf5e678fbb2019f1e30944a206f079/sandbox/linux-sandbox/890/execroot/_main/bazel-out/k8-opt/bin/barrel/core/common/logger/logger_tests_py_test.runfiles/_main/barrel/cor
e/common/logger/_logger_tests_py_test_actual_stage2_bootstrap.py", line 319, in _run_py_path                                                                                                                                                
    runpy.run_path(main_filename, run_name="__main__")                                                                                                                                                                                      
  File "/scratch/nikolay_nikolov/.cache/bazel/_bazel_nikolay_nikolov/79bf5e678fbb2019f1e30944a206f079/execroot/_main/external/rules_python++python+python_3_10_x86_64-unknown-linux-gnu/lib/python3.10/runpy.py", line 289, in run_path     
    return _run_module_code(code, init_globals, run_name,                                                                                                                                                                                   
  File "/scratch/nikolay_nikolov/.cache/bazel/_bazel_nikolay_nikolov/79bf5e678fbb2019f1e30944a206f079/execroot/_main/external/rules_python++python+python_3_10_x86_64-unknown-linux-gnu/lib/python3.10/runpy.py", line 96, in _run_module_co
de                                                                                                                                                                                                                                          
    _run_code(code, mod_globals, init_globals,                                                                                                                                                                                              
  File "/scratch/nikolay_nikolov/.cache/bazel/_bazel_nikolay_nikolov/79bf5e678fbb2019f1e30944a206f079/execroot/_main/external/rules_python++python+python_3_10_x86_64-unknown-linux-gnu/lib/python3.10/runpy.py", line 86, in _run_code     
    exec(code, run_globals)                                                                                                                                                                                                                 
  File "/scratch/nikolay_nikolov/.cache/bazel/_bazel_nikolay_nikolov/79bf5e678fbb2019f1e30944a206f079/sandbox/linux-sandbox/890/execroot/_main/bazel-out/k8-opt/bin/barrel/core/common/logger/logger_tests_py_test.runfiles/_main/build_supp
ort/python/pytest_proxy.py", line 7, in                                                                                                                                                                                             
    from pytest import console_main                                                                                                                                                                                                         
  File "/scratch/nikolay_nikolov/.cache/bazel/_bazel_nikolay_nikolov/79bf5e678fbb2019f1e30944a206f079/sandbox/linux-sandbox/890/execroot/_main/bazel-out/k8-opt/bin/barrel/core/common/logger/logger_tests_py_test.runfiles/_main/barrel/cor
e/common/logger/_logger_tests_py_test_actual.venv/lib/python3.10/site-packages/pytest/__init__.py", line 6, in                                                                                                                      
    from _pytest.assertion import register_assert_rewrite                                                                                                                                                                                   
  File "/scratch/nikolay_nikolov/.cache/bazel/_bazel_nikolay_nikolov/79bf5e678fbb2019f1e30944a206f079/sandbox/linux-sandbox/890/execroot/_main/bazel-out/k8-opt/bin/barrel/core/common/logger/logger_tests_py_test.runfiles/_main/barrel/cor
e/common/logger/_logger_tests_py_test_actual.venv/lib/python3.10/site-packages/_pytest/assertion/__init__.py", line 9, in                                                                                                           
    from _pytest.assertion import rewrite                                                                                                                                                                                                   
  File "/scratch/nikolay_nikolov/.cache/bazel/_bazel_nikolay_nikolov/79bf5e678fbb2019f1e30944a206f079/sandbox/linux-sandbox/890/execroot/_main/bazel-out/k8-opt/bin/barrel/core/common/logger/logger_tests_py_test.runfiles/_main/barrel/cor
e/common/logger/_logger_tests_py_test_actual.venv/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 34, in                                                                                                           
    from _pytest.assertion import util
  File "/scratch/nikolay_nikolov/.cache/bazel/_bazel_nikolay_nikolov/79bf5e678fbb2019f1e30944a206f079/sandbox/linux-sandbox/890/execroot/_main/bazel-out/k8-opt/bin/barrel/core/common/logger/logger_tests_py_test.runfiles/_main/barrel/cor
e/common/logger/_logger_tests_py_test_actual.venv/lib/python3.10/site-packages/_pytest/assertion/util.py", line 19, in 
    from _pytest.config import Config
  File "/scratch/nikolay_nikolov/.cache/bazel/_bazel_nikolay_nikolov/79bf5e678fbb2019f1e30944a206f079/sandbox/linux-sandbox/890/execroot/_main/bazel-out/k8-opt/bin/barrel/core/common/logger/logger_tests_py_test.runfiles/_main/barrel/cor
e/common/logger/_logger_tests_py_test_actual.venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 45, in 
    from .findpaths import determine_setup
  File "/scratch/nikolay_nikolov/.cache/bazel/_bazel_nikolay_nikolov/79bf5e678fbb2019f1e30944a206f079/sandbox/linux-sandbox/890/execroot/_main/bazel-out/k8-opt/bin/barrel/core/common/logger/logger_tests_py_test.runfiles/_main/barrel/cor
e/common/logger/_logger_tests_py_test_actual.venv/lib/python3.10/site-packages/_pytest/config/findpaths.py", line 12, in 
    import iniconfig
ModuleNotFoundError: No module named 'iniconfig'

🌍 Your Environment

Operating System:

  
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 13 (trixie)
Release:        13
Codename:       trixie
  

Output of bazel version:

  
Build label: 8.3.1
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Mon Jun 30 16:23:40 2025 (1751300620)
Build timestamp: 1751300620
Build timestamp as int: 1751300620
  

Rules_python version:

  
1.9.0
  

Anything else relevant?

No

[aignas]

Relevant: https://pypi-browser.org/package/pytest/pytest-9.0.2-py3-none-any.whl

Just to confirm, this is working as expected with venvs_site_packages=no?

[nikonikolov]

Yes, it's working correctly with venvs_site_packages=no. Also I am using an earlier version of pytest https://pypi-browser.org/package/pytest/pytest-7.1.3-py3-none-any.whl

[aignas]

OK, looking at https://pypi-browser.org/package/py/py-1.11.0-py2.py3-none-any.whl it seems that iniconfig is one of things that py is vendoring so it is likely that there is something wrong with how the venv is built or how the vendored packages are included into the sys.path.

The py/__init__.py is pretty interesting - I wonder where the problem is. I'll leave the full dig to someone else.

[nikonikolov]

There's an external pip package iniconfig https://pypi-browser.org/package/iniconfig/iniconfig-2.0.0-py3-none-any.whl . Even venvs_site_packages=no ends up using that one. The METADATA for pytest includes

Requires-Dist: attrs (>=19.2.0)
Requires-Dist: iniconfig
Requires-Dist: packaging
Requires-Dist: pluggy (<2.0,>=0.12)
Requires-Dist: py (>=1.8.2)
Requires-Dist: tomli (>=1.0.0)
Requires-Dist: importlib-metadata (>=0.12) ; python_version < "3.8"
Requires-Dist: colorama ; sys_platform == "win32"
Provides-Extra: testing
Requires-Dist: argcomplete ; extra == 'testing'
Requires-Dist: hypothesis (>=3.56) ; extra == 'testing'
Requires-Dist: mock ; extra == 'testing'
Requires-Dist: nose ; extra == 'testing'
Requires-Dist: pygments (>=2.7.2) ; extra == 'testing'
Requires-Dist: requests ; extra == 'testing'
Requires-Dist: xmlschema ; extra == 'testing'

However, there seems to be some clash between pytest, py and iniconfig.

1. Adding only pytest as dep, doesn't bring py in the venv (like it should), but it does bring iniconfig
2. Explicitly adding py as dep BEFORE pytest includes py in the venv, but for some reason removes iniconfig
3. Explicitly adding py as dep AFTER pytest does NOT include py in the venv and iniconfig is included in the venv

Doesn't seem to be a python import issue on the surface, but what gets included in the venv

[nikonikolov]

Just to clarify, the issue happens with pytest==7.1.3. It goes away with pytest==9.0.2 (likely because py is no longer a dependency of pytest)