tests: detect stale Cython .so files at test startup#756
tests: detect stale Cython .so files at test startup#756mykaul wants to merge 2 commits intoscylladb:masterfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Adds early test-session detection of stale in-place Cython extension builds (to avoid running tests against outdated compiled code) and documents the rebuild requirement for contributors.
Changes:
- Add
pytest_configurehook intests/conftest.pyto warn when a compiled extension is older than its correspondingcassandra/*.pysource. - Document the required
python setup.py build_ext --inplacerebuild step inCONTRIBUTING.rstand note that tests will warn when staleness is detected.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
tests/conftest.py |
New pytest startup hook that scans for stale compiled extensions vs .py sources and emits a warning. |
CONTRIBUTING.rst |
Adds contributor guidance about rebuilding extensions to avoid testing stale compiled code. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Add a pytest_configure hook in tests/conftest.py that compares mtime of each compiled .so against its .py source and warns when the source is newer. This prevents silently testing stale compiled code after editing a Cython-compiled module without rebuilding. Also document the rebuild requirement in CONTRIBUTING.rst.
91aeef3 to
668275b
Compare
There was a problem hiding this comment.
Pull request overview
Adds a pytest startup check to help developers avoid accidentally running tests against stale Cython-compiled extensions when the corresponding cassandra/*.py sources have been edited without rebuilding, and documents the rebuild requirement for contributors.
Changes:
- Introduces a
pytest_configurehook that scans for Cython extension modules and warns when a compiled extension appears older than its.pysource. - Documents the need to rebuild extensions after editing Cython-compiled
cassandra/modules, noting that the test suite will warn on staleness.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
tests/conftest.py |
Adds a pytest session-start hook that attempts to detect stale Cython extension artifacts and emits a warning. |
CONTRIBUTING.rst |
Adds contributor guidance about rebuilding extensions after modifying Cython-compiled driver modules. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
tests/conftest.py
Outdated
| seen = set() | ||
| stale = [] | ||
| # Use the current interpreter's extension suffixes so we only check | ||
| # extensions that would actually be loaded (correct ABI tag), and | ||
| # handle both .so (POSIX) and .pyd (Windows) automatically. | ||
| for suffix in importlib.machinery.EXTENSION_SUFFIXES: | ||
| for ext_path in glob.glob(os.path.join(_CASSANDRA_DIR, f"*{suffix}")): | ||
| module_name = os.path.basename(ext_path).split(".")[0] | ||
| if module_name in seen: | ||
| continue | ||
| py_path = os.path.join(_CASSANDRA_DIR, module_name + ".py") | ||
| if os.path.exists(py_path) and os.path.getmtime(py_path) > os.path.getmtime( | ||
| ext_path | ||
| ): | ||
| seen.add(module_name) | ||
| stale.append((module_name, ext_path, py_path)) |
There was a problem hiding this comment.
Using glob('*{suffix}') while iterating importlib.machinery.EXTENSION_SUFFIXES can yield false positives because the generic suffixes (e.g. .so/.pyd) will match ABI-tagged extension files that the import system would not load for that suffix (and can also match extensions built for other Python versions, like query.cpython-310-...so). This may warn that tests will run stale compiled code even when the current interpreter would import the .py (or a different, newer extension).
Consider iterating over .py sources and, per module, selecting the first exact existing module_name + suffix in EXTENSION_SUFFIXES order (i.e., the one Python would actually import) and only comparing that file’s mtime to the .py mtime; also mark the module as processed regardless of staleness to avoid redundant checks.
| seen = set() | |
| stale = [] | |
| # Use the current interpreter's extension suffixes so we only check | |
| # extensions that would actually be loaded (correct ABI tag), and | |
| # handle both .so (POSIX) and .pyd (Windows) automatically. | |
| for suffix in importlib.machinery.EXTENSION_SUFFIXES: | |
| for ext_path in glob.glob(os.path.join(_CASSANDRA_DIR, f"*{suffix}")): | |
| module_name = os.path.basename(ext_path).split(".")[0] | |
| if module_name in seen: | |
| continue | |
| py_path = os.path.join(_CASSANDRA_DIR, module_name + ".py") | |
| if os.path.exists(py_path) and os.path.getmtime(py_path) > os.path.getmtime( | |
| ext_path | |
| ): | |
| seen.add(module_name) | |
| stale.append((module_name, ext_path, py_path)) | |
| stale = [] | |
| # Iterate over .py sources and, for each module, look for the first | |
| # existing compiled extension in EXTENSION_SUFFIXES order. This mirrors | |
| # how Python's import machinery selects an extension module, and avoids | |
| # globbing patterns like "*{suffix}" that can pick up ABI-tagged | |
| # extensions built for other Python versions. | |
| if os.path.isdir(_CASSANDRA_DIR): | |
| for entry in os.listdir(_CASSANDRA_DIR): | |
| if not entry.endswith(".py"): | |
| continue | |
| module_name, _ = os.path.splitext(entry) | |
| py_path = os.path.join(_CASSANDRA_DIR, entry) | |
| # For this module, find the first extension file Python would load. | |
| for suffix in importlib.machinery.EXTENSION_SUFFIXES: | |
| ext_path = os.path.join(_CASSANDRA_DIR, module_name + suffix) | |
| if not os.path.exists(ext_path): | |
| continue | |
| if os.path.getmtime(py_path) > os.path.getmtime(ext_path): | |
| stale.append((module_name, ext_path, py_path)) | |
| # Only consider the first matching suffix; this is the one | |
| # the import system would actually use. | |
| break |
…sions Rewrite the stale-extension scan to iterate over .py source files and check for the first matching compiled extension per EXTENSION_SUFFIXES order. This mirrors Python's import machinery and avoids false positives from ABI-tagged extensions built for other Python versions that the glob approach could pick up. Also removes the now-unused glob import.
Add a pytest_configure hook in tests/conftest.py that compares mtime of each compiled .so against its .py source and warns when the source is newer. This prevents silently testing stale compiled code after editing a Cython-compiled module without rebuilding.
(Bit me here - #749 )
Also document the rebuild requirement in CONTRIBUTING.rst.
Pre-review checklist
./docs/source/.Fixes:annotations to PR description.