Skip to content

tests: detect stale Cython .so files at test startup#756

Open
mykaul wants to merge 2 commits intoscylladb:masterfrom
mykaul:fix/detect-stale-cython-extensions
Open

tests: detect stale Cython .so files at test startup#756
mykaul wants to merge 2 commits intoscylladb:masterfrom
mykaul:fix/detect-stale-cython-extensions

Conversation

@mykaul
Copy link

@mykaul mykaul commented Mar 19, 2026

Add a pytest_configure hook in tests/conftest.py that compares mtime of each compiled .so against its .py source and warns when the source is newer. This prevents silently testing stale compiled code after editing a Cython-compiled module without rebuilding.
(Bit me here - #749 )

Also document the rebuild requirement in CONTRIBUTING.rst.

Pre-review checklist

  • I have split my patch into logically separate commits.
  • All commit messages clearly explain what they change and why.
  • I added relevant tests for new features and bug fixes.
  • All commits compile, pass static checks and pass test.
  • PR description sums up the changes and reasons why they should be introduced.
  • I have provided docstrings for the public items that I want to introduce.
  • I have adjusted the documentation in ./docs/source/.
  • I added appropriate Fixes: annotations to PR description.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds early test-session detection of stale in-place Cython extension builds (to avoid running tests against outdated compiled code) and documents the rebuild requirement for contributors.

Changes:

  • Add pytest_configure hook in tests/conftest.py to warn when a compiled extension is older than its corresponding cassandra/*.py source.
  • Document the required python setup.py build_ext --inplace rebuild step in CONTRIBUTING.rst and note that tests will warn when staleness is detected.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
tests/conftest.py New pytest startup hook that scans for stale compiled extensions vs .py sources and emits a warning.
CONTRIBUTING.rst Adds contributor guidance about rebuilding extensions to avoid testing stale compiled code.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@Lorak-mmk Lorak-mmk self-requested a review March 19, 2026 17:27
Add a pytest_configure hook in tests/conftest.py that compares mtime
of each compiled .so against its .py source and warns when the source
is newer. This prevents silently testing stale compiled code after
editing a Cython-compiled module without rebuilding.

Also document the rebuild requirement in CONTRIBUTING.rst.
@mykaul mykaul force-pushed the fix/detect-stale-cython-extensions branch from 91aeef3 to 668275b Compare March 20, 2026 10:46
@mykaul mykaul requested a review from Copilot March 20, 2026 12:38
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a pytest startup check to help developers avoid accidentally running tests against stale Cython-compiled extensions when the corresponding cassandra/*.py sources have been edited without rebuilding, and documents the rebuild requirement for contributors.

Changes:

  • Introduces a pytest_configure hook that scans for Cython extension modules and warns when a compiled extension appears older than its .py source.
  • Documents the need to rebuild extensions after editing Cython-compiled cassandra/ modules, noting that the test suite will warn on staleness.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
tests/conftest.py Adds a pytest session-start hook that attempts to detect stale Cython extension artifacts and emits a warning.
CONTRIBUTING.rst Adds contributor guidance about rebuilding extensions after modifying Cython-compiled driver modules.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +36 to +51
seen = set()
stale = []
# Use the current interpreter's extension suffixes so we only check
# extensions that would actually be loaded (correct ABI tag), and
# handle both .so (POSIX) and .pyd (Windows) automatically.
for suffix in importlib.machinery.EXTENSION_SUFFIXES:
for ext_path in glob.glob(os.path.join(_CASSANDRA_DIR, f"*{suffix}")):
module_name = os.path.basename(ext_path).split(".")[0]
if module_name in seen:
continue
py_path = os.path.join(_CASSANDRA_DIR, module_name + ".py")
if os.path.exists(py_path) and os.path.getmtime(py_path) > os.path.getmtime(
ext_path
):
seen.add(module_name)
stale.append((module_name, ext_path, py_path))
Copy link

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using glob('*{suffix}') while iterating importlib.machinery.EXTENSION_SUFFIXES can yield false positives because the generic suffixes (e.g. .so/.pyd) will match ABI-tagged extension files that the import system would not load for that suffix (and can also match extensions built for other Python versions, like query.cpython-310-...so). This may warn that tests will run stale compiled code even when the current interpreter would import the .py (or a different, newer extension).

Consider iterating over .py sources and, per module, selecting the first exact existing module_name + suffix in EXTENSION_SUFFIXES order (i.e., the one Python would actually import) and only comparing that file’s mtime to the .py mtime; also mark the module as processed regardless of staleness to avoid redundant checks.

Suggested change
seen = set()
stale = []
# Use the current interpreter's extension suffixes so we only check
# extensions that would actually be loaded (correct ABI tag), and
# handle both .so (POSIX) and .pyd (Windows) automatically.
for suffix in importlib.machinery.EXTENSION_SUFFIXES:
for ext_path in glob.glob(os.path.join(_CASSANDRA_DIR, f"*{suffix}")):
module_name = os.path.basename(ext_path).split(".")[0]
if module_name in seen:
continue
py_path = os.path.join(_CASSANDRA_DIR, module_name + ".py")
if os.path.exists(py_path) and os.path.getmtime(py_path) > os.path.getmtime(
ext_path
):
seen.add(module_name)
stale.append((module_name, ext_path, py_path))
stale = []
# Iterate over .py sources and, for each module, look for the first
# existing compiled extension in EXTENSION_SUFFIXES order. This mirrors
# how Python's import machinery selects an extension module, and avoids
# globbing patterns like "*{suffix}" that can pick up ABI-tagged
# extensions built for other Python versions.
if os.path.isdir(_CASSANDRA_DIR):
for entry in os.listdir(_CASSANDRA_DIR):
if not entry.endswith(".py"):
continue
module_name, _ = os.path.splitext(entry)
py_path = os.path.join(_CASSANDRA_DIR, entry)
# For this module, find the first extension file Python would load.
for suffix in importlib.machinery.EXTENSION_SUFFIXES:
ext_path = os.path.join(_CASSANDRA_DIR, module_name + suffix)
if not os.path.exists(ext_path):
continue
if os.path.getmtime(py_path) > os.path.getmtime(ext_path):
stale.append((module_name, ext_path, py_path))
# Only consider the first matching suffix; this is the one
# the import system would actually use.
break

Copilot uses AI. Check for mistakes.
…sions

Rewrite the stale-extension scan to iterate over .py source files and
check for the first matching compiled extension per EXTENSION_SUFFIXES
order.  This mirrors Python's import machinery and avoids false positives
from ABI-tagged extensions built for other Python versions that the glob
approach could pick up.  Also removes the now-unused glob import.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants