fix: diagnose silent test-to-mutant key mismatch in stats#516
Open
irajaryan wants to merge 1 commit intoboxed:mainfrom
Open
fix: diagnose silent test-to-mutant key mismatch in stats#516irajaryan wants to merge 1 commit intoboxed:mainfrom
irajaryan wants to merge 1 commit intoboxed:mainfrom
Conversation
When tests import source under a different module path than mutmut derives from the file path (e.g. via pythonpath in pytest config or conftest sys.path injection), every mutant is silently marked "No Tests" because trampoline hits are recorded under __module__-based keys that the path-derived mutant lookup can never match. Add a check after collect_source_file_mutation_data that detects this disjoint-keys case and exits early with sample recorded vs expected keys, mirroring the existing "Stopping early" diagnostic in run_stats_collection. Fixes boxed#515
irajaryan
force-pushed
the
fix/diagnose-test-mutant-key-mismatch
branch
from
e5e5bf6 to
50425d5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds a sanity check after stats collection that detects when tests recorded trampoline hits but none match any mutant lookup key. Without it, the run silently completes with every mutant marked "No Tests" and 0.00 mutations/second.
The trampoline records
orig.__module__ + '.' + orig.__name__while per-mutant lookup uses the path-derived dotted name fromget_mutant_name. They match in healthy projects (qualified package imports, source under./,src/, orsource/). They diverge when tests use top-level imports enabled by pytest'spythonpathconfig, conftestsys.path.insert, or non-conventional source dirs. The existing zero-check inrun_stats_collectiondoesn't catch the divergence because hits are recorded, just under unmatchable keys.The new
_check_test_to_mutant_associationsruns aftercollect_source_file_mutation_dataand exits early with sample recorded vs expected keys, in the same style as the existing "Stopping early" diagnostic.Same symptom appears in #341, #351, #416, #419, all closed after users fixed their project setup. This change surfaces the cause earlier.
Fixes #515