Harden SWTBench conversion by filtering non-test diffs #337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

simonrosenberg wants to merge 1 commit into main from swtbench-test-filtering

Collaborator

simonrosenberg commented Jan 17, 2026

Summary

filter SWTBench patches to drop setup files, top-level helper scripts, and non-test files before harness conversion
skip submissions that contain only top-level or non-test diffs and log counts
add reusable patch helpers (predicate filtering, test-file heuristic)

Testing

python3 -m py_compile benchmarks/swtbench/eval_infer.py benchmarks/utils/patch_utils.py


          Add test-only patch filtering for SWTBench

3f9aecf

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet