Skip to content

Adapt decoder calibration CV for sparse folds#185

Merged
FlorianPfaff merged 3 commits into
mainfrom
codex/adaptive-calibration-cv-20260526
May 26, 2026
Merged

Adapt decoder calibration CV for sparse folds#185
FlorianPfaff merged 3 commits into
mainfrom
codex/adaptive-calibration-cv-20260526

Conversation

@FlorianPfaff
Copy link
Copy Markdown
Member

Summary

  • Add an adaptive calibration wrapper for decoder factories that currently request CalibratedClassifierCV(cv=3).
  • Reduce the inner calibration CV at fit time to the largest feasible value given the fitted fold's per-class counts.
  • Fall back to uncalibrated score-derived probabilities only when a calibration split is impossible, preserving probability-shaped emissions for sparse smoke tests and small held-out-subject folds.
  • Add regression tests for sparse calibrated linear-SVM folds.

Testing

  • Added tests/test_adaptive_calibration.py for the 2-per-class adaptive CV case and the 1-per-class fallback case.
  • I sanity-checked the wrapper behavior locally with installed NumPy/scikit-learn. I could not run the full repository test suite locally because the execution container cannot clone GitHub repositories directly, so CI should be treated as the authoritative validation.

@github-actions
Copy link
Copy Markdown

Test Results

    4 files  ±0      4 suites  ±0   1m 47s ⏱️ +5s
  641 tests +2    632 ✅ +2   9 💤 ±0  0 ❌ ±0 
2 564 runs  +8  2 528 ✅ +8  36 💤 ±0  0 ❌ ±0 

Results for commit c80d67e. ± Comparison against base commit 46ac769.

@github-actions
Copy link
Copy Markdown

⚠️MegaLinter analysis: Success with warnings

Descriptor Linter Files Fixed Errors Warnings Elapsed time
✅ ACTION actionlint 14 0 0 0.46s
✅ COPYPASTE jscpd yes no no 8.65s
⚠️ MARKDOWN markdownlint 55 0 1 0 1.78s
✅ PYTHON ruff 245 0 0 0 0.39s
✅ REPOSITORY git_diff yes no no 0.06s
✅ YAML prettier 48 0 0 0 1.48s
✅ YAML v8r 48 0 0 8.23s
✅ YAML yamllint 48 0 0 1.3s

Detailed Issues

⚠️ MARKDOWN / markdownlint - 1 error
.github/PULL_REQUEST_TEMPLATE.md:1 error MD041/first-line-heading/first-line-h1 First line in a file should be a top-level heading [Context: "## Summary"]

Notices

📣 MegaLinter 9.5.0 is out! Discover the new features and security recommendations in the release announcement. (Skip this info by defining SECURITY_SUGGESTIONS: false)

See detailed reports in MegaLinter artifacts

MegaLinter is graciously provided by OX Security
Show us your support by starring ⭐ the repository

@FlorianPfaff FlorianPfaff merged commit 8545378 into main May 26, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant