feat: skill-side closed-loop validation for systematic-debugging

[jramos]

Summary

Brings closed-loop validation to the skill-evolution path. Five commits, all on top of the closed-loop infrastructure shipped on the tool path. Closes PLAN.md Phase 1 deviation #3 for systematic-debugging; arxiv known-paper-recall and github-code-review planted-issue remain deferred (different verdict mechanisms).

The framework can now evaluate evolved skills not just by LM-judge composite score but by whether following the skill actually causes a real Hermes agent to debug + fix a planted bug correctly.

Commits:

1. test_command verdict mode for skill-shaped tasks — extends Task with optional test_command; score_task branches on it (run the command in fixture_dir, pass iff exit code zero). Tool-side scoring unchanged.
2. SkillFileInstaller + cache artifact_writer injection + runner skills-source plumbing — installer copies baseline skill into a writable workdir, runner stages it into each per-task sandbox via TaskRunContext.skills_src. ClosedLoopFeedbackCache parameterized to accept any artifact writer (default = current MCP manifest, preserving tool path).
3. systematic-debugging planted-bug suite (5 tasks: off-by-one, inverted condition, missing base case, wrong operator, mutable default arg) + fake target skill fixture (deliberately weak baseline so GEPA has headroom). Sanity tests verify every planted bug is real and every known fix passes.
4. Wire closed-loop end-to-end through evolve_skill — five --closed-loop-* CLI flags mirroring evolve_tool, _maybe_build_closed_loop_cache_skill helper, SkillModule.forward behavioral branch, behavioral-example injection into trainset/valset, metric construction with the cache.
5. Docs + manual smoke harness + PLAN deviation closure.

Test coverage: 1004 passing (was 950; +54 new). All mock-only in CI. Manual smoke at tests/manual/skill_closed_loop_smoke.py drives the closed-loop layer end-to-end against real hermes -z subprocesses (10 calls per run, real LM spend; not in CI).

Default --closed-loop-mode is feedback, not trainset. Skill bodies mutate heavily, so trainset mode's gate_mode=always (validator fires on every novel candidate) is 3-10x more expensive than on the tool path. Users opt into trainset explicitly when the cost is acceptable.

Manual validation outcome

Stage 1 smoke confirmed the closed-loop wiring works end-to-end against real hermes -z subprocesses:

Decision: pass
Baseline:  5/5 (100%)
Evolved:   5/5 (100%)
Per-task:  0 wins, 0 losses, 5 ties

All 10 hermes calls completed cleanly in ~4 min, candidate-text cache dedup'd on the second invocation, no orphaned per-task tmpdirs.

The 5/5 baseline pass rate is itself the headline finding: a reasoning-class model (gpt-5.4-mini in this case) solves the textbook planted bugs even with the deliberately weak baseline skill text. This is the saturation outcome (#3 from the design plan's three-outcome triage) — proves the framework is wired correctly but tells us the bug suite is too easy to discriminate skill text variants on capable models.

Stage 2 (heavy-budget GEPA run with --closed-loop-mode trainset) was skipped because the saturated baseline guarantees zero behavioral signal for GEPA to act on. Two follow-ups are scoped to fix this in a future PR: (a) add an optional --model kwarg to HermesAgentRunner so closed-loop validation can run against a deliberately weaker agent model independently of the user's Hermes default; (b) expand the suite with harder bugs (subtle binary-search off-by-one, generator exhaustion, mutable cached return, float-precision equality, async/state ordering) that survive a textbook debugging methodology.

Test plan

• uv run pytest tests/ -q --ignore=tests/manual — 1004 passed
• CI green on Python 3.10–3.13 (4 versions, mock-only tests)
• Tool-path regression check: test_closed_loop_feedback.py, test_validator.py, test_safety.py, test_evolve_tool_closed_loop.py all green after cache kwarg rename + verify_backup Protocol extension
• Manual smoke (tests/manual/skill_closed_loop_smoke.py) — passed end-to-end, ~4 min, all assertions green; wiring confirmed via real hermes -z × 10
• Heavy-budget GEPA classification — deferred; baseline saturates at 5/5 against the configured reasoning model, leaving GEPA no behavioral signal to act on. Follow-up PR adds model override + harder bugs to make the classification meaningful.