eval: consolidate onto the unified Evaluator pipeline (+ viewer/docs)#437

Open

eugenevinitsky wants to merge 20 commits into

vcha/updatefrom

ev/eval-csv-coverage

Commits on May 21, 2026

eval: per-episode CSV + scenario-coverage in the Evaluator pipeline

authored and
Eugene Vinitsky
committed
eval: ruff format (collapse coverage print to one line)

authored and
Eugene Vinitsky
committed
eval: validation evaluators reproduce the multi-scenario eval setup

authored and
Eugene Vinitsky
committed
eval: run validation evaluators every 250 epochs

authored and
Eugene Vinitsky
committed
eval (G1): attach map_name + scenario_id to completed_episode summary

authored and
Eugene Vinitsky
committed
eval (G2,G5): per-map coverage + observation rendering

authored and
Eugene Vinitsky
committed
eval (G3,G4): checkpoint arch merge + ad-hoc CLI; rewire eval scripts

authored and
Eugene Vinitsky
committed
docs: evaluation guide; track docs/*.md

authored and
Eugene Vinitsky
committed
eval: honest coverage completeness + obs gallery index

authored and
Eugene Vinitsky
committed
eval: remove superseded eval_multi_scenarios; sweep all replay maps

authored and
Eugene Vinitsky
committed

Commits on May 22, 2026

eval: count episodes for the stop condition; sync num_eval_scenarios

authored and
Eugene Vinitsky
committed
eval: merge obs/action-layout env keys from checkpoint config.yaml

authored and
Eugene Vinitsky
committed
eval: merge obs normalization + spatial-extent from checkpoint config.yaml

authored and
Eugene Vinitsky
committed
eval: basename map_name in the html render path

authored and
Eugene Vinitsky
committed
viz: make the Scenario Info panel click-to-minimize

authored and
Eugene Vinitsky
committed
viz: collapse the Scenario Info panel by default

authored and
Eugene Vinitsky
committed
viz: select the SDC by default in the replay viewer

authored and
Eugene Vinitsky
committed
eval: ruff format wrap long lines in _render_pass_obs

authored and
Eugene Vinitsky
committed
docs: clarify which evaluator the ad-hoc eval uses

authored and
Eugene Vinitsky
committed
eval: clearer render naming (render_backend enum) + --num_maps

authored and
Eugene Vinitsky
committed