Add per-task and per-model token breakdown to end-of-session summary

[samkeen]

Summary

This PR adds visibility into token spending by introducing a per-task and per-model breakdown in the end-of-session summary. The breakdown is computed by replaying events.jsonl rather than relying on in-memory state, ensuring it survives crashes and --resume cycles.

Key Changes

• Instrumented all three model call sites to emit model_call events with a kind field:

  • _run_task() (worker): kind="worker"
  • _judge_task() (judge): kind="judge"
  • _self_improve() (self-improvement): kind="self_improve"
  • Each event now includes the model field for attribution across providers

• Added _token_breakdowns() function that replays events.jsonl to aggregate tokens:

  • Returns (by_model, by_task) tuples
  • by_model: maps model name to total tokens
  • by_task: maps task ID to a dict with total and per-kind breakdowns
  • Gracefully handles missing files, malformed JSON lines, and legacy events without kind/model fields (defaults to "unknown"/"worker")
  • Skips zero-token events to avoid polluting the breakdown

• Enhanced end-of-session summary to display the breakdowns:

  • Per-task section sorted by token usage (descending), showing total and split by kind
  • Per-model section sorted by token usage (descending)
  • Helps identify which tasks consumed the most budget and which models were used

• Updated visualization (render.py) to display model call metadata:

  • Shows kind in badge labels (e.g., "judge (iter 3)" vs "iter 1")
  • Includes model name in the meta strip when available
  • Handles legacy events without kind/model gracefully

• Documentation updates (USAGE.md, deep-dives.md, session.py):

  • Documented the new per-task and per-model breakdown feature
  • Clarified that all three sites now emit model_call events
  • Explained the replay-based approach and its benefits

Implementation Details

• The breakdown computation is defensive: it tolerates missing fields, malformed JSON, and zero-token events
• Legacy events (pre-breakdown) are bucketed under "unknown" model and "worker" kind, maintaining backward compatibility
• The iter field is only emitted for worker and judge calls (not self-improve), as self-improvement doesn't have a meaningful iteration number
• Token counts are computed as prompt_tokens + eval_tokens, matching the existing add_tokens() pattern

https://claude.ai/code/session_01DYjQrDCX3FaFBppR9NisKZ