Add immediate sub-LLM trajectory steps and incremental metrics #681
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Instead of adding the sub-LLM
TrajectorySteps and the metric during cleanup, this PR changes that to happen incrementally as the sub-LLM responses come in; and the metrics are also updated incrementally as the main model's steps come in.Type of Change
Testing
uv run pytestlocally.Checklist
Additional Notes
Note
Implements immediate insertion of sub-LLM turns and incremental metric tracking throughout RLM execution.
update_rlm_metrics_from_step,_extract_tokens_from_response, and_ensure_rlm_metric_stateto track sub-LLM/main turns, tokens, tool calls, batches, and REPL timing incrementally; overridesadd_trajectory_stepto update metrics on each step_handle_sub_llm_requestnow constructs per-turnTrajectorySteps with extras and either immediately callsadd_trajectory_step(when enabled) or updates metrics only;active_rolloutsstoresstatefor this_update_rlm_repl_metrics; initializes metric fields insetup_statecall_python_replnow uses the last main-model step (ignores sub-LLM steps)cleanup_rlm_stateto just remove rollout and release tunnels (metrics/steps no longer computed here)Written by Cursor Bugbot for commit d6beb5c. This will update automatically on new commits. Configure here.