[codex] Add structured SWE rollout failure observability

[rasdani]

Summary

• Add a serialized failure rollout payload with stable reason, origin, error type metadata, message, and bounded diagnostic logs.
• Serialize failure from state_to_output() while preserving the existing error, error_chain, and long_error_chain fields.
• Redefine CLI-agent monitor metrics so agent_error means failure.origin == "agent", while agent_nonzero_exit preserves the old narrow non-zero-exit meaning and agent_poll_failed tracks background polling/read failures.
• Classify sandbox, tunnel, model, and rollout-timeout failures away from agent_error; keep sandbox_oom, sandbox_timeout, and agent_timeout metrics.
• Collect best-effort failure diagnostics before sandbox deletion: agent stdout/stderr tails, harness/agent log file tails, /tmp/vf_observed_command.log, and /tmp/install_progress.log.
• Wrap composable agent install/post-install shell commands with grep-friendly observed-command traces and failure lines without masking the original setup failure.
• Keep rollout logs grep-friendly with event=... key=value messages for aborts, finishes, empty trajectories, setup failures, and agent polling failures.

Companion scheduler metrics PR: PrimeIntellect-ai/prime-rl#2411.

Compatibility Note

• agent_error is intentionally broadened to all agent-origin failures.
• agent_nonzero_exit preserves the previous narrow non-zero process exit signal.

Validation

• uv run ruff check verifiers/utils/failure_utils.py verifiers/envs/experimental/cli_agent_env.py verifiers/envs/experimental/composable/composable_env.py verifiers/envs/experimental/composable/harnesses/opencode.py verifiers/envs/experimental/opencode_env.py verifiers/utils/save_utils.py verifiers/types.py verifiers/envs/environment.py tests/test_cli_agent_env.py tests/test_swe_rollout_observability.py
• python -m py_compile verifiers/utils/failure_utils.py verifiers/envs/experimental/cli_agent_env.py verifiers/envs/experimental/composable/composable_env.py verifiers/envs/experimental/opencode_env.py verifiers/utils/save_utils.py verifiers/types.py verifiers/envs/environment.py
• uv run pytest tests/test_cli_agent_env.py tests/test_swe_rollout_observability.py tests/test_environment_extra.py::test_state_to_output_uses_state_usage_not_trajectory tests/test_save_utils.py

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit ed668b8. Configure here.}

[cursor]

Redundant tail_text re-application corrupts truncation prefix

Low Severity

add_failure_logs passes logs to ensure_rollout_failure(state, logs=logs), which already applies tail_text to every log value internally (via make_rollout_failure or the existing-failure merge path). Then on line 234, it redundantly calls failure["logs"].update({str(k): tail_text(v) for k, v in logs.items()}), applying tail_text a second time to the same original values. Since callers like collect_failure_diagnostics already tail_text the values before passing them in, this is actually a triple application. For logs originally exceeding DEFAULT_FAILURE_LOG_CHARS (12000), the first tail_text produces a ~12025-char string with a "...<truncated N chars>\n" prefix; subsequent applications re-truncate this, replacing the accurate prefix with a misleading one (e.g., "...<truncated 25 chars>" instead of "...<truncated 1000 chars>").

 

^{Reviewed by Cursor Bugbot for commit ed668b8. Configure here.}