You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Summary\n\nWhen an agent produces a review or follow-up response that is substantively clear but fails the strict grammar/schema check, the orchestrator could invoke a smaller analyzer LLM to normalize the result into the expected structured shape.\n\nThe analyzer would read a small input: the raw agent output plus the exact schema contract and the current unresolved-item context. It would then return a strict structured object indicating:\n\n- approved or blocking\n- addressed items\n- remaining items\n- any new items introduced\n- optionally a short normalized summary or quoted evidence\n\n## Motivation\n\nRecent failures show the main issue is often not the underlying review content, but the model drifting away from the required wrapper format after a long turn. A focused repair pass may recover those near-miss outputs much more reliably than retrying the full review prompt.\n\n## Guardrails\n\n- The primary contract should remain structured output from the original agent.\n- The analyzer is only a fallback when strict parsing fails.\n- If the analyzer cannot confidently justify a repair, the orchestrator should treat the result as blocking rather than inventing approval.\n- The analyzer should not be able to override a clearly blocking review into approval without evidence from the raw output.\n\n## Relationship to #140\n\nThis is a follow-up idea to the structured-response migration in #140, not a replacement for it. Structured output should remain the primary path; the analyzer is only a recovery layer for malformed near-miss responses.\n\n## Example\n\nOn recent llm-dialectic runs, some reviews clearly contained the needed disposition information, but the model wrapped the JSON in fences or mixed in prose, causing the strict parser to fail. A repair pass could salvage those cases and reduce unnecessary reruns.\n\n-- OpenAI Codex
Summary\n\nWhen an agent produces a review or follow-up response that is substantively clear but fails the strict grammar/schema check, the orchestrator could invoke a smaller analyzer LLM to normalize the result into the expected structured shape.\n\nThe analyzer would read a small input: the raw agent output plus the exact schema contract and the current unresolved-item context. It would then return a strict structured object indicating:\n\n- approved or blocking\n- addressed items\n- remaining items\n- any new items introduced\n- optionally a short normalized summary or quoted evidence\n\n## Motivation\n\nRecent failures show the main issue is often not the underlying review content, but the model drifting away from the required wrapper format after a long turn. A focused repair pass may recover those near-miss outputs much more reliably than retrying the full review prompt.\n\n## Guardrails\n\n- The primary contract should remain structured output from the original agent.\n- The analyzer is only a fallback when strict parsing fails.\n- If the analyzer cannot confidently justify a repair, the orchestrator should treat the result as blocking rather than inventing approval.\n- The analyzer should not be able to override a clearly blocking review into approval without evidence from the raw output.\n\n## Relationship to #140\n\nThis is a follow-up idea to the structured-response migration in #140, not a replacement for it. Structured output should remain the primary path; the analyzer is only a recovery layer for malformed near-miss responses.\n\n## Example\n\nOn recent llm-dialectic runs, some reviews clearly contained the needed disposition information, but the model wrapped the JSON in fences or mixed in prose, causing the strict parser to fail. A repair pass could salvage those cases and reduce unnecessary reruns.\n\n-- OpenAI Codex