Skip to content

Include per-commit reviews in branch review prompts#626

Merged
wesm merged 1 commit intoroborev-dev:mainfrom
shoyer:duplicate-refine-findings
Apr 5, 2026
Merged

Include per-commit reviews in branch review prompts#626
wesm merged 1 commit intoroborev-dev:mainfrom
shoyer:duplicate-refine-findings

Conversation

@shoyer
Copy link
Copy Markdown
Contributor

@shoyer shoyer commented Apr 5, 2026

Summary

  • When the refine workflow runs a branch-level range review (mergeBase..HEAD), the reviewer previously had no context about what individual per-commit reviews had already found, fixed, or dismissed — causing it to re-raise the same issues
  • Add writeInRangeReviews() to include per-commit review results (with verdicts, agent names, and dismissal comments) in range/branch review prompts, with a "do not re-raise" instruction (InRangeReviewsHeader)
  • Extract shared lookupReviewContexts() helper to deduplicate the review-fetching loop between writeInRangeReviews and getPreviousReviewContexts
  • Update both claude and codex refine skills to note that dismissal comments are included in re-review prompts

Companion to #622, which ensures comments reach fix agents. This PR ensures per-commit reviews and their comments reach the reviewing agent during branch reviews, closing the other half of the duplicate-findings loop.

🤖 Generated with Claude Code

@roborev-ci
Copy link
Copy Markdown

roborev-ci bot commented Apr 5, 2026

roborev: Combined Review (2422099)

Verdict: Changes look mostly sound, but there is one medium-severity correctness issue to address before merge.

Medium

  • internal/prompt/prompt.go:826
    writeInRangeReviews looks up prior reviews with GetReviewByCommitSHA(sha) without scoping by repoID. If two repositories share the same commit object, a range review can pull review output or dismissal comments from the wrong repo. Because the prompt now tells the reviewer not to re-raise listed issues, this can incorrectly suppress valid findings or inject irrelevant context.
    Suggested fix: scope the lookup by both repository and commit SHA, pass repoID into writeInRangeReviews, and add a regression test covering two repos with the same commit SHA but different stored reviews.

Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

@shoyer
Copy link
Copy Markdown
Contributor Author

shoyer commented Apr 5, 2026

The unscoped GetReviewByCommitSHA lookup is consistent with the existing getPreviousReviewContexts which has the same behavior. Two registered repos sharing the same commit SHA is a hypothetical scenario (requires locally registered forks) that doesn't justify the added complexity. If this becomes a real problem, we can scope all review lookups by repo in a single pass.

Comment by Claude Code

@roborev-ci
Copy link
Copy Markdown

roborev-ci bot commented Apr 5, 2026

roborev: Combined Review (e6f69ff)

Verdict: 2 medium-severity issues should be addressed before merge.

Medium

  • Cross-repo review context leakage

    • Location: internal/prompt/prompt.go:846, internal/prompt/prompt.go:854
    • lookupReviewContexts resolves reviews by commit SHA only and does not scope by repoID. In this change, that lookup feeds prior review output and comments back into range-review prompts, so identical SHAs across tracked repos can pull in review content from the wrong repo. That can leak unrelated review text/comments into the active prompt and bias the next review.
    • Recommended fix: make the lookup repo-scoped using (repoID, commitSHA), thread repoID through writeInRangeReviews/lookupReviewContexts, and add a regression test covering identical SHAs in multiple repos.
  • Unbounded prior-review prompt growth

    • Location: internal/prompt/prompt.go:621, internal/prompt/prompt.go:819
    • Range prompts now append full stored review output for every reviewed commit in the range without any cap or truncation. On longer branches, this can crowd out the actual diff/context or exceed model input limits, degrading review quality.
    • Recommended fix: cap by number of prior reviews and/or total bytes/tokens, and truncate or summarize each appended review before adding it to the prompt.

Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

Note: gemini review skipped (agent quota exhausted)

@shoyer
Copy link
Copy Markdown
Contributor Author

shoyer commented Apr 5, 2026

Re: unbounded prior-review prompt growth — this is already handled. writeInRangeReviews writes into optionalContext, which buildPromptPreservingCurrentSection truncates first when the prompt exceeds the budget. The diff and required sections are always preserved. This is the same mechanism used for project guidelines and previous reviews.

Re: cross-repo review context leakage — same finding as the first review round, already dismissed above.

Comment by Claude Code

@wesm
Copy link
Copy Markdown
Collaborator

wesm commented Apr 5, 2026

I'm rebasing this

…ate findings

- Scope writeInRangeReviews lookup by repo ID to avoid cross-repo collisions
- Extract lookupReviewContexts to deduplicate review fetching
- Update refine skills to handle duplicate findings across iterations
- Revise refine skills: emphasize dismissal comments over duplicate skipping
- Tighten refine skill comment guidance
- Tone down InRangeReviewsHeader wording
- Revert PreviousReviewsHeader wording change
- Revert repo-scoped writeInRangeReviews lookup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@wesm wesm force-pushed the duplicate-refine-findings branch from e6f69ff to 225897e Compare April 5, 2026 19:20
@roborev-ci
Copy link
Copy Markdown

roborev-ci bot commented Apr 5, 2026

roborev: Combined Review (225897e)

Verdict: Changes are not clean; 2 medium-severity correctness issues should be addressed before merge.

Medium

  • Prompt growth can become unbounded

    • Location: internal/prompt/prompt.go:821
    • writeInRangeReviews appends full stored review output plus all review comments for every reviewed commit in the range without a size cap. On larger branches, this can bloat the prompt enough to risk context overruns or truncation, which makes range reviews less reliable.
    • Suggested fix: Bound by commit count and/or per-review length, or include a summarized prior verdict/findings view instead of embedding each full review verbatim.
  • Review context lookup is not repository-scoped

    • Location: internal/prompt/prompt.go:859
    • lookupReviewContexts looks up reviews by commit SHA alone and does not scope the query to the current repository. The same SHA can exist across different clones or forks, so this can pull unrelated review history into the prompt and potentially suppress valid findings for the current repo.
    • Suggested fix: Thread repoID through this path and query by (repo_id, commit_sha) rather than SHA alone.

Security review found no issues under the local-only trust model.


Synthesized from 3 reviews (agents: codex, gemini | types: default, security)

Note: gemini review skipped (agent quota exhausted)

@wesm wesm merged commit dc42e2a into roborev-dev:main Apr 5, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants