fix(analyze-hook): warn when submitter name/description is absent#578
Merged
Conversation
§6 step 4 and §7 step 3 of classify-hook.md said "If rejected or absent, ... add a warning to the warnings array", but §8 only enumerated the rejected-case templates. The AI followed §8's literal forms and silently dropped warnings when a submitter left the Hook Name or Description form fields blank. Result: registry entries where 100% of the user-facing identity was AI-authored looked indistinguishable from entries where a submitter provided text that was vetted and accepted. Caught on PR #570 audit — TTTHook had empty submitter fields and no warning was emitted. Add two §8 templates for the absent case so warnings now flag every field whose final value came from the AI rather than the submitter: - Absent submitter name: "Submitter did not provide a name. Using AI-derived "<Y>"." - Absent submitter description: "Submitter did not provide a description. Using AI-generated description." Also add a closing paragraph making explicit that warnings should fire on rejection OR absence — the threat model is "AI text landed in the registry; reviewer needs an audit trail" — so absence and rejection get the same treatment. No code changes; assemble_hook.py already renders warnings as opaque bullets via the existing test_generate_pr_body_renders_warnings_as_bullets path. Existing tests unchanged and passing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Today's PR-triage audit on six bot-authored hook PRs surfaced that PR #570 (TTTHook) landed in the registry with zero warnings even though both the submitter-provided Hook Name and Description were
_No response_— meaning 100% of the user-facing identity was AI-authored, with no audit trail saying so.The prompt at
.claude/prompts/classify-hook.mdalready says (§6 step 4, §7 step 3) that warnings should fire on rejected OR absent submitter fields. But §8 only enumerates the rejected-case templates. The AI followed §8's literal forms and silently dropped warnings for the absent case.What this changes
.claude/prompts/classify-hook.md§8 now has two more templates:Submitter did not provide a name. Using AI-derived "<Y>".Submitter did not provide a description. Using AI-generated description.Plus a closing paragraph making explicit that warnings fire on rejection or absence — the threat model is "AI-authored text landed in the registry; the reviewer needs to see it." Absence and rejection get the same treatment.
What this does NOT change
assemble_hook.py— no code changes. Warnings already flow opaquely throughgenerate_pr_body()as bullets.analyze-hook.yml's JSON schema forwarnings(max 20 entries, 300 chars each) — new templates are ~80 chars.Test plan
Submitter did not provide…bullets.Audit context
The four PRs from today's batch that did emit warnings (#572 length, #576 platform-scope, #577 audit claim) continue to work the same — they're all "rejected"-case templates and unchanged here.
🤖 Generated with Claude Code