feat(citability): add answer-first hint#51
Conversation
|
@KimHyeongRae0 is attempting to deploy a commit to the Cytonic Team on Vercel. A member of the Team first needs to authorize it. |
00d8b1e to
0a96be3
Compare
|
Queued for substantive review. Touching citability scoring deserves a careful read — I'll come back after the current review queue clears. Thanks for the patience @KimHyeongRae0. |
|
@greptileai review |
Greptile SummaryThis PR adds an "answer-first" citability hint that fires when a page has at least one answer-quality paragraph but doesn't lead with one. It also extracts the inline answer-quality predicate into a reusable
Confidence Score: 5/5The change adds a new informational hint only — scores are unchanged and no existing paths are altered. The implementation is well-contained: the new hint fires only when answerCount > 0, the startsContextually check is logically correct, and the refactored isAnswerQualityParagraph helper is a straightforward extraction. The one test that passes for a slightly different reason than its comment claims does not affect production behaviour. src/core/citability.test.ts — the 200+ word opener regression test would be more robust with an explicit answer-quality paragraph to ensure answerCount > 0. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[scoreAnswerBlocks called] --> B[Loop paragraphs\ncount answer-quality via isAnswerQualityParagraph]
B --> C{answerCount == 0?}
C -->|yes| D[Push 'No direct answer\nparagraphs' warning]
C -->|no| E[Find firstSubstantialParagraph\n≥ 20 words, non-heading]
D --> F[Long-paragraph loop]
E --> G{firstSubstantialParagraph\nexists?}
G -->|no| F
G -->|yes| H{startsContextually?\nnot capital-non-pronoun}
H -->|yes| I[Push 'Lead with a direct\nanswer' suggestion]
H -->|no| F
I --> F
F --> J[Return CitabilityDimension]
Prompt To Fix All With AIFix the following 1 code review issue. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 1
src/core/citability.test.ts:115-129
**Regression test passes for the wrong reason**
The test comment says it validates that a 200+ word direct-answer opener doesn't trigger the hint via `startsContextually`. However, with this content `answerCount` is always 0: the long opener (~210 words) fails `goodLength` (`<= 200`), and the 14-word second paragraph fails the `>= 20` minimum. The `answerCount > 0` guard short-circuits the check before `startsContextually` is even reached, so the test doesn't exercise the path it claims to protect.
To properly exercise `startsContextually` for a 200+ word opener, the content needs at least one independent answer-quality paragraph (20–200 words, capital non-pronoun) so `answerCount` reaches 1 before the hint check runs.
Reviews (4): Last reviewed commit: "fix(citability): don't false-positive an..." | Re-trigger Greptile |
Greptile flagged two P2 issues on PR multivmlabs#51: 1. Threshold mismatch — firstSubstantialParagraph used >=15 words while isAnswerQualityParagraph requires >=20. A 15-19-word opening would be selected as "first substantial" but fail the answer-quality check even when it's actually a direct answer, firing a false-positive hint. Aligned both at >=20. 2. Double-hint noise — when answerCount === 0, both the existing warning ("No direct answer paragraphs found...") AND the new suggestion ("Lead with a direct answer paragraph...") fired. The suggestion is only useful when at least one answer paragraph already exists somewhere on the page. Gated the suggestion behind answerCount > 0. Adjusted the positive test to use a long-enough contextual first paragraph plus a real answer-quality second paragraph (so answerCount > 0). Added a new test asserting that pages with zero answer paragraphs get only the warning, not both hints. 15 tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Aligned thresholds at >=20 words and gated the new suggestion behind answerCount > 0 so it doesn't double up with the existing zero-answer warning. Test updated + new test for the zero-answer case. 15 tests pass. @greptileai re-review please. |
|
Want your agent to iterate on Greptile's feedback? Try greploops. |
…openers isAnswerQualityParagraph caps at 200 words. A well-formed 201+ word opener starting with a capital noun is still a direct answer — splitting it is what the long-paragraph hint is for. Check only the "starts contextually" half (capital noun, not pronoun) for the answer-first suggestion. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Fixed in f19fae9 — the answer-first hint no longer gates on isAnswerQualityParagraph (which caps at 200 words). It checks only the contextual-opener heuristic (capital noun, not pronoun), so a 201+ word opener that does lead with a direct answer no longer false-positives. Added a regression test covering the >200 word case. @greptileai review |
|
@greptileai review |
Adds an answer-first citability hint without changing scores. Validation:
npm run lint,npm run test -- --run src/core/citability.test.ts,npm run test -- --run,npm run build.