test(github): enforce explicit ReadOnlyHint on every mcp.Tool literal#2486
test(github): enforce explicit ReadOnlyHint on every mcp.Tool literal#2486jluocsa wants to merge 3 commits into
Conversation
Adds a source-level (AST) validation test that walks every non-test Go file in pkg/github and fails if any mcp.Tool composite literal omits Annotations.ReadOnlyHint. The existing TestAllToolsHaveRequiredMetadata can only assert that Annotations is non-nil at runtime: Go cannot distinguish an unset bool field from one explicitly set to false. The new test closes that gap so future read-intent tools cannot silently default to ReadOnlyHint=false, which has caused downstream agents to prompt for human approval on safe read operations. All 97 current mcp.Tool registrations pass. Fault-injected by removing ReadOnlyHint from issue_read and confirmed the test reports the exact file, line, tool name, and reason. Refs github#2483
69ee9bb to
a973416
Compare
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds a static-analysis Go test to enforce that every mcp.Tool composite literal in pkg/github explicitly sets Annotations.ReadOnlyHint, preventing unintended defaulting to false.
Changes:
- Introduces an AST-based package scan over non-test Go files to find
mcp.Tool{...}literals. - Validates that
Annotationsis present and statically inspectable as amcp.ToolAnnotations{...}literal. - Fails the test with a detailed, file/line/tool-name-specific report when
ReadOnlyHintis not explicitly set.
- Resolve each file's local alias for github.com/modelcontextprotocol/go-sdk/mcp via file.Imports rather than hard-coding the "mcp" qualifier, so the check also covers files that import the SDK under a non-default alias. - Detect positional (unkeyed) composite literals and report a dedicated diagnostic instead of producing misleading "missing field" violations. - Drop the brittle 'expected to discover at least one mcp.Tool literal' assertion: if registrations move behind constructors/factories the AST walker legitimately finds nothing. - Use strconv.Unquote to decode tool-name string literals (handles escapes in interpreted strings); fall back to the raw lexeme on parse error.
|
@pachecocordovamoiseseduardo-byte Thanks for the review! The feedback was addressed in 3356343:
Could you take another look when you have a moment? 🙏 |
…package Move the AST-based ReadOnlyHint scan introduced in #2486 out of pkg/github's test file and into a new exported package, pkg/toolvalidation, so downstream consumers (notably github/github-mcp-server-remote, which uses this repo as a library) can apply the same guardrail to their own tool registrations with a one-line test: violations, err := toolvalidation.ScanReadOnlyHint(pkgDir) Changes: - New pkg/toolvalidation/readonlyhint.go with ScanReadOnlyHint, FormatReadOnlyHintViolations, and the ReadOnlyHintViolation type. - Dedicated unit tests for the scanner using in-memory fixtures (compliant, missing-hint, missing-annotations, non-literal, aliased import, positional fields, file without mcp import). - pkg/github/tools_static_validation_test.go shrunk to a thin wrapper that calls ScanReadOnlyHint against its own package directory; the existing behavior for pkg/github is preserved. No production-code, schema, or toolsnap changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Commits merged separately. |
|
To capture the rationale here for reviewers: The read-only ( One open question for maintainers on reuse: the logic currently lives in a |
1 similar comment
|
To capture the rationale here for reviewers: The read-only ( One open question for maintainers on reuse: the logic currently lives in a |
What
Adds a source-level (AST) validation test that statically walks every non-test
.gofile inpkg/github, finds everymcp.Tool{...}composite literal, and fails if any of them omits an explicitAnnotations.ReadOnlyHint:field.Refs #2483 (item 3 in the description: "Add a CI check that fails if any tool registered via
mcp.NewToollacks an explicitReadOnlyHint").Why
The existing
TestAllToolsHaveRequiredMetadataalready enforces non-nilAnnotationsand explicitly documents its limitation:That gap is exactly what #2483 wants closed. The Go runtime cannot tell an unset
boolfrom one explicitly set tofalse, but the AST can. This test surfaces the omission at build time so a future read-intent tool cannot silently default toReadOnlyHint: false— which has triggered downstream agents to prompt for human approval on safe read operations.How
TestAllToolRegistrationsExplicitlySetReadOnlyHintinpkg/github/tools_static_validation_test.go.go/parser+go/ast(stdlib only — no new dependencies).*ast.CompositeLitof typemcp.Tool:Annotations:is present.&mcp.ToolAnnotations{...}literal that can be statically inspected (anything else is flagged so a human can confirm).ReadOnlyHint:is among the keyed fields.<file>:<line> tool=<name>: <reason>.Verification
go test ./pkg/github -run TestAllToolRegistrationsExplicitlySetReadOnlyHint -v→ PASS (97/97 current tool literals are compliant).ReadOnlyHint: truefromissue_read:go test ./...→ all green.go vet ./pkg/github/...→ clean.Scope notes
This PR is intentionally minimal: it covers item 3 of #2483 (the CI guardrail). The audit portion of the issue (items 1–2) is currently a no-op against
mainbecause all 97 registrations already set the hint, so the value of this change is purely forward-protection. Happy to fold in additional checks (e.g., explicitTitle:annotation, OWASP-flavored side-effect classification) as follow-ups if maintainers want them.