Skip to content

Fail closed on unknown toolsets in strict mode#2184

Open
macarronesc wants to merge 1 commit intogithub:mainfrom
macarronesc:fix/2117-strict-toolset-validation
Open

Fail closed on unknown toolsets in strict mode#2184
macarronesc wants to merge 1 commit intogithub:mainfrom
macarronesc:fix/2117-strict-toolset-validation

Conversation

@macarronesc
Copy link

Closes #2117

Summary

This change adds explicit fail-closed validation for unknown toolset names when strict validation is enabled.

Unknown toolset names were already detected during inventory construction, but only surfaced as warnings. In stricter deployments that rely on explicit allow-lists, that can hide configuration mistakes and weaken the intended permission boundary.

Changes

  • add strict toolset validation to the inventory builder
  • fail stdio startup when strict-toolsets is enabled and unknown toolsets are configured
  • preserve existing warning-only behavior by default for backward compatibility
  • add unit and integration coverage for both strict and non-strict behavior
  • document strict-toolsets in the README and server configuration guide

Validation

  • script/lint
  • script/test
  • go test ./pkg/inventory -run 'TestBuildErrorsOnUnrecognizedToolsetsWhenStrict|TestBuildAllowsUnrecognizedToolsetsWhenNotStrict' -v
  • go test ./internal/ghmcp -run 'TestNewStdioMCPServer_StrictToolsetValidation|TestNewStdioMCPServer_AllowsUnknownToolsetsWhenNotStrict' -v
  • go test ./pkg/github -run TestResolveEnabledToolsets -v

Manual verification

Without strict validation:

  • GITHUB_PERSONAL_ACCESS_TOKEN=dummy go run ./cmd/github-mcp-server stdio --toolsets=repos,typo
  • Result: warning logged, server starts

With strict validation:

  • GITHUB_PERSONAL_ACCESS_TOKEN=dummy go run ./cmd/github-mcp-server stdio --toolsets=repos,typo --strict-toolsets
  • Result: startup fails with unknown toolsets specified in WithToolsets: typo

Copilot AI review requested due to automatic review settings March 8, 2026 19:23
@macarronesc macarronesc requested a review from a team as a code owner March 8, 2026 19:23
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a strict validation mode to the local (stdio) server startup path so misconfigured/unknown toolset names fail closed instead of only logging warnings, strengthening allow-list based deployments.

Changes:

  • Add strict toolset validation to pkg/inventory builder (error on unknown toolsets when enabled).
  • Thread strict validation configuration from CLI (--strict-toolsets) through ghmcp into inventory building.
  • Add unit/integration tests covering strict vs non-strict behavior and document the new mode in the server configuration guide.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
pkg/inventory/builder.go Adds strict mode option and returns an error when unknown toolsets are configured under strict validation.
pkg/inventory/registry_test.go Adds tests ensuring strict mode errors and non-strict mode records unknown toolsets.
pkg/github/server.go Extends MCPServerConfig with StrictToolsetValidation to carry config through server creation.
internal/ghmcp/server.go Passes strict validation setting into the inventory builder during stdio server creation.
internal/ghmcp/server_test.go Adds coverage for stdio server strict vs non-strict startup behavior.
cmd/github-mcp-server/main.go Introduces --strict-toolsets flag and wires it via viper into stdio config.
docs/server-configuration.md Documents strict toolset validation behavior and provides a local-server config example.

Comment on lines +341 to +343
**Best for:** Locked-down environments where toolset allow-lists must fail closed.

By default, unknown toolset names are ignored and logged as warnings so existing configurations remain backward compatible. If you want startup to fail when a configured toolset name is unknown, enable strict validation.
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description says strict-toolsets is documented in the README, but there don’t appear to be any README updates in this change set (and the only doc update is in this file). Either update README.md (tool configuration section) to mention --strict-toolsets / GITHUB_STRICT_TOOLSETS, or adjust the PR description so it matches what’s actually being changed.

Copilot uses AI. Check for mistakes.
Comment on lines +339 to +343
### Strict Toolset Validation

**Best for:** Locked-down environments where toolset allow-lists must fail closed.

By default, unknown toolset names are ignored and logged as warnings so existing configurations remain backward compatible. If you want startup to fail when a configured toolset name is unknown, enable strict validation.
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new Strict Toolset Validation section doesn’t mention the corresponding env var (viper key strict_toolsetsGITHUB_STRICT_TOOLSETS) and the doc’s “Quick Reference” table near the top doesn’t list this new configuration option. To keep docs consistent with other settings, consider documenting both the flag and env var here and adding an entry to the Quick Reference table (Local Server column).

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fail closed in strict mode on unknown toolset names

2 participants