Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/hooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ hooks.Register(engine.BeforeToolExec, func(ctx context.Context, hctx *engine.Hoo

`AfterToolExec` hooks can modify `hctx.ToolOutput` to redact sensitive content before it enters the LLM context. The agent loop reads back `ToolOutput` from the `HookContext` after all hooks fire.

The runner registers a guardrail hook that scans tool output for secrets and PII patterns. See [Tool Output Scanning](security/guardrails.md#tool-output-scanning) for details.
The runner registers a guardrail hook that scans tool output for secrets and PII patterns. The hook passes `hctx.ToolName` to the guardrail engine, enabling per-tool exemptions via `allow_tools` config. See [Tool Output Scanning](security/guardrails.md#tool-output-scanning) for details.

```go
hooks.Register(engine.AfterToolExec, func(ctx context.Context, hctx *engine.HookContext) error {
Expand Down
37 changes: 35 additions & 2 deletions docs/security/guardrails.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ All four built-in guardrails (`content_filter`, `no_pii`, `jailbreak_protection`

## Tool Output Scanning

The guardrail engine scans tool output via an `AfterToolExec` hook, catching secrets and PII before they enter the LLM context or outbound messages.
The guardrail engine scans tool output via an `AfterToolExec` hook, catching secrets and PII before they enter the LLM context or outbound messages. The hook passes the tool name to enable per-tool exemptions (see [Per-Tool PII Exemptions](#per-tool-pii-exemptions) below).

| Guardrail | What it detects in tool output |
|-----------|-------------------------------|
Expand All @@ -86,11 +86,44 @@ The guardrail engine scans tool output via an `AfterToolExec` hook, catching sec

| Mode | Behavior |
|------|----------|
| `enforce` | Returns a generic error (`"tool output blocked by content policy"`), blocking the result from entering the LLM context. The error message intentionally omits which guardrail matched to avoid leaking security internals to the LLM or channel. |
| `enforce` | Returns an error identifying the guardrail that triggered (e.g., `"tool output blocked by no_pii guardrail (PII detected in output)"`), blocking the result from entering the LLM context. |
| `warn` | Replaces matched patterns with `[REDACTED]`, logs a warning, and allows the redacted output through |

The hook writes the redacted text back to `HookContext.ToolOutput`, which the agent loop reads after all hooks fire. This is backwards-compatible — existing hooks that don't modify `ToolOutput` leave it unchanged.

### Per-Tool PII Exemptions

Some tools legitimately return PII as part of their function (e.g., `github_get_user` returning public email addresses). The `allow_tools` config option lets specific tools bypass a guardrail entirely.

```json
{
"guardrails": [
{
"type": "no_pii",
"config": {
"allow_tools": [
"github_get_user",
"github_pr_author_profiles",
"github_stargazer_profiles",
"file_create",
"code_agent_write",
"code_agent_edit"
]
}
}
]
}
```

**Key behaviors:**

| Behavior | Detail |
|----------|--------|
| Per-guardrail scope | `allow_tools` on `no_pii` does **not** bypass `no_secrets` — each guardrail has its own allowlist |
| Write tools included | `file_create`, `code_agent_write`, and `code_agent_edit` are included because they echo back content the LLM already has — blocking the echo is redundant |
| Default config | The default policy scaffold pre-configures `allow_tools` for GitHub profile tools and write tools |
| Custom overrides | Override via `policy-scaffold.json` to add or remove tools from the allowlist |

## Path Containment

The `cli_execute` tool confines filesystem path arguments to the agent's working directory. This prevents social-engineering attacks where an LLM is tricked into listing or reading files outside the project.
Expand Down
14 changes: 12 additions & 2 deletions docs/skills.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ forge skills list --tags kubernetes,incident-response

| Skill | Icon | Category | Description | Scripts |
|-------|------|----------|-------------|---------|
| `github` | 🐙 | developer | Clone repos, create issues/PRs, and manage git workflows | `github-clone.sh`, `github-checkout.sh`, `github-commit.sh`, `github-push.sh`, `github-create-pr.sh`, `github-status.sh` |
| `github` | 🐙 | developer | Clone repos, create issues/PRs, query GitHub API, and manage git workflows | `github-clone.sh`, `github-checkout.sh`, `github-commit.sh`, `github-push.sh`, `github-create-pr.sh`, `github-status.sh`, `github-list-prs.sh`, `github-get-user.sh`, `github-list-stargazers.sh`, `github-list-forks.sh`, `github-pr-author-profiles.sh`, `github-stargazer-profiles.sh` |
| `code-agent` | 🤖 | developer | Autonomous code generation, modification, and project scaffolding | — (builtin tools) |
| `weather` | 🌤️ | utilities | Get weather data for a location | — (binary-backed) |
| `tavily-search` | 🔍 | research | Search the web using Tavily AI search API | `tavily-search.sh` |
Expand Down Expand Up @@ -365,7 +365,7 @@ The `github` skill provides a complete git + GitHub workflow through script-back
forge skills add github
```

This registers eight tools:
This registers fourteen tools:

| Tool | Purpose |
|------|---------|
Expand All @@ -377,9 +377,19 @@ This registers eight tools:
| `github_create_pr` | Create a pull request |
| `github_create_issue` | Create a GitHub issue |
| `github_list_issues` | List open issues for a repository |
| `github_list_prs` | List pull requests with state filter and pagination |
| `github_get_user` | Get a GitHub user's public profile |
| `github_list_stargazers` | List stargazers for a repository with pagination |
| `github_list_forks` | List forks of a repository with pagination |
| `github_pr_author_profiles` | List PR authors and fetch their full profiles (compound 2-step) |
| `github_stargazer_profiles` | List stargazers and fetch their full profiles (compound 2-step) |

**Workflow:** Clone → explore → edit → status → commit → push → create PR. The skill's system prompt enforces this sequence and prevents raw `git` commands via `cli_execute`.

**Pagination:** List tools (`github_list_prs`, `github_list_stargazers`, `github_list_forks`, `github_pr_author_profiles`, `github_stargazer_profiles`) support `page` (1-based) and `per_page` (default 30, max 100) parameters. Responses include `pagination.has_next_page` to indicate more results are available.

**PII exemption:** Profile-returning tools (`github_get_user`, `github_pr_author_profiles`, `github_stargazer_profiles`) are pre-configured in the default policy scaffold's `no_pii` `allow_tools` list, so they can return public profile data (emails, bios) without triggering PII guardrails. See [Per-Tool PII Exemptions](security/guardrails.md#per-tool-pii-exemptions).

Requires: `gh`, `git`, `jq`. Optional: `GH_TOKEN`. Egress: `api.github.com`, `github.com`.

### Code-Agent Skill
Expand Down
14 changes: 13 additions & 1 deletion forge-cli/runtime/guardrails_loader.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,19 @@ func DefaultPolicyScaffold() *agentspec.PolicyScaffold {
Type: "content_filter",
Config: map[string]any{"enabled": true},
},
{Type: "no_pii"},
{
Type: "no_pii",
Config: map[string]any{
"allow_tools": []any{
"github_get_user",
"github_pr_author_profiles",
"github_stargazer_profiles",
"file_create",
"code_agent_write",
"code_agent_edit",
},
},
},
{Type: "jailbreak_protection"},
{Type: "no_secrets"},
},
Expand Down
2 changes: 1 addition & 1 deletion forge-cli/runtime/runner.go
Original file line number Diff line number Diff line change
Expand Up @@ -1474,7 +1474,7 @@ func (r *Runner) registerGuardrailHooks(hooks *coreruntime.HookRegistry, guardra
if hctx.ToolOutput == "" {
return nil
}
redacted, err := guardrails.CheckToolOutput(hctx.ToolOutput)
redacted, err := guardrails.CheckToolOutput(hctx.ToolName, hctx.ToolOutput)
if err != nil {
return err
}
Expand Down
37 changes: 34 additions & 3 deletions forge-core/runtime/guardrails.go
Original file line number Diff line number Diff line change
Expand Up @@ -252,20 +252,30 @@ func (g *GuardrailEngine) checkNoSecrets(text string) error {
// because tool outputs are internal (sent to the LLM, not the user) and
// blocking would kill the entire agent session. Search tools routinely find
// code containing API key patterns in test files, config examples, etc.
func (g *GuardrailEngine) CheckToolOutput(text string) (string, error) {
//
// The toolName parameter enables per-tool PII exemptions: if a guardrail's
// config contains "allow_tools" (a list of tool names), tools in that list
// skip the corresponding check. This lets tools like github_get_user return
// public profile data (emails, bios) without triggering PII blocks.
func (g *GuardrailEngine) CheckToolOutput(toolName, text string) (string, error) {
if text == "" {
return text, nil
}

for _, gr := range g.scaffold.Guardrails {
// Check if this tool is in the guardrail's allow_tools list.
if g.toolAllowed(toolName, gr) {
continue
}

switch gr.Type {
case "no_secrets":
for _, re := range secretPatterns {
if !re.MatchString(text) {
continue
}
if g.enforce {
return "", fmt.Errorf("tool output blocked by content policy")
return "", fmt.Errorf("tool output blocked by no_secrets guardrail (secret/credential detected in output)")
}
text = re.ReplaceAllString(text, "[REDACTED]")
g.logger.Warn("guardrail redaction", map[string]any{
Expand Down Expand Up @@ -295,7 +305,7 @@ func (g *GuardrailEngine) CheckToolOutput(text string) (string, error) {
continue
}
if g.enforce {
return "", fmt.Errorf("tool output blocked by content policy")
return "", fmt.Errorf("tool output blocked by no_pii guardrail (PII detected in output)")
}
// Warn mode: redact only validated matches
if p.validate != nil {
Expand All @@ -322,6 +332,27 @@ func (g *GuardrailEngine) CheckToolOutput(text string) (string, error) {
return text, nil
}

// toolAllowed checks whether toolName is in the guardrail's "allow_tools" config list.
func (g *GuardrailEngine) toolAllowed(toolName string, gr agentspec.Guardrail) bool {
if toolName == "" || gr.Config == nil {
return false
}
allowRaw, ok := gr.Config["allow_tools"]
if !ok {
return false
}
list, ok := allowRaw.([]any)
if !ok {
return false
}
for _, v := range list {
if s, ok := v.(string); ok && s == toolName {
return true
}
}
return false
}

// --- PII Validators ---
// Ported from the reference guardrails library to reduce false positives.

Expand Down
92 changes: 88 additions & 4 deletions forge-core/runtime/guardrails_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ func TestCheckToolOutput_RedactsWithValidation(t *testing.T) {
}, false, logger) // warn mode

// Valid SSN should be redacted
out, err := g.CheckToolOutput("SSN is 456-78-9012")
out, err := g.CheckToolOutput("some_tool", "SSN is 456-78-9012")
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
Expand All @@ -211,7 +211,7 @@ func TestCheckToolOutput_RedactsWithValidation(t *testing.T) {
}

// Invalid SSN (area 000) should NOT be redacted
out, err = g.CheckToolOutput("code 000-12-3456 here")
out, err = g.CheckToolOutput("some_tool", "code 000-12-3456 here")
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
Expand All @@ -228,7 +228,7 @@ func TestCheckToolOutput_K8sBytesNotBlocked(t *testing.T) {

// K8s memory byte counts should not trigger PII detection
k8sOutput := `{"memory": "4294967296", "cpu": "2000m", "pods": "110", "allocatable_memory": "3221225472"}`
out, err := g.CheckToolOutput(k8sOutput)
out, err := g.CheckToolOutput("some_tool", k8sOutput)
if err != nil {
t.Fatalf("k8s output blocked as PII: %v", err)
}
Expand All @@ -243,10 +243,94 @@ func TestCheckToolOutput_EnforceBlocksValidPII(t *testing.T) {
Guardrails: []agentspec.Guardrail{{Type: "no_pii"}},
}, true, logger) // enforce mode

_, err := g.CheckToolOutput("SSN: 456-78-9012")
_, err := g.CheckToolOutput("some_tool", "SSN: 456-78-9012")
if err == nil {
t.Error("expected enforce mode to block valid SSN")
}
if !strings.Contains(err.Error(), "no_pii") {
t.Errorf("expected error to mention no_pii guardrail, got: %v", err)
}
}

func TestCheckToolOutput_AllowToolsBypassesPII(t *testing.T) {
logger := &testLogger{}
g := NewGuardrailEngine(&agentspec.PolicyScaffold{
Guardrails: []agentspec.Guardrail{
{
Type: "no_pii",
Config: map[string]any{
"allow_tools": []any{"github_get_user", "github_pr_author_profiles"},
},
},
},
}, true, logger) // enforce mode

// Allowed tool should pass through with PII
out, err := g.CheckToolOutput("github_get_user", `{"email": "user@example.com"}`)
if err != nil {
t.Fatalf("allowed tool should not be blocked: %v", err)
}
if !strings.Contains(out, "user@example.com") {
t.Error("expected email to pass through for allowed tool")
}

// Non-allowed tool should still be blocked
_, err = g.CheckToolOutput("some_other_tool", `{"email": "user@example.com"}`)
if err == nil {
t.Error("expected non-allowed tool to be blocked for PII")
}
}

func TestCheckToolOutput_AllowToolsOnlyAffectsConfiguredGuardrail(t *testing.T) {
logger := &testLogger{}
g := NewGuardrailEngine(&agentspec.PolicyScaffold{
Guardrails: []agentspec.Guardrail{
{Type: "no_secrets"}, // no allow_tools — applies to all tools
{
Type: "no_pii",
Config: map[string]any{
"allow_tools": []any{"github_get_user"},
},
},
},
}, true, logger)

// Allowed tool bypasses PII but NOT secrets
_, err := g.CheckToolOutput("github_get_user", "token: ghp_abcdefghijklmnopqrstuvwxyz0123456789")
if err == nil {
t.Error("allow_tools for no_pii should not bypass no_secrets")
}
if !strings.Contains(err.Error(), "no_secrets") {
t.Errorf("expected error to mention no_secrets, got: %v", err)
}
}

func TestCheckToolOutput_ErrorMessageMentionsGuardrailType(t *testing.T) {
logger := &testLogger{}

// Test no_secrets error message
g := NewGuardrailEngine(&agentspec.PolicyScaffold{
Guardrails: []agentspec.Guardrail{{Type: "no_secrets"}},
}, true, logger)
_, err := g.CheckToolOutput("some_tool", "key: sk-ant-abcdefghijklmnopqrstuv")
if err == nil {
t.Fatal("expected error for secret")
}
if !strings.Contains(err.Error(), "no_secrets") {
t.Errorf("expected error to mention no_secrets, got: %v", err)
}

// Test no_pii error message
g2 := NewGuardrailEngine(&agentspec.PolicyScaffold{
Guardrails: []agentspec.Guardrail{{Type: "no_pii"}},
}, true, logger)
_, err = g2.CheckToolOutput("some_tool", "email: test@example.com")
if err == nil {
t.Fatal("expected error for PII")
}
if !strings.Contains(err.Error(), "no_pii") {
t.Errorf("expected error to mention no_pii, got: %v", err)
}
}

// --- CheckOutbound message tests ---
Expand Down
4 changes: 3 additions & 1 deletion forge-core/runtime/loop.go
Original file line number Diff line number Diff line change
Expand Up @@ -672,7 +672,9 @@ func toolPhase(name string) workflowPhase {
switch name {
case "github_clone", "code_agent_scaffold", "github_checkout":
return phaseSetup
case "code_agent_read", "grep_search", "glob_search", "directory_tree", "read_skill", "github_status":
case "code_agent_read", "grep_search", "glob_search", "directory_tree", "read_skill", "github_status",
"github_list_prs", "github_get_user", "github_list_stargazers", "github_list_forks",
"github_pr_author_profiles", "github_stargazer_profiles":
return phaseExplore
case "code_agent_edit", "code_agent_write", "code_agent_patch", "file_create", "code_agent_run":
return phaseEdit
Expand Down
6 changes: 6 additions & 0 deletions forge-core/runtime/loop_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -368,6 +368,12 @@ func TestToolPhaseClassification(t *testing.T) {
{"directory_tree", phaseExplore},
{"read_skill", phaseExplore},
{"github_status", phaseExplore},
{"github_list_prs", phaseExplore},
{"github_get_user", phaseExplore},
{"github_list_stargazers", phaseExplore},
{"github_list_forks", phaseExplore},
{"github_pr_author_profiles", phaseExplore},
{"github_stargazer_profiles", phaseExplore},
{"code_agent_edit", phaseEdit},
{"code_agent_write", phaseEdit},
{"code_agent_patch", phaseEdit},
Expand Down
Loading
Loading