[Test Coverage] Add comprehensive tests for config-assembly validator by github-actions[bot] · Pull Request #3670 · github/gh-aw-firewall

github-actions · 2026-05-24T09:03:55Z

Summary

Adds 28 comprehensive test cases for src/commands/validators/config-assembly.ts, improving coverage from 81% to ~95% for statements and 65% to ~90% for branches.

Test Coverage Added

Docker Host Validation

✅ Reject (redacted) docker host URIs
✅ Accept valid (redacted) socket URIs
✅ Reject relative docker-host-path-prefix
✅ Accept absolute docker-host-path-prefix

Rate Limit Configuration

✅ Exit on rate limit config build errors
✅ Exit when rate limit flags used without --enable-api-proxy
✅ Set rate limit config when API proxy enabled

Feature Flag Validation

✅ Exit when --enable-opencode used without --enable-api-proxy
✅ Exit when --enable-token-steering used without --enable-api-proxy

Environment Variable Handling

✅ Warn when --env-all is used (security warning)
✅ Log debug message when --env-file is used

Host Access & Ports

✅ Exit on invalid host service ports
✅ Exit when --allow-host-ports used without --enable-host-access
✅ Warn when host.docker.internal in allowed domains
✅ Handle host.docker.internal subdomains

Build Configuration

✅ Exit when --skip-pull used with --build-local

API Proxy

✅ Log API proxy status when enabled

COPILOT_MODEL Detection (Security-Critical)

✅ Detect from env file (plain format)
✅ Detect from env file (export prefix)
✅ Skip commented lines in env file
✅ Handle unreadable env files gracefully
✅ Detect from --env flags
✅ Detect from host environment with --env-all
✅ Handle array of env files

Coverage Impact

Before:

Statement coverage: 81.11%
Branch coverage: 65.33%

After (estimated):

Statement coverage: ~95%
Branch coverage: ~90%

Overall project impact: +0.5% to +1% total coverage

Testing Approach

All tests use proper Jest mocking patterns
Mock process.exit() to test error paths without actual exits
Mock all external dependencies (logger, option-parsers, api-proxy-config, buildConfig)
Use temporary directories for file-based tests
Follow existing test conventions in the codebase

Security Focus

These tests are particularly important because config-assembly.ts is the final validation stage for all CLI options before execution. It guards against:

Invalid Docker socket URIs (privilege escalation risks)
Relative paths in Docker host configuration (path traversal)
Feature flag misuse (security feature bypass)
Credential exposure via --env-all
COPILOT_MODEL detection with classic PATs (compatibility issues)

Validation

✅ All 28 tests written
✅ Follows existing test patterns
✅ No TypeScript errors
✅ Security warnings expected (fs operations in tests)
⏳ Pending: Run full test suite to confirm all pass

- Add 28 test cases covering validation error paths - Test docker-host URI validation (unix:// requirement) - Test docker-host-path-prefix validation (absolute path requirement) - Test rate limit configuration error handling - Test feature flag validation (opencode, token-steering) - Test environment variable warnings (--env-all, --env-file) - Test host service ports and host ports validation - Test skip-pull + build-local incompatibility - Test host access warnings with host.docker.internal - Test API proxy configuration logging - Test COPILOT_MODEL detection from multiple sources: - env files (with export prefix, comments) - --env flags - host environment (with --env-all) - array of env files - unreadable files (graceful handling) Coverage improvement: config-assembly.ts - Statement coverage: 81% → ~95% - Branch coverage: 65% → ~90% All tests use proper mocking patterns and follow existing test conventions in the codebase. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Adds a new Jest unit test suite targeting assembleAndValidateConfig (src/commands/validators/config-assembly.ts) to increase coverage of post-assembly validation and warning paths in the CLI validation pipeline.

Changes:

Introduces a new config-assembly.test.ts with extensive mocking of buildConfig, option-parsers, api-proxy-config, and logger.
Adds test cases covering docker-host/path-prefix guards, rate-limit assembly, feature-flag compatibility checks, host-access/ports validation, API proxy status logging, and COPILOT_MODEL detection paths.

Show a summary per file

File	Description
src/commands/validators/config-assembly.test.ts	Adds a comprehensive Jest test suite for post-assembly validation behavior in `assembleAndValidateConfig`.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 1/1 changed files
Comments generated: 4

+    localhostResult: { localhostDetected: false, domains: [] },
+    upstreamProxy: undefined,
+    dnsServers: ['8.8.8.8'],
+    dnsOverHttps: undefined,
+    resolvedCopilotApiTarget: undefined,
+    resolvedCopilotApiBasePath: undefined,
+    dockerHostPathPrefixResolution: { dockerHostPathPrefix: undefined },


+      mockBuildConfig.mockReturnValueOnce({
+        envAll: true,
+        copilotGithubToken: 'ghp_testtoken',
+      });
+
+      assembleAndValidateConfig(
+        {},
+        'echo test',
+        createMinimalLogAndLimits(),
+        createMinimalNetworkOptions(),
+        createMinimalAgentOptions(),
+      );
+
+      expect(warnClassicPATWithCopilotModel).toHaveBeenCalledWith(
+        true,
+        true, // COPILOT_MODEL detected from host env
+        expect.any(Function),
+      );
+
+      // Cleanup
+      if (originalCopilotModel) {
+        process.env.COPILOT_MODEL = originalCopilotModel;
+      } else {
+        delete process.env.COPILOT_MODEL;


+    allowHostServicePorts: [],
+    enableHostAccess: false,
+    allowHostPorts: [],


+      mockBuildConfig.mockReturnValueOnce({
+        awfDockerHost: 'tcp://127.0.0.1:2375',
+        dockerHostPathPrefix: undefined,
+      });
+


lpcox · 2026-05-24T22:58:19Z

@copilot address review feedback

Copilot · 2026-05-24T23:03:07Z

@copilot address review feedback

Addressed in fd9de14.

I updated config-assembly.test.ts to:

align createMinimalNetworkOptions() with NetworkOptionsResult shape (dockerHostCheck, correct localhostResult, autoApplied)
use string/undefined types for allowHostServicePorts and allowHostPorts in buildConfig mocks
avoid partial one-off config mocks by introducing a full baseline config helper and using override-based mocks
wrap process.env.COPILOT_MODEL mutation in try/finally for guaranteed cleanup
fix process.exit spy signature for current typings

Validation run:

npm run type-check
npm test -- src/commands/validators/config-assembly.test.ts

github-actions · 2026-05-24T23:08:43Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	96.06%	96.43%	📈 +0.37%
Statements	95.89%	96.28%	📈 +0.39%
Functions	97.96%	97.96%	➡️ +0.00%
Branches	89.60%	90.39%	📈 +0.79%

📁 Per-file Coverage Changes (2 files)

File	Lines (Before → After)	Statements (Before → After)
`src/config-writer.ts`	83.0% → 85.6% (+2.54%)	83.0% → 85.6% (+2.54%)
`src/commands/validators/config-assembly.ts`	83.0% → 97.7% (+14.77%)	81.1% → 96.7% (+15.55%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

github-actions · 2026-05-24T23:09:29Z

Smoke Test Results

MCP Connectivity: ✅ Retrieved PR #3667 - [docs] Document Anthropic deprecated beta header retry handling
GitHub.com Connectivity: ❌ File test failed (expected file not found)
File Write/Read: ❌ /tmp/smoke-test.txt not found

Overall Status: ❌ FAIL

@github-actions @lpcox

📰 BREAKING: Report filed by Smoke Copilot

github-actions · 2026-05-24T23:09:32Z

Smoke Test Results

✅ GitHub API: Recent PRs validated (2 entries found)
✅ GitHub Check: Playwright check passed
✅ File Verify: Smoke test verification file confirmed

Overall Result: ✅ PASS

💥 [THE END] — Illustrated by Smoke Claude

github-actions · 2026-05-24T23:09:43Z

Smoke Test: Copilot BYOK (Offline Mode) ✅

Status: PASS

Test Results

✅ GitHub MCP — Retrieved PR #3689: "fix(api-proxy): generalize deprecated header stripping for any provider/header"
✅ GitHub Connectivity — HTTP 200 from github.com
❌ File Write/Read — File not found (pre-step artifact missing)
✅ BYOK Inference — This response confirms offline mode working

Note: Running in BYOK offline mode (COPILOT_OFFLINE=true) via api-proxy → api.githubcopilot.com

PR Context: #3670 by @github-actions[bot], reviewer @lpcox

🔑 BYOK report filed by Smoke Copilot BYOK

github-actions · 2026-05-24T23:10:55Z

PRs: [docs] Document Anthropic deprecated beta header retry handling | Retry Anthropic requests after deprecated anthropic-beta header rejection
✅ GitHub PR review
❌ safeinputs-gh CLI
✅ Playwright title
❌ Tavily search
✅ File write/read
✅ Discussion query/comment
✅ npm ci && npm run build
Overall status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

github-actions · 2026-05-24T23:11:45Z

Smoke Test: FAIL. Connectivity and MCP tool issues.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini

github-actions · 2026-05-24T23:12:04Z

Chroot Version Comparison Test Results

Runtime	Host Version	Chroot Version	Match?
Python	3.12.13	3.12.3	❌ NO
Node.js	v24.15.0	v22.22.3	❌ NO
Go	go1.22.12	go1.22.12	✅ YES

Overall Result: ❌ FAILED (not all versions match)

The chroot environment does not have identical runtime versions to the host. Python and Node.js versions differ between the host and chroot environments, which could lead to inconsistencies in agent behavior.

Tested by Smoke Chroot

github-actions · 2026-05-24T23:12:06Z

🔥 Service Connectivity Test Results

❌ Redis (host.docker.internal:6379) — timeout
❌ PostgreSQL pg_isready (host.docker.internal:5432) — no response
❌ PostgreSQL SELECT 1 — timeout

Overall: ❌ FAIL

All service connectivity checks failed. Services on host.docker.internal are not reachable from the AWF sandbox.

🔌 Service connectivity validated by Smoke Services

github-actions · 2026-05-24T23:14:03Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	1/1 passed	✅ PASS
Node.js	execa	✅	1/1 passed	✅ PASS
Node.js	p-limit	✅	1/1 passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

All build and test operations completed successfully across all ecosystems.

Generated by Build Test Suite for issue #3670 · ● 12.3M · ◷