Add sandboxed network-isolated test execution#987
Open
Soner (shyim) wants to merge 8 commits intomainfrom
Open
Add sandboxed network-isolated test execution#987Soner (shyim) wants to merge 8 commits intomainfrom
Soner (shyim) wants to merge 8 commits intomainfrom
Conversation
Adds scripts/run-tests.sh that runs the Go test suite inside a network-disabled sandbox: sandbox-exec with the existing sandbox-no-network.sb profile on macOS, and bubblewrap with --unshare-net on Linux. The GitHub Actions workflow installs bubblewrap and invokes the script so any test that performs a network call fails, matching the constraints of the NixOS build.
The sandboxed go test run has no network, so any uncompiled test package or unresolved module would hang or fail with a confusing proxy error. Pre-build all test binaries with `go test -run='^$'` while the network is still available, then enter the sandbox with GOPROXY=off and -mod=readonly so any cache miss surfaces as a fast, explicit error rather than a network attempt.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5bf54e3b0c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
| echo "error: sandbox-exec not found" >&2 | ||
| exit 1 | ||
| fi | ||
| exec sandbox-exec -f "$REPO_DIR/sandbox-no-network.sb" go test "$@" |
There was a problem hiding this comment.
The Darwin branch executes sandbox-exec with sandbox-no-network.sb, but that profile currently allows network* and only denies outbound TCP on ports 80/443. This means tests can still access external services over other ports/protocols, so the new script does not actually provide the claimed "no network" isolation on macOS.
Useful? React with 👍 / 👎.
util-linux is preinstalled on every linux distro and on the github ubuntu-latest runner, so `unshare --user --map-root-user --net` gives us a netns-isolated test run with zero apt install. drops the extra workflow step and avoids needing harden-runner allowlist entries for ubuntu mirrors.
Ubuntu 24.04 (the ubuntu-latest image) ships with kernel.apparmor_restrict_unprivileged_userns=1, which lets unshare create the user namespace but blocks the uid_map write, so `unshare --user --map-root-user --net` exits non-zero even though the wrapped command ran successfully. Toggle the sysctl off before the sandboxed test step so the wrapper exits cleanly.
Without an active lo interface, tests that use httptest.NewServer (bound to 127.0.0.1) and UDP loopback (e.g. internal/tracking) fail even though they make no external network calls. Match the nix-build sandbox semantics by bringing lo up inside the namespace so only external traffic is blocked.
Three classes of test failures showed up under the new sandboxed test runner; this commit addresses the ones that need code changes (loopback-only failures are handled by the runner script itself). - internal/phplint, internal/esbuild: tests that fetch PHP wasm binaries or dart-sass from github.com now also skip when SHOPWARE_CLI_NO_NETWORK is set, alongside the existing NIX_CC skip. The sandbox script exports that variable. - internal/extension/TestValidateTheme_ReadError: the test relied on a 0000-permission file to provoke a read error, which root bypasses. Replaced with a directory at the same path so os.ReadFile fails regardless of UID.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This change adds a new test execution script that runs Go tests in a sandboxed environment with network access disabled, ensuring tests don't depend on external network resources.
Key Changes
New script
scripts/run-tests.sh: Implements platform-specific sandboxing for test executionsandbox-execwith a sandbox profile to restrict network accessbubblewrap(bwrap) to create a network-isolated containerGOPROXY=offand-mod=readonlyflags to fail fast on any cache misses-v ./...)Updated GitHub Actions workflow: Modified the test step to use the new sandboxed test script
bubblewrapinstallation for Linux runnersgo test -v ./...to./scripts/run-tests.shImplementation Details
set -euo pipefailfor strict error handlinghttps://claude.ai/code/session_01W1JE67zRHGp8JndUxGRDaF