Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 3 additions & 10 deletions .devcontainer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This devcontainer provides a complete development environment for the Galactic m
## Features

### Languages & Runtimes
- **Go 1.24.2** - For operator, agent, and CNI development
- **Go 1.24.2** - For agent and CNI development

### Kubernetes Tools
- **kubectl** - Kubernetes CLI
Expand Down Expand Up @@ -82,11 +82,10 @@ These are required for:

The `post-create.sh` script automatically:
1. Installs Go development tools (gopls, delve, golangci-lint)
2. Installs Kubernetes tools (controller-gen, kustomize, setup-envtest, kind)
2. Installs Kubernetes tools (kustomize, kind)
3. Installs Protocol Buffer compiler and Go plugins
4. Installs network diagnostic tools
5. Generates Kubernetes manifests and DeepCopy methods
7. Configures git safe directory
5. Configures git safe directory

## Getting Started

Expand All @@ -99,12 +98,6 @@ make build
# Run unit tests
make test

# Run E2E tests (creates a Kind cluster)
make test-e2e

# Run the operator locally
make run-operator

# Run the agent locally
make run-agent

Expand Down
11 changes: 11 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Galactic is the SRv6 data plane for multi-cloud VPC networking. It consists of a
- VPC identifiers are 48-bit hex; VPCAttachment identifiers are 16-bit hex. These are embedded into IPv6 SRv6 endpoint addresses for deterministic route lookups. Both are supplied by an external operator via the CNI config.
- Identifiers are also Base62-encoded for interface naming (VRF: `vrfX-Y`, veth host side: `galX-Y`) to keep kernel interface name length within limits.
- The binary auto-detects CNI mode via the `CNI_COMMAND` env var; otherwise runs as a Cobra CLI with `agent`, `cni`, and `version` subcommands.
- The Kubernetes operator, VPC/VPCAttachment CRDs, and webhook code have been removed from this repository. They live in a separate companion operator project.

## Tech Stack

Expand Down Expand Up @@ -43,6 +44,15 @@ Summary:
- **Known debt:** Agent and CNI kernel-path code (`internal/agent/srv6/`, `internal/cni/`) has no unit coverage; these paths are best covered by integration or e2e tests. Only `pkg/common/util` has unit test coverage.
- **In flux:** The SRv6 route management (`internal/agent/srv6/`) and VRF utilities (`pkg/common/vrf/`) are the least tested and most likely to change as multi-cloud routing matures. BGP integration is in progress.

## Lab Environments

Two ContainerLab-based environments live under `lab/`:

- **`lab/network/`** — Standalone SRv6 underlay lab. Eight FRR + GoBGP nodes across PE, transit, and route-reflector roles. Use to develop and test BGP/SRv6 routing behaviour independently of Kubernetes.
- **`lab/gvpc/`** — Three Kind clusters (iad, sjc, infra) wired over an SRv6 transit mesh. FRR runs as a hostNetwork DaemonSet on each cluster's worker for the eBGP underlay; GoBGP on iad and sjc workers handles L3VPN type-5 routes over iBGP to the infra route reflector.

See `lab/README.md` for quick-start commands and prerequisites for each environment.

## New Developer Entry Points

1. Run `make build` to verify toolchain; run `make test` to confirm unit tests pass.
Expand All @@ -53,3 +63,4 @@ Summary:

**Likely trip-ups:**
- `make run-agent` requires elevated privileges (netlink, VRF, SRv6 operations need `CAP_NET_ADMIN`).
- There is no operator or webhook in this repository; those components are in a separate project.
24 changes: 6 additions & 18 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,7 @@ vet: ## Run go vet against code.
test: fmt vet ## Run tests.
GOOS=linux go test $$(go list ./... | grep -v /e2e) -coverprofile cover.out

# TODO(user): To use a different vendor for e2e tests, modify the setup under 'tests/e2e'.
# The default setup assumes Kind is pre-installed and builds/loads the Manager Docker image locally.
# CertManager is installed by default; skip with:
# - CERT_MANAGER_INSTALL_SKIP=true
KIND_CLUSTER ?= galactic-test-e2e
KIND_CLUSTER ?= galactic-e2e

.PHONY: setup-test-e2e
setup-test-e2e: ## Set up a Kind cluster for e2e tests if it does not exist
Expand All @@ -73,11 +69,6 @@ setup-test-e2e: ## Set up a Kind cluster for e2e tests if it does not exist
$(KIND) create cluster --name $(KIND_CLUSTER) ;; \
esac

.PHONY: test-e2e
test-e2e: setup-test-e2e fmt vet ## Run the e2e tests. Expected an isolated environment using Kind.
KIND_CLUSTER=$(KIND_CLUSTER) go test ./test/e2e/ -v -ginkgo.v
$(MAKE) cleanup-test-e2e

.PHONY: cleanup-test-e2e
cleanup-test-e2e: ## Tear down the Kind cluster used for e2e tests
@$(KIND) delete cluster --name $(KIND_CLUSTER)
Expand Down Expand Up @@ -108,7 +99,7 @@ build: fmt vet ## Build galactic binary.
run-agent: fmt vet ## Run agent from your host.
go run ./cmd/galactic/main.go agent

# If you wish to build the manager image targeting other platforms you can use the --platform flag.
# If you wish to build the image targeting other platforms you can use the --platform flag.
# (i.e. docker build --platform linux/arm64). However, you must enable docker buildKit for it.
# More info: https://docs.docker.com/develop/develop-images/build_enhancements/
.PHONY: docker-build
Expand All @@ -119,13 +110,10 @@ docker-build: ## Build docker image with the unified binary.
docker-push: ## Push docker image with the unified binary.
$(CONTAINER_TOOL) push ${IMG}

# PLATFORMS defines the target platforms for the manager image be built to provide support to multiple
# architectures. (i.e. make docker-buildx IMG=myregistry/mypoperator:0.0.1). To use this option you need to:
# - be able to use docker buildx. More info: https://docs.docker.com/build/buildx/
# - have enabled BuildKit. More info: https://docs.docker.com/develop/develop-images/build_enhancements/
# - be able to push the image to your registry (i.e. if you do not set a valid value via IMG=<myregistry/image:<tag>> then the export will fail)
# To adequately provide solutions that are compatible with multiple platforms, you should consider using this option.
PLATFORMS ?= linux/arm64,linux/amd64,linux/s390x,linux/ppc64le
# PLATFORMS defines the target platforms for the image build. Requires docker buildx and BuildKit.
# Usage: make docker-buildx IMG=myregistry/galactic:0.0.1
# Requires IMG to be set to a valid registry/tag or the push will fail.
PLATFORMS ?= linux/amd64,linux/arm64
.PHONY: docker-buildx
docker-buildx: ## Build and push docker image for the unified binary for cross-platform support
# copy existing Dockerfile and insert --platform=${BUILDPLATFORM} into Dockerfile.cross, and preserve the original Dockerfile
Expand Down
7 changes: 6 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,12 @@ Under the hood, Galactic uses Segment Routing over IPv6 (SRv6) for efficient, de

## Getting Started

See the [`lab/`](./lab/) directory for example topologies and the [DevContainer](./.devcontainer/) for development environment setup.
Two ContainerLab environments are available under [`lab/`](./lab/):

- **[`lab/network/`](./lab/network/)** — Standalone SRv6 underlay lab (FRR + GoBGP, no Kubernetes). Good starting point for understanding the routing layer.
- **[`lab/gvpc/`](./lab/gvpc/)** — Three Kind clusters wired over an SRv6 transit mesh. The full GVPC multi-cluster environment.

See the [DevContainer](./.devcontainer/) for development environment setup.

## License

Expand Down
82 changes: 18 additions & 64 deletions lab/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,8 @@ Local development and integration-testing environments for [Galactic VPC](https:

```
lab/
├── network/ # ContainerLab SRv6 underlay network
└── containers/
└── kindest-node-galactic/ # Custom Kind node image with Galactic pre-installed
├── network/ # ContainerLab SRv6 underlay network (standalone)
└── gvpc/ # ContainerLab GVPC multi-cluster lab (3 Kind clusters over SRv6 mesh)
```

---
Expand Down Expand Up @@ -38,74 +37,29 @@ make down # tear down

---

## `containers/kindest-node-galactic/` — Custom Kind Node Image
## `gvpc/` — GVPC Multi-Cluster Lab

A `kindest/node` image extended with the tooling and Kubernetes manifests needed to
run a full Galactic stack inside a [Kind](https://kind.sigs.k8s.io/) cluster.
Three Kind clusters (iad, sjc, infra) connected over an IPv6 SRv6 transit mesh. Each
cluster runs FRR as a node routing daemon (hostNetwork DaemonSet) to peer with the
transit layer via BGP unnumbered. GoBGP runs alongside FRR on the iad and sjc workers
to exchange L3VPN type-5 routes with the infra route reflector over iBGP.

```
kindest-node-galactic/
├── Dockerfile
├── resources/ # Kubernetes manifests applied at cluster boot
│ ├── agent.k8s.yaml
│ └── operator.k8s.yaml
└── scripts/
└── install.sh # Installs Cilium, cert-manager, Multus, and Galactic
```

`install.sh` is invoked once per node after the cluster comes up. On the control-plane
node it applies each Kubernetes manifest in order (Cilium → cert-manager → Multus →
Galactic operator → agent). On worker nodes it loads kernel modules,
sets SRv6 sysctls, and drops in the CNI binaries.
See [`gvpc/README.md`](gvpc/README.md) for topology details, addressing, and
verification commands.

### Prerequisites

- ContainerLab ≥ 0.54
- Docker
- [`kind`](https://kind.sigs.k8s.io/docs/user/quick-start/#installation)
- Linux kernel with `vrf` module and SRv6 support (or a VM with those features)

### Build

```bash
cd containers/kindest-node-galactic
docker build -t kindest/node:galactic .
```
- `kind` CLI
- Host kernel with SRv6 support

### Use with Kind

Reference the custom image in your Kind cluster config:

```yaml
# kind-config.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
image: kindest/node:galactic
- role: worker
image: kindest/node:galactic
- role: worker
image: kindest/node:galactic
```

```bash
kind create cluster --config kind-config.yaml
```

After the cluster is up, run `install.sh` on each node:
### Quick start

```bash
for node in $(kind get nodes); do
docker exec "$node" /galactic/scripts/install.sh
done
cd gvpc
make up # build Kind node image, apply host sysctls, deploy lab
make underlay # apply FRR DaemonSets to all three clusters
make overlay # apply GoBGP DaemonSets to iad and sjc clusters
make down # tear down
```

### Component versions (pinned in `scripts/install.sh`)

| Component | Version |
|--------------|----------|
| Cilium CLI | v0.18.8 |
| cert-manager | v1.19.1 |
| Multus CNI | v4.2.3 |
| CNI plugins | v1.8.0 |
| Galactic | v0.0.5 |
Loading