Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions config/crds/authzed.com_spicedbclusters.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,9 @@ spec:
baseImage:
description: |-
BaseImage specifies the base container image to use for SpiceDB.
This is useful for air-gapped environments or when using a private registry.
The operator will append the appropriate tag based on version/channel.
Must not include a tag or digest - use spec.version or spec.config.image instead.
If not specified, will fall back to the operator's --base-image flag,
then to the imageName defined in the update graph.
type: string
Expand Down Expand Up @@ -244,6 +247,12 @@ spec:
description: Phase is the currently running phase (used for phased
migrations)
type: string
resolvedBaseImage:
description: |-
ResolvedBaseImage is the base image that was resolved for this cluster.
This shows which registry/image the operator is using before appending
the version tag. Useful for debugging alternative registry configurations.
type: string
secretHash:
description: SecretHash is a digest of the last applied secret
type: string
Expand Down
128 changes: 128 additions & 0 deletions examples/alternative-registry/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
# Using Alternative Container Registry

This example demonstrates how to configure the SpiceDB operator to use an alternative container registry instead of the default one.

## Overview

The SpiceDB operator supports specifying a custom base image for SpiceDB containers through the `baseImage` field in the `SpiceDBCluster` spec. This is useful when:

- You need to use a private container registry
- You want to mirror images to your own registry for security or compliance reasons
- You need to use a registry proxy for better performance
- You're running in an air-gapped environment

## Configuration

The image selection follows this precedence order (highest to lowest):

1. `.spec.config.image` with explicit tag/digest (overrides everything)
2. `.spec.baseImage` field (what this example uses)
3. The operator's `--base-image` flag
4. The `imageName` defined in the update graph

**Important:** The `baseImage` field must NOT contain a tag (`:tag`) or digest (`@sha256:...`). The operator will automatically append the appropriate tag based on the `version` or `channel` you specify. If you need to specify an exact image with tag, use `.spec.config.image` instead.

## Example

See [spicedb-cluster.yaml](spicedb-cluster.yaml) for a complete example.

```yaml
apiVersion: authzed.com/v1alpha1
kind: SpiceDBCluster
metadata:
name: example-with-custom-registry
spec:
# Specify your alternative registry here (NO TAG!)
baseImage: "my-registry.company.com/authzed/spicedb"

# The operator will append the appropriate tag based on the version/channel
version: "v1.33.0"

config:
datastoreEngine: postgres
# ... other config

# If using a private registry, use patches to add imagePullSecrets
patches:
- kind: Deployment
patch: |
spec:
template:
spec:
imagePullSecrets:
- name: registry-credentials
```

## How it Works

When you specify a `baseImage`, the operator will:

1. Use your specified registry as the base
2. Append the appropriate tag or digest based on the `version` or `channel` you specify
3. The final image will be: `<baseImage>:<tag>` or `<baseImage>@<digest>`

For example, if you specify:

- `baseImage: "my-registry.company.com/authzed/spicedb"`
- `version: "v1.33.0"`

The operator will use: `my-registry.company.com/authzed/spicedb:v1.33.0`

## Private Registry Authentication

If your alternative registry requires authentication, you need to:

1. Create an image pull secret with your registry credentials:

```bash
kubectl create secret docker-registry registry-credentials \
--docker-server=my-registry.company.com \
--docker-username=YOUR-USERNAME \
--docker-password=YOUR-PASSWORD \
--namespace=spicedb-custom-registry
```

2. Use the `patches` field to inject the image pull secret into the deployment:

```yaml
spec:
patches:
- kind: Deployment
patch: |
spec:
template:
spec:
imagePullSecrets:
- name: registry-credentials
```

## Common Mistakes

### Including a tag in baseImage

**Wrong:**

```yaml
spec:
baseImage: "my-registry.company.com/authzed/spicedb:v1.33.0" # Don't include tag!
```

**Correct:**

```yaml
spec:
baseImage: "my-registry.company.com/authzed/spicedb"
version: "v1.33.0"
```

### Confusing baseImage with config.image

- Use `baseImage` when you want the operator to manage versions via the update graph
- Use `config.image` (with full tag/digest) when you want to bypass the update graph entirely

## Important Notes

- Make sure your Kubernetes nodes can pull from your alternative registry
- If using a private registry, use the `patches` field to configure image pull secrets
- The operator still uses the update graph to determine valid versions and migration paths
- The alternative registry must contain the exact same images as the official registry
58 changes: 58 additions & 0 deletions examples/alternative-registry/spicedb-cluster.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
apiVersion: v1
kind: Namespace
metadata:
name: spicedb-custom-registry
---
apiVersion: authzed.com/v1alpha1
kind: SpiceDBCluster
metadata:
name: example-with-custom-registry
namespace: spicedb-custom-registry
spec:
# Use an alternative container registry
# The operator will append the appropriate tag based on version/channel
# NOTE: Do NOT include a tag here - just the registry and image name
baseImage: "my-registry.company.com/authzed/spicedb"

# Specify the version to use
version: "v1.33.0"

# Alternatively, use a channel for automatic updates within that channel
# channel: "stable"

config:
datastoreEngine: postgres
logLevel: info

secretName: spicedb-config

# If using a private registry, use patches to add imagePullSecrets to the deployment
patches:
- kind: Deployment
patch: |
spec:
template:
spec:
imagePullSecrets:
- name: registry-credentials
---
apiVersion: v1
kind: Secret
metadata:
name: spicedb-config
namespace: spicedb-custom-registry
stringData:
datastore_uri: "postgresql://<CHANGE-ME-USERNAME>:<CHANGE-ME-PASSWORD>@<CHANGE-ME-POSTGRES-HOST>:5432/<CHANGE-ME-DATABASE>?sslmode=require"
preshared_key: "<CHANGE-ME-TO-A-VERY-SECRET-PRESHARED-KEY>"
---
# If using a private registry, you may need an image pull secret
apiVersion: v1
kind: Secret
metadata:
name: registry-credentials
namespace: spicedb-custom-registry
type: kubernetes.io/dockerconfigjson
data:
# CHANGE-ME: This is a placeholder - replace with your actual registry credentials
# Generate with: kubectl create secret docker-registry registry-credentials --docker-server=my-registry.company.com --docker-username=YOUR-USERNAME --docker-password=YOUR-PASSWORD --dry-run=client -o yaml
.dockerconfigjson: <CHANGE-ME-BASE64-ENCODED-DOCKER-CONFIG>
10 changes: 10 additions & 0 deletions pkg/apis/authzed/v1alpha1/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,9 @@ type ClusterSpec struct {
Patches []Patch `json:"patches,omitempty"`

// BaseImage specifies the base container image to use for SpiceDB.
// This is useful for air-gapped environments or when using a private registry.
// The operator will append the appropriate tag based on version/channel.
// Must not include a tag or digest - use spec.version or spec.config.image instead.
// If not specified, will fall back to the operator's --base-image flag,
// then to the imageName defined in the update graph.
// +optional
Expand Down Expand Up @@ -143,6 +146,12 @@ type ClusterStatus struct {
// Image is the image that is or will be used for this cluster
Image string `json:"image,omitempty"`

// ResolvedBaseImage is the base image that was resolved for this cluster.
// This shows which registry/image the operator is using before appending
// the version tag. Useful for debugging alternative registry configurations.
// +optional
ResolvedBaseImage string `json:"resolvedBaseImage,omitempty"`

// Migration is the name of the last migration applied
Migration string `json:"migration,omitempty"`

Expand All @@ -169,6 +178,7 @@ func (s ClusterStatus) Equals(other ClusterStatus) bool {
s.CurrentMigrationHash == other.TargetMigrationHash &&
s.SecretHash == other.SecretHash &&
s.Image == other.Image &&
s.ResolvedBaseImage == other.ResolvedBaseImage &&
s.Migration == other.Migration &&
s.Phase == other.Phase &&
s.CurrentVersion.Equals(other.CurrentVersion) &&
Expand Down
21 changes: 21 additions & 0 deletions pkg/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,7 @@ type MigrationConfig struct {
DatastoreURI string
SpannerCredsSecretRef string
TargetSpiceDBImage string
ResolvedBaseImage string
EnvPrefix string
SpiceDBCmd string
DatastoreTLSSecretName string
Expand Down Expand Up @@ -233,12 +234,32 @@ func NewConfig(cluster *v1alpha1.SpiceDBCluster, globalConfig *OperatorConfig, s
// unless the current config is equal to the input.
image := imageKey.pop(config)

// Validate that baseImage does not contain a tag or digest
if cluster.Spec.BaseImage != "" {
if strings.Contains(cluster.Spec.BaseImage, "@") {
errs = append(errs, fmt.Errorf("baseImage must not contain a digest (@sha256:...) - version is determined by the update graph"))
} else {
// Check for tag - a tag appears after the last colon, but only if that colon
// isn't part of a port number. Port numbers are followed by a slash (path),
// while tags are at the end of the string.
lastColon := strings.LastIndex(cluster.Spec.BaseImage, ":")
if lastColon != -1 {
afterColon := cluster.Spec.BaseImage[lastColon+1:]
// If there's no slash after the colon, it's a tag (not a port)
if !strings.Contains(afterColon, "/") {
errs = append(errs, fmt.Errorf("baseImage must not contain a tag (:tag) - version is determined by the update graph. Use spec.version or spec.config.image instead"))
}
}
}
}

baseImage, targetSpiceDBVersion, state, err := globalConfig.ComputeTarget(globalConfig.ImageName, cluster.Spec.BaseImage, image, cluster.Spec.Version, cluster.Spec.Channel, datastoreEngine, cluster.Status.CurrentVersion, cluster.RolloutInProgress())
if err != nil {
errs = append(errs, err)
}

migrationConfig.SpiceDBVersion = targetSpiceDBVersion
migrationConfig.ResolvedBaseImage = baseImage
migrationConfig.TargetPhase = state.Phase
migrationConfig.TargetMigration = state.Migration
if len(migrationConfig.TargetMigration) == 0 {
Expand Down
Loading
Loading