Kuadrant E2E testsuite

This repository contains end-to-end (E2E) tests for the Kuadrant project, intended for contributors and maintainers to validate Kuadrant behavior in both single- and multi-cluster environments.

What’s tested:

• Core policies: AuthPolicy, RateLimitPolicy, TokenRateLimitPolicy, DNSPolicy, TLSPolicy
• Policy extensions: OIDCPolicy, PlanPolicy, TelemetryPolicy
• Policy behavior: defaults, overrides, reconciliation
• Observability: metrics and tracing
• Multi-cluster: load balancing, global rate limiting, CoreDNS delegation
• UI: Console Plugin

Prerequisites

Local Development

• Python 3.11+ and Poetry
• kubectl or oc (OpenShift CLI)
• CFSSL
• git
• Access to one or more Kubernetes clusters with Kuadrant already deployed

Once all prerequisites are installed, install dependencies and create a Python virtual environment by running:

make poetry

Container-based Testing

• Container runtime (podman or docker)
• Access to one or more Kubernetes clusters with Kuadrant already deployed

For Kuadrant installation instructions, see:

• Kuadrant Helm Charts for any Kubernetes cluster
• Deploying Kuadrant via OLM for OpenShift (recommended as it also deploys testing tools)

Local Kind Cluster Setup

For local development and testing, you can set up a complete Kuadrant environment using Kind (Kubernetes in Docker).

Prerequisites

• Kind
• Helm
• jq (JSON processor)

⚠️ macOS Limitation: MetalLB LoadBalancer services have limited functionality on macOS due to Docker Desktop's VM isolation. While MetalLB will work inside the cluster, LoadBalancer IPs won't be accessible from your Mac host machine. For macOS users, we recommend running tests in containers (see From a Container section above) in addition to the local Kind setup.

Quick Start

Set up a complete local environment with one command:

# Run the setup (defaults to Istio gateway)
make local-setup

# Optional: Apply additional manifests (e.g., DNS provider credentials, secrets, etc.)
ADDITIONAL_MANIFESTS=./my-secrets.yaml make local-setup

# Optional: Install Prometheus CRDs for observability testing
INSTALL_PROMETHEUS=true make local-setup

# Or specify EnvoyGateway
GATEWAYAPI_PROVIDER=envoygateway make local-setup

# Combine options
GATEWAYAPI_PROVIDER=envoygateway INSTALL_PROMETHEUS=true ADDITIONAL_MANIFESTS=./secrets.yaml make local-setup

This will:

1. Create a Kind cluster named kuadrant-local
2. Install metrics-server and MetalLB (LoadBalancer support)
3. Install Gateway API CRDs
4. Install cert-manager and create a self-signed ClusterIssuer
5. Install Prometheus CRDs (only if INSTALL_PROMETHEUS=true) - ServiceMonitor, PodMonitor, etc.
6. Install Istio or EnvoyGateway (based on GATEWAYAPI_PROVIDER)
7. Create test namespaces (kuadrant, kuadrant2)
8. Apply additional manifests (only if ADDITIONAL_MANIFESTS is provided)
9. Deploy Kuadrant Operator and Kuadrant CR
10. Deploy testing tools - Keycloak, Mockserver, etc.

Cleanup:

make local-cleanup  # Delete the Kind cluster

Configuration

The Kuadrant testsuite uses Dynaconf for configuration.

Settings Files

For local development, create a YAML configuration file at config/settings.local.yaml. See config/settings.local.yaml.tpl for all available configuration options.

Environment Variables

Settings can also be configured using environment variables. All variables use the KUADRANT prefix, for example:

export KUADRANT_KEYCLOAK__url="https://my-sso.net"

For more details, see the Dynaconf wiki page.

Kubernetes Auto-Fetching

Some configuration options can be fetched from Kubernetes. To install helper services (e.g., Keycloak, Jaeger, MockServer, Redis), see Testing charts:

# Install tools operators
helm install --values values-tools.yaml --wait -g charts/tools-operators
# Install tools instances
helm install --values values-tools.yaml --wait --timeout 10m -g charts/tools-instances

Test Requirements

Single-cluster tests

Test Type	Requirements	Make Target
Kuadrant	Kuadrant deployment* Gateway API* cert-manager DNS Secret* TLS ClusterIssuer*	make test or make kuadrant
Authorino standalone	Authorino Operator	make authorino-standalone
DNS & TLS Policies	Kuadrant deployment* Gateway API* DNS Secret* cert-manager TLS ClusterIssuer*	make dnstls
Console Plugin	OpenShift Kuadrant Console Plugin enabled HTPasswd authentication	make ui

Important Notes:

• Kuadrant deployment* represents multiple operators: Kuadrant Operator, Authorino Operator, Limitador, and DNS Operator.
• Gateway API* requires an implementation (e.g., Istio, Envoy Gateway). On OpenShift, this is typically provided by Service Mesh.
• DNS Secret* needs base_domain annotation and type kuadrant.io/aws|gcp|azure (see example below).
• TLS ClusterIssuer* can be a self-signed CA from helm-charts-olm or Let's Encrypt (letsencrypt-staging-issuer).
• Keycloak can be auto-fetched if deployed via helm or configured manually. Required for most AuthPolicy tests.

DNS Provider Secret example (click to expand)

Save this as a file (e.g., additionalManifests.yaml) and provide it via ADDITIONAL_MANIFESTS:

ADDITIONAL_MANIFESTS=./additionalManifests.yaml make local-setup

apiVersion: v1
kind: Secret
metadata:
  name: aws-credentials
  namespace: kuadrant
  annotations:
    base_domain: example.com
stringData:
  AWS_ACCESS_KEY_ID: <your-key>
  AWS_REGION: <your-region>
  AWS_SECRET_ACCESS_KEY: <your-secret>
type: kuadrant.io/aws
---
# You can include multiple resources in the same file
# For example, GCP credentials:
apiVersion: v1
kind: Secret
metadata:
  name: gcp-credentials
  namespace: kuadrant
  annotations:
    base_domain: example.com
stringData:
  PROJECT_ID: <your-project-id>
  GOOGLE: <base64-encoded-service-account-json>
type: kuadrant.io/gcp

Multi-cluster tests

Base requirements: 2+ clusters (cluster2 required, cluster3 optional), matching namespaces on all clusters, and DNS Secret + TLS ClusterIssuer on all clusters.

Test Type	Additional Requirements	Make Target
Load balancing	DNS servers with geo-codes	make multicluster
CoreDNS delegation	CoreDNS zone + CoreDNS tools deployed on all clusters	make coredns_one_primary or make coredns_two_primaries
Global rate limiting	Shared storage (Redis/Dragonfly/Valkey)	make multicluster

Running the Tests

Locally

For development and debugging, running the tests locally is recommended.

Test commands:

make smoke          # Quick smoke test to verify environment setup
make test           # Run the full test suite
make <test-path>    # Run a specific test file or directory
# or
poetry run pytest -v <test-path>

Run make help to list all available targets. Most make targets run in parallel by default.

You can also pass pytest flags to make targets using the flags environment variable. Note: The flags variable must be placed before the make command (see pytest command-line flags for more options):

flags=--lf make test      # Run last failed tests
flags=-n1 make test       # Run tests with just one thread
flags=-v make test        # Run in verbose mode
flags="-v --lf" make test # Multiple flags (use quotes)

From a Container

To simply run tests, using the container image is the easiest option. Run it with your kubeconfig mounted (it must be readable by the container). If you omit any variables (for example, Auth0 credentials), the corresponding tests will be skipped. Mount a local directory to /test-run-results to persist test results.

E2E tests - quay.io/kuadrant/testsuite:latest

With tools setup:

podman run \
  -v $HOME/.kube/config:/run/kubeconfig:z \
  -v $(pwd)/test-run-results:/test-run-results:z \
  -e KUADRANT_SERVICE_PROTECTION__PROJECT=authorino \
  -e KUADRANT_SERVICE_PROTECTION__PROJECT2=authorino2 \
  -e KUADRANT_AUTH0__url="AUTH0_URL" \
  -e KUADRANT_AUTH0__client_id="AUTH0_CLIENT_ID" \
  -e KUADRANT_AUTH0__client_secret="AUTH0_CLIENT_SECRET" \
  quay.io/kuadrant/testsuite:latest

Without tools (manual Keycloak config):

podman run \
  -v $HOME/.kube/config:/run/kubeconfig:z \
  -v $(pwd)/test-run-results:/test-run-results:z \
  -e KUADRANT_SERVICE_PROTECTION__PROJECT=authorino \
  -e KUADRANT_SERVICE_PROTECTION__PROJECT2=authorino2 \
  -e KUADRANT_KEYCLOAK__url="https://my-sso.net" \
  -e KUADRANT_KEYCLOAK__password="ADMIN_PASSWORD" \
  -e KUADRANT_KEYCLOAK__username="ADMIN_USERNAME" \
  -e KUADRANT_AUTH0__url="AUTH0_URL" \
  -e KUADRANT_AUTH0__client_id="AUTH0_CLIENT_ID" \
  -e KUADRANT_AUTH0__client_secret="AUTH0_CLIENT_SECRET" \
  quay.io/kuadrant/testsuite:latest

UI tests - quay.io/kuadrant/testsuite-ui:unstable

The UI container expects a settings file to be mounted, containing the console username and password used to authenticate against the OpenShift console.

podman run --rm \
  -v $HOME/.kube/config:/run/kubeconfig:z \
  -v $(pwd)/test-run-results:/test-run-results:z \
  -v $(pwd)/settings.local.yaml:/run/secrets.yaml:Z \
  quay.io/kuadrant/testsuite-ui:unstable

Developing Authorino Tests

When developing Authorino tests, you may need to inspect the full authorization JSON returned by Authorino.

AuthConfig example for returning full authorization context

apiVersion: authorino.kuadrant.io/v1beta3
kind: AuthConfig
metadata:
  name: example
spec:
  hosts:
    - '*'
  response:
    success:
      headers:
        auth-json:
          json:
            properties:
              auth:
                selector: auth
              context:
                selector: context

Another useful tool is the OPA Playground for developing and validating OPA policies.

Contributing

See the Kuadrant Testsuite Contribution Guide for information on how to contribute to the Kuadrant testsuite.