feat: migrate from Ingress to Gateway API

.github/workflows/e2e-tests.yaml

@@ -44,6 +44,8 @@ jobs:
 
       - name: "Setup Kubectl"
         uses: azure/setup-kubectl@v4
+      - name: "Set up Helm"
+        uses: azure/setup-helm@v4
 
       - name: "Build runtime images"
         run: |
@@ -53,7 +55,6 @@ jobs:
       - name: "Create k8s Kind Cluster"
         uses: helm/kind-action@v1.13.0
         with:
-          config: kind.config.yaml
           cluster_name: jad
 
       - name: "Load runtime images into KinD"
@@ -65,13 +66,20 @@ jobs:
                                 ghcr.io/metaform/jad/issuerservice:latest \
 
 
-      - name: "Install nginx ingress controller"
+      - name: "Install Traefik Gateway controller"
         run: |-
-          kubectl apply -f https://kind.sigs.k8s.io/examples/ingress/deploy-ingress-nginx.yaml
-          kubectl wait --namespace ingress-nginx \
-            --for=condition=ready pod \
-            --selector=app.kubernetes.io/component=controller \
-            --timeout=120s
+          helm repo add traefik https://traefik.github.io/charts
+          helm repo update
+          helm upgrade --install --namespace traefik traefik traefik/traefik --create-namespace -f values.yaml
+
+          # Wait for traefik to be ready
+          kubectl rollout status deployment/traefik -n traefik --timeout=120s
+
+          # forward port 80 -> 8080
+          kubectl -n traefik port-forward svc/traefik 8080:80 &
+
+          # install Gateway API CRDs
+          kubectl apply --server-side --force-conflicts -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.1/experimental-install.yaml
 
           sleep 5 # to be safe
 
@@ -110,12 +118,6 @@ jobs:
         run: |
           ./gradlew test -DincludeTags="EndToEndTest"
 
-      - name: "Print log if test failed"
-        if: failure()
-        run: |-
-          kubectl logs deployment/controlplane -n edc-v
-          kubectl logs deployment/dataplane -n edc-v
-
       - name: "Destroy the KinD cluster"
         run: >-
           kind delete cluster -n jad

README.md

@@ -25,6 +25,7 @@ Such a dataspace requires – at a minimum – the following components:
 - Java 17+
 - Docker
 - `kubectl`
+- Helm (for Traefik installation)
 - macOS or Linux as an operating system. **Windows is not natively supported**!
 - a POSIX-compliant shell (e.g., bash, zsh)
 - [Bruno](https://www.usebruno.com) (or similar). The API requests here are optimized for Bruno, but other tools work as
@@ -41,20 +42,88 @@ To create a KinD cluster, run:
 
 ```shell
 cp ~/.kube/config ~/.kube/config.bak # to save your existing kubeconfig
-kind create cluster -n edcv --config kind.config.yaml --kubeconfig ~/.kube/edcv-kind.conf
+kind create cluster -n edcv --kubeconfig ~/.kube/edcv-kind.conf
 ln -sf ~/.kube/edcv-kind.conf ~/.kube/config # to use KinD's kubeconfig
 ```
 
-Next, deploy the NGINX ingress controller:
+Next, we need to deploy a Gateway controller to allow access to services from outside the cluster. There are several
+popular choices, and we've opted for [Traefik](https://doc.traefik.io/traefik/setup/kubernetes/) in this case. We could
+have gone for Envoy, but Traefik is easier to set up and lighter-weight.
 
 ```shell
-kubectl apply -f https://kind.sigs.k8s.io/examples/ingress/deploy-ingress-nginx.yaml
-kubectl wait --namespace ingress-nginx \
-  --for=condition=ready pod \
-  --selector=app.kubernetes.io/component=controller \
-  --timeout=90s
+helm repo add traefik https://traefik.github.io/charts
+helm repo update
+helm upgrade --install --namespace traefik traefik traefik/traefik --create-namespace -f values.yaml
 ```
 
+Then, install the custom resource definitions (CRDs) for the Gateway API:
+
+```shell
+kubectl apply --server-side --force-conflicts -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.1/experimental-install.yaml
+```
+
+#### Enable network access to services
+
+With the Gateway API, there are three main ways to access services from outside the cluster:
+
+- using port-forwarding: this is a manual way to forward ports from the host to the cluster. This is not
+  recommended for production use but works fine for local testing. We will use this approach here for simplicity.
+
+- via a LoadBalancer service: this is typically used in cloud-hosted Kubernetes clusters, where the cloud provider
+  provisions a load balancer automatically. This is the recommended approach for production use when running in
+  cloud-hosted environments, and DNS names are used. For KinD, there is no built-in load balancer, but one can be
+  installed
+
+
+- via `NodePort` services: this exposes services on high-numbered ports on the host machine. This is not
+  recommended for production use, and it gets complicated quickly when multiple services are involved, as is the case
+  here. _This is not shown here_.
+
+##### Option 1 (recommended): via port-forwarding
+
+To set up port-forwarding, run the following command:
+
+```shell
+kubectl -n traefik port-forward svc/traefik 80
+```
+
+This forwards port 80 from the host to the Traefik service inside the cluster. You may need to run this with `sudo`
+privileges on some systems.
+
+##### Option 2: via LoadBalancer (alternative)
+
+The KinD project provides
+a [cloud-like load balancer implementation](https://github.com/kubernetes-sigs/cloud-provider-kind). It emulates an
+external LB as you would get on cloud-hosted K8s clusters.
+Install it according to the instructions in the repository.
+
+Verify, that the `EXTERNAL-IP` of the `traefik` service is not yet assigned:
+
+```shell
+kubectl get svc -n traefik
+NAME      TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
+traefik   LoadBalancer   10.96.251.221   <pending>     80:31415/TCP,443:31650/TCP   22s
+```
+
+To assign an IP address to the Traefik LoadBalancer service, we need to run the external LB:
+
+```shell
+cloud-provider-kind 
+# on macOS:
+sudo cloud-provider-kind
+```
+
+Now, if we rerun the previous command, we should see a similar output to this:
+
+```shell
+kubectl get svc -n traefik
+NAME      TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
+traefik   LoadBalancer   10.96.251.221   172.18.0.3    80:31415/TCP,443:31650/TCP   2m31s
+```
+
+> Note that with the external LB, services inside the cluster must be accessed via the external IP address, e.g.
+`172.18.0.3` in this case. Variables inside the Bruno collection must be adjusted accordingly!
+
 ### 2. Deploy applications
 
 #### 2.1 Option 1: Use pre-built images
@@ -435,3 +504,4 @@ livenessProbe:
   successThreshold: 1
   failureThreshold: 15 # changed
 ```
+

k8s/apps/controlplane.yaml

@@ -75,7 +75,7 @@ metadata:
   name: controlplane
   namespace: edc-v
 spec:
-  type: NodePort
+  type: ClusterIP
   selector:
     app: controlplane
   ports:
@@ -93,23 +93,24 @@ spec:
       targetPort: 1044
 
 ---
-apiVersion: networking.k8s.io/v1
-kind: Ingress
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
 metadata:
   name: controlplane
   namespace: edc-v
-  annotations:
-    nginx.ingress.kubernetes.io/rewrite-target: "/$2"
-    nginx.ingress.kubernetes.io/use-regex: "true"
 spec:
-  ingressClassName: nginx
+  parentRefs:
+    - name: edcv-gateway
+      kind: Gateway
+      sectionName: http
+  hostnames:
+    - cp.localhost
   rules:
-    - http:
-        paths:
-          - path: /cp(/|$)(.*)
-            pathType: ImplementationSpecific
-            backend:
-              service:
-                name: controlplane
-                port:
-                  number: 8081
+    - matches:
+        - path:
+            type: PathPrefix
+            value: /
+      backendRefs:
+        - name: controlplane
+          port: 8081
+          weight: 1

k8s/apps/dataplane.yaml

@@ -79,7 +79,7 @@ metadata:
   name: dataplane
   namespace: edc-v
 spec:
-  type: NodePort
+  type: ClusterIP
   selector:
     app: dataplane
   ports:
@@ -97,66 +97,62 @@ spec:
       targetPort: 11002
 
 ---
-apiVersion: networking.k8s.io/v1
-kind: Ingress
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
 metadata:
-  name: ingress-dataplane
+  name: httproute-dataplane
   namespace: edc-v
-  annotations:
-    nginx.ingress.kubernetes.io/rewrite-target: "/$2"
-    nginx.ingress.kubernetes.io/use-regex: "true"
 spec:
-  ingressClassName: nginx
+  parentRefs:
+    - name: edcv-gateway
+      namespace: edc-v
+  hostnames:
+    - dp.localhost
   rules:
-    - http:
-        paths:
-          - path: /dp/public(/|$)(.*)
-            pathType: ImplementationSpecific
-            backend:
-              service:
-                name: dataplane
-                port:
-                  number: 11002
----
-apiVersion: networking.k8s.io/v1
-kind: Ingress
-metadata:
-  name: ingress-dataplane-control
-  namespace: edc-v
-  annotations:
-    nginx.ingress.kubernetes.io/rewrite-target: "/$2"
-    nginx.ingress.kubernetes.io/use-regex: "true"
-spec:
-  ingressClassName: nginx
-  rules:
-    - http:
-        paths:
-          - path: /app/internal(/|$)(.*)
-            pathType: ImplementationSpecific
-            backend:
-              service:
-                name: dataplane
-                port:
-                  number: 8083
+    # /dp/public → public port 11002
+    - matches:
+        - path:
+            type: PathPrefix
+            value: /public
+      filters:
+        - type: URLRewrite
+          urlRewrite:
+            path:
+              type: ReplacePrefixMatch
+              replacePrefixMatch: /
+      backendRefs:
+        - name: dataplane
+          port: 11002
+          weight: 1
 
----
-apiVersion: networking.k8s.io/v1
-kind: Ingress
-metadata:
-  name: ingress-dataplane-certs-control
-  namespace: edc-v
-  annotations:
-    nginx.ingress.kubernetes.io/rewrite-target: "/$2"
-    nginx.ingress.kubernetes.io/use-regex: "true"
-spec:
-  ingressClassName: nginx
-  rules:
-    - http:
-        paths:
-          - path: /app/public(/|$)(.*)
-            pathType: ImplementationSpecific
-            backend:
-              service:
-                name: dataplane
-                port:
-                  number: 8186
+    # /app/internal → control port 8083
+    - matches:
+        - path:
+            type: PathPrefix
+            value: /app/internal
+      filters:
+        - type: URLRewrite
+          urlRewrite:
+            path:
+              type: ReplacePrefixMatch
+              replacePrefixMatch: /
+      backendRefs:
+        - name: dataplane
+          port: 8083
+          weight: 1
+
+    # /app/public → certs port 8186
+    - matches:
+        - path:
+            type: PathPrefix
+            value: /app/public
+      filters:
+        - type: URLRewrite
+          urlRewrite:
+            path:
+              type: ReplacePrefixMatch
+              replacePrefixMatch: /
+      backendRefs:
+        - name: dataplane
+          port: 8186
+          weight: 1