[Bug] Invalid Memory in antivirus container - nil pointer dereference

[suse-coder]

Describe the bug

I have the latest opencloud RC 4.0.0 running (in the older one it was working) and when I deploy it I get the error:

antivirus container:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x557fcd388ead]
goroutine 1 [running]:
github.com/opencloud-eu/opencloud/services/antivirus/pkg/command.Server.func2(0xc0008c0380)
github.com/opencloud-eu/opencloud/services/antivirus/pkg/command/server.go:45 +0x26d
github.com/urfave/cli/v2.(*Command).Run(0xc0012ad1e0, 0xc0008c0380, {0xc0008f4320, 0x1, 0x1})
github.com/urfave/cli/v2@v2.27.7/command.go:276 +0x7c2
github.com/urfave/cli/v2.(*Command).Run(0xc0012ad600, 0xc0008c02c0, {0xc0008b2680, 0x2, 0x2})
github.com/urfave/cli/v2@v2.27.7/command.go:269 +0xa30
github.com/urfave/cli/v2.(*Command).Run(0xc0008d6160, 0xc0008c0200, {0xc0000529f0, 0x3, 0x3})
github.com/urfave/cli/v2@v2.27.7/command.go:269 +0xa30
github.com/urfave/cli/v2.(*App).RunContext(0xc0012a2200, {0x557fd083c9a8, 0xc000de6500}, {0xc0000529f0, 0x3, 0x3})
github.com/urfave/cli/v2@v2.27.7/app.go:333 +0x5a5
github.com/opencloud-eu/opencloud/opencloud/pkg/command.Execute()
github.com/opencloud-eu/opencloud/opencloud/pkg/command/root.go:32 +0x1e5
main.main()
github.com/opencloud-eu/opencloud/opencloud/cmd/opencloud/main.go:11 +0x13

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2025-12-09T10:55:47Z"
  generateName: antivirus-5ffb79c75c-
  labels:
    app: antivirus
    app.kubernetes.io/instance: opencloud-gwis
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: opencloud-microservices
    app.kubernetes.io/version: 4.0.0
    helm.sh/chart: opencloud-microservices-0.3.12_72bde002f4ee
    pod-template-hash: 5ffb79c75c
  name: antivirus-5ffb79c75c-dbdfr
  namespace: opencloud-gwis
  ownerReferences:
  - apiVersion: apps/v1
    blockOwnerDeletion: true
    controller: true
    kind: ReplicaSet
    name: antivirus-5ffb79c75c
    uid: 50e8cd3e-1b88-4771-af84-17649867a53e
  resourceVersion: "144034542"
  uid: 3cb04a0f-10cd-48ef-9e2f-d47ad03897c4
spec:
  containers:
  - args:
    - antivirus
    - server
    command:
    - opencloud
    env:
    - name: MICRO_REGISTRY
      value: nats-js-kv
    - name: MICRO_REGISTRY_ADDRESS
      value: nats:9233
    - name: OC_EVENTS_ENDPOINT
      value: nats:9233
    - name: ANTIVIRUS_LOG_COLOR
      value: "false"
    - name: ANTIVIRUS_LOG_LEVEL
      value: debug
    - name: ANTIVIRUS_LOG_PRETTY
      value: "false"
    - name: ANTIVIRUS_DEBUG_ADDR
      value: 0.0.0.0:9277
    - name: ANTIVIRUS_DEBUG_PPROF
      value: "false"
    - name: ANTIVIRUS_INFECTED_FILE_HANDLING
      value: abort
    - name: ANTIVIRUS_SCANNER_TYPE
      value: clamav
    - name: ANTIVIRUS_CLAMAV_SOCKET
      value: tcp://clamav.clamav.svc.cluster.local:3310
    - name: ANTIVIRUS_ICAP_SCAN_TIMEOUT
      value: "300"
    - name: ANTIVIRUS_ICAP_URL
      value: http://clamav-icap.clamav:1344
    - name: ANTIVIRUS_ICAP_SERVICE
      value: avscan
    - name: ANTIVIRUS_MAX_SCAN_SIZE
    - name: ANTIVIRUS_WORKERS
      value: "10"
    image: opencloudeu/opencloud-rolling:4.0.0
    imagePullPolicy: IfNotPresent
    livenessProbe:
      failureThreshold: 3
      httpGet:
        path: /healthz
        port: metrics-debug
        scheme: HTTP
      initialDelaySeconds: 60
      periodSeconds: 20
      successThreshold: 1
      timeoutSeconds: 10
    name: antivirus
    ports:
    - containerPort: 9277
      name: metrics-debug
      protocol: TCP
    resources: {}
    securityContext:
      readOnlyRootFilesystem: true
      runAsGroup: 1000
      runAsNonRoot: true
      runAsUser: 1000
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /etc/opencloud/messaging-system-ca
      name: messaging-system-ca
      readOnly: true
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: kube-api-access-5q64r
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  nodeName: talos-worker-0
  preemptionPolicy: PreemptLowerPriority
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext:
    fsGroup: 1000
    fsGroupChangePolicy: OnRootMismatch
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  volumes:
  - emptyDir: {}
    name: messaging-system-ca
  - name: kube-api-access-5q64r
    projected:
      defaultMode: 420
      sources:
      - serviceAccountToken:
          expirationSeconds: 3607
          path: token
      - configMap:
          items:
          - key: ca.crt
            path: ca.crt
          name: kube-root-ca.crt
      - downwardAPI:
          items:
          - fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
            path: namespace
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2025-12-09T10:55:49Z"
    status: "True"
    type: PodReadyToStartContainers
  - lastProbeTime: null
    lastTransitionTime: "2025-12-09T10:55:47Z"
    status: "True"
    type: Initialized
  - lastProbeTime: null
    lastTransitionTime: "2025-12-09T10:55:47Z"
    message: 'containers with unready status: [antivirus]'
    reason: ContainersNotReady
    status: "False"
    type: Ready
  - lastProbeTime: null
    lastTransitionTime: "2025-12-09T10:55:47Z"
    message: 'containers with unready status: [antivirus]'
    reason: ContainersNotReady
    status: "False"
    type: ContainersReady
  - lastProbeTime: null
    lastTransitionTime: "2025-12-09T10:55:47Z"
    status: "True"
    type: PodScheduled
  containerStatuses:
  - containerID: containerd://f5e4130885db5deb7427f0d38bfbd372d036f75457bde6d67f21b575abc9e1bd
    image: docker.io/opencloudeu/opencloud-rolling:4.0.0
    imageID: docker.io/opencloudeu/opencloud-rolling@sha256:1cbce735bfdde97d38422a52b92104dfd2fd320e2e3febc05314b00d6d115d7e
    lastState:
      terminated:
        containerID: containerd://f5e4130885db5deb7427f0d38bfbd372d036f75457bde6d67f21b575abc9e1bd
        exitCode: 2
        finishedAt: "2025-12-09T10:56:32Z"
        reason: Error
        startedAt: "2025-12-09T10:56:32Z"
    name: antivirus
    ready: false
    restartCount: 3
    started: false
    state:
      waiting:
        message: back-off 40s restarting failed container=antivirus pod=antivirus-5ffb79c75c-dbdfr_opencloud-gwis(3cb04a0f-10cd-48ef-9e2f-d47ad03897c4)
        reason: CrashLoopBackOff
    volumeMounts:
    - mountPath: /etc/opencloud/messaging-system-ca
      name: messaging-system-ca
      readOnly: true
      recursiveReadOnly: Disabled
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: kube-api-access-5q64r
      readOnly: true
      recursiveReadOnly: Disabled
  hostIP: 192.168.0.235
  hostIPs:
  - ip: 192.168.0.235
  phase: Running
  podIP: 10.244.4.73
  podIPs:
  - ip: 10.244.4.73
  qosClass: BestEffort
  startTime: "2025-12-09T10:55:47Z"

[butonic]

Hm this is

- traceProvider, err := tracing.GetServiceTraceProvider(cfg.Tracing, cfg.Service.Name)
+ traceProvider, err := tracing.GetTraceProvider(c.Context, cfg.Commons.TracesExporter, cfg.Service.Name)

We introduced the cfg.Commons.TracesExporter because we only want to have one env var for the tracer config.

👀

It seems we broke the config parsing when updating the tracer with #1918.

In compose and our helm chart we always run opencloud server and use OC_RUN_SERVICES to start a list of services.

@suse-coder you can work around this by setting OC_RUN_SERVICES=antivirus and using the opencloud server command in your helm chart.

This might be an easy fix ... but I'm not sure we want to support starting individual services via sub commands in the future. We are reworking the env parsing anyway #1954 and are planning to reduce the number of services anyway. See #1958

[suse-coder]

Thanks will do that (so that will also be available in the future?):

spec:
  containers:
  - args:
    - server                        
    command:
    - opencloud
    env:
    - name: OC_RUN_SERVICES         
      value: antivirus

[fschade]

tracked here: #2009

[butonic]

@suse-coder Well, the opencloud subcommands will likely be available for some time. I would like to merge all the request handlers int a single api/frontend service as well as all event handlers into a backend service. In addition a dedicated storage service that can be reused to start either a posix driver or a decomposedfs ... and then register that in the storage registry makes sense when sharding an instance becomes necessary. Finally, a proxy service that can be used to also shard users and complete instances.

But this can be done over time. The compose file uses opencloud server + OC_RUN_SERVICES to determine which services to start. That will be the mechanism we use as well. The env vars to configure the individual services / handlers are prefixed so there should not be any collisions.

I recommend using a single pod with all services as started by the opencloud-compose deployment example. The ocis like deployment that creates one pod per service is not going to be a deployment goal. It wastes too much resources and does not offer any scaling benefits. On the contrary: all the service hops make scale up an down quite brittle.