Skip to content

Missing workqueue metrics #1026

@piepmatz

Description

@piepmatz

In comparison with other operators, the metrics endpoint doesn't show any low-level workqueue metrics, such as workqueue_depth or workqueue_queue_duration_seconds_bucket. Kubebuilder mentions both in the list of default exported metrics.

The metrics server as well as the used registry is provided by controller-runtime, currently in version v0.22.4.

While those metrics originate from k8s.io/component-base/metrics/prometheus/workqueue, controller-runtime maintains their own copy: https://github.com/kubernetes-sigs/controller-runtime/blob/v0.22.4/pkg/internal/metrics/workqueue.go#L100-L110.
When starting Kamaji with a debugger, I see that code running.

That registry is defined at https://github.com/kubernetes-sigs/controller-runtime/blob/v0.22.4/pkg/metrics/registry.go#L30.

Custom Kamaji metrics use the same registry: https://github.com/clastix/kamaji/blob/edge-25.11.5/internal/resources/metrics.go#L46

When querying the metrics endpoint, I see custom Kamaji metrics, controller-runtime metrics etc., but no workqueue-related ones. I don't know why.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions