Skip to content

fix(cluster): fix alert templating and replica secret config #828

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
philippemnoel wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix(cluster): fix alert templating and replica secret config #828

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion charts/cluster/prometheus_rules/cluster-logical_replication_errors-critical.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ alert: {{ $alert }}
annotations:
summary: CNPG Cluster critical logical replication errors
description: |-
CloudNativePG Cluster's "{{ .namespace }}/{{ .cluster }}" "{{ "{{ .subname }}" }}" subscription has experienced {{ .value }} errors in the last 5 minutes.
CloudNativePG Cluster's "{{ .namespace }}/{{ .cluster }}" "{{ .labels.subname }}" subscription has experienced {{ .value }} errors in the last 5 minutes.

CRITICAL: High error rate indicates persistent replication issues requiring immediate attention. This could lead to significant data inconsistency or complete replication failure. Errors include both apply errors and sync errors. The subscription may stop working if errors continue.
runbook_url: https://github.com/cloudnative-pg/charts/blob/main/charts/cluster/docs/runbooks/CNPGClusterLogicalReplicationErrors.md
Expand Down
2 changes: 1 addition & 1 deletion charts/cluster/prometheus_rules/cluster-logical_replication_errors.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ alert: {{ $alert }}
annotations:
summary: CNPG Cluster logical replication errors detected
description: |-
CloudNativePG Cluster's "{{ .namespace }}/{{ .cluster }}" "{{ "{{ .subname }}" }}" subscription has experienced {{ .value }} errors.
CloudNativePG Cluster's "{{ .namespace }}/{{ .cluster }}" "{{ .labels.subname }}" subscription has experienced {{ .value }} errors.

This includes both apply errors (during normal replication) and sync errors (during initial table sync). Errors indicate data consistency issues that need immediate attention to prevent data divergence.
runbook_url: https://github.com/cloudnative-pg/charts/blob/main/charts/cluster/docs/runbooks/{{ $alert }}.md
Expand Down
2 changes: 1 addition & 1 deletion charts/cluster/prometheus_rules/cluster-logical_replication_lagging-critical.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ alert: {{ $alert }}
annotations:
summary: CNPG Cluster critical logical replication lag
description: |-
CloudNativePG Cluster's "{{ .namespace }}/{{ .cluster }}" "{{ "{{ .subname }}" }}" subscription is experiencing critical replication lag!
CloudNativePG Cluster's "{{ .namespace }}/{{ .cluster }}" "{{ .labels.subname }}" subscription is experiencing critical replication lag!

{{- if .labels.lag_type }}
Lag type: {{ .labels.lag_type }}
Expand Down
2 changes: 1 addition & 1 deletion charts/cluster/prometheus_rules/cluster-logical_replication_lagging.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ alert: {{ $alert }}
annotations:
summary: CNPG Cluster logical replication lagging
description: |-
CloudNativePG Cluster's "{{ .namespace }}/{{ .cluster }}" "{{ "{{ .subname }}" }}" subscription is experiencing replication lag.
CloudNativePG Cluster's "{{ .namespace }}/{{ .cluster }}" "{{ .labels.subname }}" subscription is experiencing replication lag.

{{- if .labels.lag_type }}
Lag type: {{ .labels.lag_type }}
Expand Down
2 changes: 1 addition & 1 deletion charts/cluster/prometheus_rules/cluster-logical_replication_stopped-critical.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ alert: {{ $alert }}
annotations:
summary: CNPG Cluster logical replication subscription CRITICAL
description: |-
CloudNativePG Cluster's "{{ .namespace }}/{{ .cluster }}" "{{ "{{ .subname }}" }}" subscription is in a critical state.
CloudNativePG Cluster's "{{ .namespace }}/{{ .cluster }}" "{{ .labels.subname }}" subscription is in a critical state.

CRITICAL: The subscription has been stopped for more than 15 minutes. This will lead to significant data divergence and requires immediate intervention.

Expand Down
2 changes: 1 addition & 1 deletion charts/cluster/prometheus_rules/cluster-logical_replication_stopped.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ alert: {{ $alert }}
annotations:
summary: CNPG Cluster logical replication subscription stopped
description: |-
CloudNativePG Cluster's "{{ .namespace }}/{{ .cluster }}" "{{ "{{ .subname }}" }}" subscription is stopped.
CloudNativePG Cluster's "{{ .namespace }}/{{ .cluster }}" "{{ .labels.subname }}" subscription is stopped.

Status: {{ .labels.stop_reason }}

Expand Down
2 changes: 1 addition & 1 deletion charts/cluster/templates/prometheus-rule.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ spec:
{{- $_ := set $dict "value" "{{ $value }}" -}}
{{- $_ := set $dict "namespace" .Release.Namespace -}}
{{- $_ := set $dict "cluster" (include "cluster.fullname" .) -}}
{{- $_ := set $dict "labels" (dict "job" "{{ $labels.job }}" "node" "{{ $labels.node }}" "pod" "{{ $labels.pod }}") -}}
{{- $_ := set $dict "labels" (dict "job" "{{ $labels.job }}" "node" "{{ $labels.node }}" "pod" "{{ $labels.pod }}" "subname" "{{ $labels.subname }}" "lag_type" "{{ $labels.lag_type }}" "stop_reason" "{{ $labels.stop_reason }}") -}}
{{- $_ := set $dict "podSelector" (printf "%s-([1-9][0-9]*)$" (include "cluster.fullname" .)) -}}
{{- $_ := set $dict "Values" .Values -}}
{{- $_ := set $dict "Template" .Template -}}
Expand Down
6 changes: 6 additions & 0 deletions charts/cluster/values.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -856,11 +856,17 @@
"passwordSecret": {
"type": "object",
"properties": {
"create": {
"type": "boolean"
},
"key": {
"type": "string"
},
"name": {
"type": "string"
},
"value": {
"type": "string"
}
}
},
Expand Down
4 changes: 4 additions & 0 deletions charts/cluster/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -568,8 +568,12 @@ replica:
name: ""
key: ""
passwordSecret:
# -- Whether to create a secret for the password
create: false
name: ""
key: ""
# -- The password value to use when creating the secret
value: ""
##
# Database management configuration
databases: []
Expand Down