Skip to content

[monitoring-template] feat(cloudspanner): add Google Cloud Spanner monitoring#4147

Open
neon-hippo wants to merge 1 commit into
apache:masterfrom
neon-hippo:add-cloudspanner-monitoring
Open

[monitoring-template] feat(cloudspanner): add Google Cloud Spanner monitoring#4147
neon-hippo wants to merge 1 commit into
apache:masterfrom
neon-hippo:add-cloudspanner-monitoring

Conversation

@neon-hippo
Copy link
Copy Markdown
Contributor

@neon-hippo neon-hippo commented May 27, 2026

Adds a new monitor type for Google Cloud Spanner, supporting both the Google Standard SQL dialect (app-cloudspanner.yml) and the PostgreSQL dialect (app-cloudspanner-pg.yml). HertzBeat connects via the open-source Cloud Spanner JDBC driver; no GCP credentials are baked in — users supply a service account key path via the Extra Params field.

The connection URL is assembled automatically from the three natural GCP identifiers:

jdbc:cloudspanner:/projects/{project}/instances/{instance}/databases/{database}

What is included

  • app-cloudspanner.yml — Google Standard SQL dialect monitor (i18n: en/zh/ja)
  • app-cloudspanner-pg.yml — PostgreSQL dialect monitor (i18n: en/zh/ja)
  • hertzbeat-collector/hertzbeat-collector-cloudspanner — new collector module that loads the Spanner JDBC driver, which is not discovered by the standard JDBC classpath scan
  • pom.xml — bumps google-cloud-spanner-jdbc 2.30.0 → 2.34.0 and pins grpc-netty-shaded:1.70.0 to resolve a runtime conflict with the GreptimeDB ingester (see Design Decisions)
  • CloudSpannerDriverLoaderTest and SpannerJdbcTemplateIntegrationTest (27 cases)

Metric groups (both dialects)

Group Source Description
tables INFORMATION_SCHEMA.TABLES Table names, types, and Spanner state
indexes INFORMATION_SCHEMA.INDEXES Index metadata including uniqueness and null-filter flags
database_options INFORMATION_SCHEMA.DATABASE_OPTIONS Database-level configuration key/value pairs
query_stats SPANNER_SYS.QUERY_STATS_TOTAL_MINUTE Last-minute aggregate: execution count, avg latency, CPU, failures
txn_stats SPANNER_SYS.TXN_STATS_TOTAL_MINUTE Last-minute aggregate: commits, aborts, retries, latency
lock_stats SPANNER_SYS.LOCK_STATS_TOTAL_MINUTE Last-minute total lock-wait seconds
table_sizes SPANNER_SYS.TABLE_SIZES_STATS_1HOUR Per-table storage: total, SSD, and HDD bytes

Design decisions

  • host field reused for Instance ID. HertzBeat's param schema requires one field of type host for validation. The Cloud Spanner JDBC URL doesn't have a traditional hostname, so Instance ID is mapped to this field. The UI label reads "Instance ID" — users never see "host."
  • grpc-netty-shaded version conflict. The GreptimeDB ingester (ingester-all) declares a transitive dependency on grpc-netty-shaded:1.56.1, which conflicts with the version required by the Spanner JDBC driver and causes a NoSuchMethodError at startup. The fix pins grpc-netty-shaded:1.70.0 in root dependencyManagement and excludes the old transitive pull from hertzbeat-warehouse. This has no impact on GreptimeDB functionality or the rest of the project — it simply prevents the older version from winning the dependency resolution.

Permissions required

The GCP service account used by HertzBeat needs read access to the target database. The minimum IAM role is:

roles/spanner.databaseReader

This grants SELECT on all tables and access to INFORMATION_SCHEMA and SPANNER_SYS views. It can be granted at the database, instance, or project level via the GCP Console or gcloud:

gcloud spanner databases add-iam-policy-binding DATABASE   --instance=INSTANCE   --project=PROJECT   --member=serviceAccount:SA_EMAIL   --role=roles/spanner.databaseReader

Live verification

Both templates were tested against a real GCP project (hertzbeat-spanner-dev, instance hertzbeat-dev). All 7 metric groups collected successfully on both the GSS and PG databases.

Dialect Emulator Live GCP
Google Standard SQL ✅ (INFORMATION_SCHEMA groups) ✅ (all 7 groups)
PostgreSQL ✅ (INFORMATION_SCHEMA groups) ✅ (all 7 groups)

Known limitations

  • SPANNER_SYS tables (query/txn/lock/table-size stats) are not available on the Cloud Spanner emulator. Those metric groups return no data in local dev environments.
  • HertzBeat's per-poll row limit applies as with all multi-row JDBC monitors.

References

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant