we just updated our hdx deployment from 2.24 to 2.27, and now the otel collector doesn't start, with these errors:
otel-collector-1 | 12:16:14 [seed] RETRY: Seed failed, retrying in 1s... (attempt 1/5): ERROR 00006_otel_logs_rollups.sql: failed to run SQL migration: failed to execute SQL query "CREATE MATERIALIZED VIEW IF NOT EXISTS default.otel_logs_attr_kv_rollup_15m_mv TO default.otel_logs_kv_rollup_15m\nAS WITH elements AS (\n SELECT\n 'ResourceAttributes' AS ColumnIdentifier,\n toStartOfFifteenMinutes(Timestamp) AS Timestamp,\n replaceRegexpAll(entry.1, '\\\\[\\\\d+\\\\]', '[*]') AS Key,\n CAST(entry.2 AS String) AS Value\n FROM default.otel_logs\n ARRAY JOIN ResourceAttributes AS entry\n UNION ALL\n SELECT\n 'LogAttributes' AS ColumnIdentifier,\n toStartOfFifteenMinutes(Timestamp) AS Timestamp,\n replaceRegexpAll(entry.1, '\\\\[\\\\d+\\\\]', '[*]') AS Key,\n CAST(entry.2 AS String) AS Value\n FROM default.otel_logs\n ARRAY JOIN LogAttributes AS entry\n UNION ALL\n SELECT\n 'ScopeAttributes' AS ColumnIdentifier,\n toStartOfFifteenMinutes(Timestamp) AS Timestamp,\n replaceRegexpAll(entry.1, '\\\\[\\\\d+\\\\]', '[*]') AS Key,\n CAST(entry.2 AS String) AS Value\n FROM default.otel_logs\n ARRAY JOIN ScopeAttributes AS entry\n UNION ALL\n SELECT\n 'NativeColumn' AS ColumnIdentifier,\n toStartOfFifteenMinutes(Timestamp) AS Timestamp,\n 'SeverityText' as Key,\n CAST(SeverityText AS String) as Value\n FROM default.otel_logs\n UNION ALL\n SELECT\n 'NativeColumn' AS ColumnIdentifier,\n toStartOfFifteenMinutes(Timestamp) AS Timestamp,\n 'ServiceName' as Key,\n CAST(ServiceName AS String) as Value\n FROM default.otel_logs\n UNION ALL\n SELECT\n 'NativeColumn' AS ColumnIdentifier,\n toStartOfFifteenMinutes(Timestamp) AS Timestamp,\n 'ScopeName' as Key,\n CAST(ScopeName AS String) as Value\n FROM default.otel_logs\n UNION ALL\n SELECT\n 'NativeColumn' AS ColumnIdentifier,\n toStartOfFifteenMinutes(Timestamp) AS Timestamp,\n 'ScopeVersion' as Key,\n CAST(ScopeVersion AS String) as Value\n FROM default.otel_logs\n UNION ALL\n SELECT\n 'NativeColumn' AS ColumnIdentifier,\n toStartOfFifteenMinutes(Timestamp) AS Timestamp,\n 'ResourceSchemaUrl' as Key,\n CAST(ResourceSchemaUrl AS String) as Value\n FROM default.otel_logs\n UNION ALL\n SELECT\n 'NativeColumn' AS ColumnIdentifier,\n toStartOfFifteenMinutes(Timestamp) AS Timestamp,\n 'ScopeSchemaUrl' as Key,\n CAST(ScopeSchemaUrl AS String) as Value\n FROM default.otel_logs\n UNION ALL\n SELECT\n 'NativeColumn' AS ColumnIdentifier,\n toStartOfFifteenMinutes(Timestamp) AS Timestamp,\n '__hdx_materialized_k8s.cluster.name' as Key,\n CAST(`__hdx_materialized_k8s.cluster.name` AS String) as Value\n FROM default.otel_logs\n UNION ALL\n SELECT\n 'NativeColumn' AS ColumnIdentifier,\n toStartOfFifteenMinutes(Timestamp) AS Timestamp,\n '__hdx_materialized_k8s.container.name' as Key,\n CAST(`__hdx_materialized_k8s.container.name` AS String) as Value\n FROM default.otel_logs\n UNION ALL\n SELECT\n 'NativeColumn' AS ColumnIdentifier,\n toStartOfFifteenMinutes(Timestamp) AS Timestamp,\n '__hdx_materialized_k8s.deployment.name' as Key,\n CAST(`__hdx_materialized_k8s.deployment.name` AS String) as Value\n FROM default.otel_logs\n UNION ALL\n SELECT\n 'NativeColumn' AS ColumnIdentifier,\n toStartOfFifteenMinutes(Timestamp) AS Timestamp,\n '__hdx_materialized_k8s.namespace.name' as Key,\n CAST(`__hdx_materialized_k8s.namespace.name` AS String) as Value\n FROM default.otel_logs\n UNION ALL\n SELECT\n 'NativeColumn' AS ColumnIdentifier,\n toStartOfFifteenMinutes(Timestamp) AS Timestamp,\n '__hdx_materialized_k8s.node.name' as Key,\n CAST(`__hdx_materialized_k8s.node.name` AS String) as Value\n FROM default.otel_logs\n UNION ALL\n SELECT\n 'NativeColumn' AS ColumnIdentifier,\n toStartOfFifteenMinutes(Timestamp) AS Timestamp,\n '__hdx_materialized_k8s.pod.name' as Key,\n CAST(`__hdx_materialized_k8s.pod.name` AS String) as Value\n FROM default.otel_logs\n UNION ALL\n SELECT\n 'NativeColumn' AS ColumnIdentifier,\n toStartOfFifteenMinutes(Timestamp) AS Timestamp,\n '__hdx_materialized_k8s.pod.uid' as Key,\n CAST(`__hdx_materialized_k8s.pod.uid` AS String) as Value\n FROM default.otel_logs\n UNION ALL\n SELECT\n 'NativeColumn' AS ColumnIdentifier,\n toStartOfFifteenMinutes(Timestamp) AS Timestamp,\n '__hdx_materialized_deployment.environment.name' as Key,\n CAST(`__hdx_materialized_deployment.environment.name` AS String) as Value\n FROM default.otel_logs\n)\nSELECT Timestamp, ColumnIdentifier, Key, Value, count() AS count FROM elements\nGROUP BY Timestamp, ColumnIdentifier, Key, Value;": code: 47, message: Unknown expression or function identifier `__hdx_materialized_k8s.cluster.name` in scope SELECT 'NativeColumn' AS ColumnIdentifier, toStartOfFifteenMinutes(Timestamp) AS Timestamp, '__hdx_materialized_k8s.cluster.name' AS Key, CAST(`__hdx_materialized_k8s.cluster.name`, 'String') AS Value FROM default.otel_logs
I could downgrade again to 2.24 for now.
I couldn't find any upgrade migration todos, so what do we need to do in order to update?
we just updated our hdx deployment from 2.24 to 2.27, and now the otel collector doesn't start, with these errors:
I could downgrade again to 2.24 for now.
I couldn't find any upgrade migration todos, so what do we need to do in order to update?