Skip to content

Conversation

@phacops
Copy link
Contributor

@phacops phacops commented Jan 15, 2026

Add support for boolean attribute keys in the autocomplete functionality by:

  • Creating migration 0051 to add attributes_bool column to autocomplete tables
  • Updating the materialized view to extract boolean keys from attributes_bool
  • Modifying the RPC endpoint to query the new boolean column for TYPE_BOOLEAN requests

@phacops phacops requested review from a team as code owners January 15, 2026 19:35
@github-actions
Copy link

github-actions bot commented Jan 15, 2026

This PR has a migration; here is the generated SQL for ./snuba/migrations/groups.py ()

-- start migrations

-- forward migration events_analytics_platform : 0051_add_bool_keys_to_autocomplete
Local op: ALTER TABLE eap_item_co_occurring_attrs_1_local ADD COLUMN IF NOT EXISTS attributes_bool Array(String) AFTER attributes_float;
Distributed op: ALTER TABLE eap_item_co_occurring_attrs_1_dist ADD COLUMN IF NOT EXISTS attributes_bool Array(String) AFTER attributes_float;
Local op: CREATE MATERIALIZED VIEW IF NOT EXISTS eap_item_co_occurring_attrs_2_mv TO eap_item_co_occurring_attrs_1_local (organization_id UInt64, project_id UInt64, item_type UInt8, date Date CODEC (DoubleDelta, ZSTD(1)), retention_days UInt16, attribute_keys_hash Array(UInt64) MATERIALIZED arrayMap(k -> cityHash64(k), arrayDistinct(arrayConcat(attributes_string, attributes_float, attributes_bool))), attributes_string Array(String), attributes_float Array(String), attributes_bool Array(String), key_hash UInt64 MATERIALIZED cityHash64(arraySort(arrayDistinct(arrayConcat(attributes_string, attributes_float, attributes_bool))))) AS 
SELECT
    organization_id AS organization_id,
    project_id AS project_id,
    item_type as item_type,
    toMonday(timestamp) AS date,
    retention_days as retention_days,
    arrayConcat(mapKeys(attributes_string_0), mapKeys(attributes_string_1), mapKeys(attributes_string_2), mapKeys(attributes_string_3), mapKeys(attributes_string_4), mapKeys(attributes_string_5), mapKeys(attributes_string_6), mapKeys(attributes_string_7), mapKeys(attributes_string_8), mapKeys(attributes_string_9), mapKeys(attributes_string_10), mapKeys(attributes_string_11), mapKeys(attributes_string_12), mapKeys(attributes_string_13), mapKeys(attributes_string_14), mapKeys(attributes_string_15), mapKeys(attributes_string_16), mapKeys(attributes_string_17), mapKeys(attributes_string_18), mapKeys(attributes_string_19), mapKeys(attributes_string_20), mapKeys(attributes_string_21), mapKeys(attributes_string_22), mapKeys(attributes_string_23), mapKeys(attributes_string_24), mapKeys(attributes_string_25), mapKeys(attributes_string_26), mapKeys(attributes_string_27), mapKeys(attributes_string_28), mapKeys(attributes_string_29), mapKeys(attributes_string_30), mapKeys(attributes_string_31), mapKeys(attributes_string_32), mapKeys(attributes_string_33), mapKeys(attributes_string_34), mapKeys(attributes_string_35), mapKeys(attributes_string_36), mapKeys(attributes_string_37), mapKeys(attributes_string_38), mapKeys(attributes_string_39)) AS attributes_string,
    mapKeys(attributes_bool) AS attributes_bool,
    arrayConcat(mapKeys(attributes_float_0), mapKeys(attributes_float_1), mapKeys(attributes_float_2), mapKeys(attributes_float_3), mapKeys(attributes_float_4), mapKeys(attributes_float_5), mapKeys(attributes_float_6), mapKeys(attributes_float_7), mapKeys(attributes_float_8), mapKeys(attributes_float_9), mapKeys(attributes_float_10), mapKeys(attributes_float_11), mapKeys(attributes_float_12), mapKeys(attributes_float_13), mapKeys(attributes_float_14), mapKeys(attributes_float_15), mapKeys(attributes_float_16), mapKeys(attributes_float_17), mapKeys(attributes_float_18), mapKeys(attributes_float_19), mapKeys(attributes_float_20), mapKeys(attributes_float_21), mapKeys(attributes_float_22), mapKeys(attributes_float_23), mapKeys(attributes_float_24), mapKeys(attributes_float_25), mapKeys(attributes_float_26), mapKeys(attributes_float_27), mapKeys(attributes_float_28), mapKeys(attributes_float_29), mapKeys(attributes_float_30), mapKeys(attributes_float_31), mapKeys(attributes_float_32), mapKeys(attributes_float_33), mapKeys(attributes_float_34), mapKeys(attributes_float_35), mapKeys(attributes_float_36), mapKeys(attributes_float_37), mapKeys(attributes_float_38), mapKeys(attributes_float_39)) AS attributes_float
FROM eap_items_1_local
;
Local op: DROP TABLE IF EXISTS eap_item_co_occurring_attrs_1_mv SYNC;
-- end forward migration events_analytics_platform : 0051_add_bool_keys_to_autocomplete




-- backward migration events_analytics_platform : 0051_add_bool_keys_to_autocomplete
Local op: CREATE MATERIALIZED VIEW IF NOT EXISTS eap_item_co_occurring_attrs_1_mv TO eap_item_co_occurring_attrs_1_local (organization_id UInt64, project_id UInt64, item_type UInt8, date Date CODEC (DoubleDelta, ZSTD(1)), retention_days UInt16, attribute_keys_hash Array(UInt64) MATERIALIZED arrayMap(k -> cityHash64(k), arrayConcat(attributes_string, attributes_float)), attributes_string Array(String), attributes_float Array(String), key_hash UInt64 MATERIALIZED cityHash64(arraySort(arrayConcat(attributes_string, attributes_float)))) AS 
SELECT
    organization_id AS organization_id,
    project_id AS project_id,
    item_type as item_type,
    toMonday(timestamp) AS date,
    retention_days as retention_days,
    arrayConcat(mapKeys(attributes_string_0), mapKeys(attributes_string_1), mapKeys(attributes_string_2), mapKeys(attributes_string_3), mapKeys(attributes_string_4), mapKeys(attributes_string_5), mapKeys(attributes_string_6), mapKeys(attributes_string_7), mapKeys(attributes_string_8), mapKeys(attributes_string_9), mapKeys(attributes_string_10), mapKeys(attributes_string_11), mapKeys(attributes_string_12), mapKeys(attributes_string_13), mapKeys(attributes_string_14), mapKeys(attributes_string_15), mapKeys(attributes_string_16), mapKeys(attributes_string_17), mapKeys(attributes_string_18), mapKeys(attributes_string_19), mapKeys(attributes_string_20), mapKeys(attributes_string_21), mapKeys(attributes_string_22), mapKeys(attributes_string_23), mapKeys(attributes_string_24), mapKeys(attributes_string_25), mapKeys(attributes_string_26), mapKeys(attributes_string_27), mapKeys(attributes_string_28), mapKeys(attributes_string_29), mapKeys(attributes_string_30), mapKeys(attributes_string_31), mapKeys(attributes_string_32), mapKeys(attributes_string_33), mapKeys(attributes_string_34), mapKeys(attributes_string_35), mapKeys(attributes_string_36), mapKeys(attributes_string_37), mapKeys(attributes_string_38), mapKeys(attributes_string_39)) AS attributes_string,
    arrayConcat(mapKeys(attributes_float_0), mapKeys(attributes_float_1), mapKeys(attributes_float_2), mapKeys(attributes_float_3), mapKeys(attributes_float_4), mapKeys(attributes_float_5), mapKeys(attributes_float_6), mapKeys(attributes_float_7), mapKeys(attributes_float_8), mapKeys(attributes_float_9), mapKeys(attributes_float_10), mapKeys(attributes_float_11), mapKeys(attributes_float_12), mapKeys(attributes_float_13), mapKeys(attributes_float_14), mapKeys(attributes_float_15), mapKeys(attributes_float_16), mapKeys(attributes_float_17), mapKeys(attributes_float_18), mapKeys(attributes_float_19), mapKeys(attributes_float_20), mapKeys(attributes_float_21), mapKeys(attributes_float_22), mapKeys(attributes_float_23), mapKeys(attributes_float_24), mapKeys(attributes_float_25), mapKeys(attributes_float_26), mapKeys(attributes_float_27), mapKeys(attributes_float_28), mapKeys(attributes_float_29), mapKeys(attributes_float_30), mapKeys(attributes_float_31), mapKeys(attributes_float_32), mapKeys(attributes_float_33), mapKeys(attributes_float_34), mapKeys(attributes_float_35), mapKeys(attributes_float_36), mapKeys(attributes_float_37), mapKeys(attributes_float_38), mapKeys(attributes_float_39)) AS attributes_float
FROM eap_items_1_local
;
Local op: DROP TABLE IF EXISTS eap_item_co_occurring_attrs_2_mv SYNC;
Distributed op: ALTER TABLE eap_item_co_occurring_attrs_1_dist DROP COLUMN IF EXISTS attributes_bool;
Local op: ALTER TABLE eap_item_co_occurring_attrs_1_local DROP COLUMN IF EXISTS attributes_bool;
-- end backward migration events_analytics_platform : 0051_add_bool_keys_to_autocomplete

@phacops phacops force-pushed the pierre/eap-add-boolean-keys-to-autocomplete branch from bd932a4 to a3abbb9 Compare January 15, 2026 21:10
@phacops phacops requested a review from volokluev January 15, 2026 22:20
@phacops phacops force-pushed the pierre/eap-add-boolean-keys-to-autocomplete branch from a3fe48c to 50317f0 Compare January 15, 2026 22:23
@phacops phacops force-pushed the pierre/eap-add-boolean-keys-to-autocomplete branch from f38ca57 to 56a622c Compare January 15, 2026 22:27
@phacops phacops enabled auto-merge (squash) January 15, 2026 23:13
@phacops phacops changed the title feat(eap): Add boolean key attributes to RPC autocomplete feat(autocomplete): Add boolean key attributes to RPC autocomplete Jan 15, 2026
toMonday(timestamp) AS date,
retention_days as retention_days,
arrayConcat({_attr_str_names}) AS attributes_string,
mapKeys(attributes_bool) AS attributes_bool,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The migration adds an attributes_bool column, but the storage schema YAML is not updated, and the endpoint still queries the old attributes_float column for booleans.
Severity: CRITICAL

Suggested Fix

Add the new attributes_bool column to the columns list in snuba/datasets/configuration/events_analytics_platform/storages/eap_item_co_occurring_attrs.yaml. Also, update the PROTO_TYPE_TO_ATTRIBUTE_COLUMN mapping in snuba/protos/common.py to map TYPE_BOOLEAN to "attributes_bool" so the endpoint queries the correct column.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location:
snuba/snuba_migrations/events_analytics_platform/0051_add_bool_keys_to_autocomplete.py#L59

Potential issue: The migration `0051_add_bool_keys_to_autocomplete.py` adds an
`attributes_bool` column to the `eap_item_co_occurring_attrs` table and updates its
materialized view. However, the corresponding storage schema file,
`eap_item_co_occurring_attrs.yaml`, is not updated to include this new column. This will
cause a schema mismatch between the application's definition and the actual database
table, leading to failures during schema loading. Furthermore, the autocomplete endpoint
for boolean attributes (`TYPE_BOOLEAN`) is still configured to query the
`attributes_float` column, but the boolean keys will now reside in the new
`attributes_bool` column, causing autocomplete for these attributes to fail.

Did we get this right? 👍 / 👎 to inform future reviews.

@phacops phacops merged commit 84aee63 into master Jan 20, 2026
34 checks passed
@phacops phacops deleted the pierre/eap-add-boolean-keys-to-autocomplete branch January 20, 2026 22:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants