Skip to content

feat: match non-set operators against array-typed user properties#74

Merged
kyeh-amp merged 2 commits into
mainfrom
multi-value-array-eval-fix
Apr 28, 2026
Merged

feat: match non-set operators against array-typed user properties#74
kyeh-amp merged 2 commits into
mainfrom
multi-value-array-eval-fix

Conversation

@kyeh-amp
Copy link
Copy Markdown
Collaborator

@kyeh-amp kyeh-amp commented Apr 22, 2026

Summary

  • Route non-set operators (is, contains, greater, etc.) through the same list-coercion path as set operators so they match array/JSON-array-string user properties using any-match semantics (including negation operators like is not and does not contain). Scalars still flow through the existing single-string path.
  • Add a startswith("[") guard to coerce_string_array so scalar strings skip JSON parsing (and its exception) on every evaluation.
  • Fix two pre-existing bugs in coerce_string_array surfaced by the new path: the JSON-array branch iterated the original string value instead of the parsed list, and scalars were wrapped as [s] instead of returning None.

Test plan

  • New engine-level unit tests: 14 spec cases + 1 defensive empty-array/non-set case, covering scalar / list / JSON-array-string inputs across set and non-set operators.
  • Existing integration tests pass against the new superset deployment key.
  • 7 new integration tests for array and JSON-array-string inputs with is, is not, contains, does not contain, and set is.
  • Full test suite: no regressions.

Note

Medium Risk
Changes core targeting semantics for multi-valued properties and JSON-array strings, which can alter flag rollout behavior across existing segments. Added tests reduce regression risk but evaluation logic changes warrant careful review.

Overview
Multi-valued user properties now work with non-set operators. EvaluationEngine.match_condition routes non-set ops (e.g., is, contains, comparisons) through array coercion and applies any-element matching via new match_strings_non_set, while preserving the existing scalar-string path.

Array coercion behavior is tightened/fixed. coerce_string_array now only returns lists for actual Python lists or JSON array strings (guarded by startswith("[")), and no longer falls back to wrapping scalars / malformed JSON; this also fixes the prior bug where the parsed JSON array wasn’t actually iterated.

Tests expanded. Adds a new unit test suite for array/non-set matching and extends integration tests (plus a new deployment key) to cover list + JSON-array-string inputs for both set and non-set operators.

Reviewed by Cursor Bugbot for commit 29dddfa. Bugbot is set up for automated code reviews on this repo. Configure here.

Route non-set operators (is, contains, greater, etc.) through the same
list-coercion path as set operators. When the property value is a list
or a JSON array string, match against each element with any-match
semantics — the condition is true if any single element satisfies the
operator, including for negation operators like `is not` and
`does not contain`. Scalar values continue to flow through the existing
single-string match path.

Add a `startswith("[")` guard to coerce_string_array so scalar strings
skip JSON parsing (and its exception) on every evaluation.

Fix two pre-existing bugs in coerce_string_array surfaced by the new
path: the JSON-array branch iterated the original string value rather
than the parsed list, and scalars were wrapped as `[s]` rather than
returning None.

Add engine-level unit tests covering scalar, list, and JSON-array-string
property values across set and non-set operators. Update the integration
test deployment key to the new superset deployment and add operator
cases for arrays and JSON array strings.
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Negation operators use wrong quantifier on arrays
    • Changed match_strings_non_set to use all() instead of any() for negation operators (IS_NOT, DOES_NOT_CONTAIN, REGEX_DOES_NOT_MATCH), ensuring correct universal quantification semantics where no element matches rather than at least one element doesn't match.

Create PR

Or push these changes by commenting:

@cursor push 0ccd8617c8
Preview (0ccd8617c8)
diff --git a/src/amplitude_experiment/evaluation/engine.py b/src/amplitude_experiment/evaluation/engine.py
--- a/src/amplitude_experiment/evaluation/engine.py
+++ b/src/amplitude_experiment/evaluation/engine.py
@@ -234,6 +234,15 @@
             filter_values: List[str]
     ) -> bool:
         """Match any element of a multi-valued property against a non-set operator."""
+        if op in {
+            EvaluationOperator.IS_NOT,
+            EvaluationOperator.DOES_NOT_CONTAIN,
+            EvaluationOperator.REGEX_DOES_NOT_MATCH,
+        }:
+            return all(
+                self.match_string(prop_value, op, filter_values)
+                for prop_value in prop_values
+            )
         return any(
             self.match_string(prop_value, op, filter_values)
             for prop_value in prop_values

You can send follow-ups to the cloud agent here.

Reviewed by Cursor Bugbot for commit b07bfd4. Configure here.

Comment thread src/amplitude_experiment/evaluation/engine.py
@kyeh-amp
Copy link
Copy Markdown
Collaborator Author

Rejecting the Bugbot suggestion to use all() for negation operators in match_strings_non_set.

The current any() behavior is intentional and matches the evaluation spec and the Kotlin reference implementation in experiment-evaluation:

  • Spec explicitly mandates any-match for negation: "Negation operators (is not, does not contain) also use any-match, meaning the condition is true if any single element satisfies the negation — not if all elements do. For example, is not \"A\" on [\"A\", \"B\"] is true because \"B\" is not \"A\", even though \"A\" is present. This matches analytics/charts behavior."
  • Reference implementation (experiment-evaluation/evaluation-core/src/commonMain/kotlin/EvaluationEngine.kt) uses propValues.any { matchString(it, op, filterValues) } with the same comment.
  • Switching to all() would break analytics/charts parity on the canonical example above (it would return no-match where charts return match).

Added an inline docstring at match_strings_non_set in 29dddfa documenting the rationale so this doesn't get re-flagged.

@kyeh-amp kyeh-amp merged commit 626e94d into main Apr 28, 2026
7 checks passed
@kyeh-amp kyeh-amp deleted the multi-value-array-eval-fix branch April 28, 2026 20:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants