fix: Correct dependency tracking for nested subqueries

[robacourt]

Summary

• Fixes dependency tracking for nested subqueries when intermediate rows change their linking column without changing the tracked column
• Previously, such updates were incorrectly filtered out, causing stale tag tracking that led to incorrect row deletions

Changes

• shape.ex: Keep updates with tag changes (removed_move_tags != []) even if selected columns are unchanged
• materializer.ex: Skip value count updates when value is unchanged to avoid spurious move_out/move_in events

Test plan

• Added unit tests in shape_test.exs for filtered_columns_changed? behavior
• Added unit tests in materializer_test.exs for tag-only update handling
• Integration tests in subquery_dependency_update_test.exs pass
• Full test suite passes (1322 tests, 0 failures)

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes

  • Fixed dependency tracking for nested subqueries so intermediate row moves that don’t change tracked values no longer cause spurious deletions.
  • Preserved tag-only updates in change detection to prevent stale tag tracking when rows move between parents.

• Tests

  • Added extensive tests covering tag-only updates and nested subquery dependency scenarios to validate correctness.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

📝 Walkthrough

Walkthrough

Fixes dependency tracking for nested subqueries by preserving tag-only updates and separating tag-index updates from value_count/index updates so move events are not emitted when tracked column values remain unchanged; adds unit and integration tests reproducing and preventing the stale-tag behavior.

Changes

Cohort / File(s)	Summary
Changeset Documentation \.changeset/fix-nested-subquery-tag-tracking.md	Adds a patch-level changeset describing the fix to dependency tracking for nested subqueries and the tag-only update behavior.
Materializer logic packages/sync-service/lib/electric/shapes/consumer/materializer.ex	Splits handling into columns_present and has_tag_updates: tag index updates run when tag changes exist; value_counts and index value updates run only when selected columns are present and the tracked value actually changed. Removes unconditional value_count/move logic for tag-only updates.
Change filtering packages/sync-service/lib/electric/shapes/shape.ex	Replaces previous predicate with a new should_keep_change?/1 predicate that preserves UpdatedRecords with non-empty removed_move_tags; filters out only true no-op updates.
Unit tests packages/sync-service/test/electric/shapes/consumer/materializer_test.exs, packages/sync-service/test/electric/shapes/shape_test.exs	Adds tests for "tag-only updates (value unchanged)" in materializer_test and a shape test ensuring updates with tag changes are kept even if selected columns unchanged.
Integration tests packages/sync-service/test/integration/subquery_dependency_update_test.exs	New integration test module reproducing the stale dependency-tracking scenario across nested relations (orgs → teams → projects → tasks) with premium-tag moves to assert no spurious deletions.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested reviewers

• icehaunter
• magnetised

Poem

🐰 I hopped through tags and nested lanes,

Where moves once left old phantom pains.
Now I nudge indexes, count what's due,
Tags shift softly — values stay true.
A little hop, no row undone, hooray!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix: Correct dependency tracking for nested subqueries' directly and clearly summarizes the main change: fixing dependency tracking for nested subqueries, which is the core issue addressed across all modified files.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

---

📜 Recent review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 84a41a5 and 3f2dc88.

📒 Files selected for processing (2)

• packages/sync-service/lib/electric/shapes/consumer/materializer.ex
• packages/sync-service/lib/electric/shapes/shape.ex

🚧 Files skipped from review as they are similar to previous changes (1)

• packages/sync-service/lib/electric/shapes/shape.ex

🔇 Additional comments (1)

packages/sync-service/lib/electric/shapes/consumer/materializer.ex (1)

330-360: LGTM! Clean separation of tag-index updates from value-count updates.

The fix correctly addresses the nested subquery dependency tracking issue by:

1. Separating tag index updates (Lines 334-337) from value count updates (Lines 339-356), ensuring tag changes are always tracked when has_tag_updates is true.
2. Skipping the decrement/increment dance when old_value == value (Lines 346-353), which prevents spurious move_out/move_in events for tag-only changes.
3. Handling the case where columns_present is false but has_tag_updates is true (Lines 354-356), correctly updating only tag indices without touching value counts.

One minor observation: at Line 342, the index is updated unconditionally before the value comparison at Line 346. This is harmless (idempotent Map.put), but you could potentially skip the Map.put when values are equal for a micro-optimization. Not worth changing given the fix is correct.

_{✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.36%. Comparing base (b2d28cf) to head (3f2dc88).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #3757       +/-   ##
===========================================
+ Coverage   75.75%   87.36%   +11.61%     
===========================================
  Files          11       23       +12     
  Lines         693     2011     +1318     
  Branches      173      534      +361     
===========================================
+ Hits          525     1757     +1232     
- Misses        167      252       +85     
- Partials        1        2        +1     

Flag	Coverage Δ
packages/experimental	87.73% <ø> (ø)
packages/react-hooks	86.48% <ø> (ø)
packages/start	82.83% <ø> (ø)
packages/typescript-client	93.47% <ø> (?)
packages/y-electric	56.05% <ø> (ø)
typescript	87.36% <ø> (+11.61%)	⬆️
unit-tests	87.36% <ø> (+11.61%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[netlify]

✅ Deploy Preview for electric-next ready!

Name	Link
🔨 Latest commit	864881c
🔍 Latest deploy log	https://app.netlify.com/projects/electric-next/deploys/6971ecd6e7d921000886f6cd
😎 Deploy Preview	https://deploy-preview-3757--electric-next.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.