Materialize vault deltas for balance sync#2592
Open
findolor wants to merge 1 commit into
Open
Conversation
Collaborator
Author
How to use the Graphite Merge QueueAdd the label Raindex-queue to this PR to add it to the merge queue. You must have a Graphite account in order to use the merge queue. Sign up using this link. An organization admin has enabled the Graphite Merge Queue in this repository. Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue. This stack of pull requests is managed by Graphite. Learn more about stacking. |
Contributor
|
Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (20)
📝 WalkthroughWalkthroughAdds a maintained derived_vault_deltas read-model: schema + indexes, a delete+insert upsert batch with unit and SQLite integration tests, pipeline wiring (after decoded inserts, before vault-balance upserts), downstream query rewires, DB schema bump to v4 with test fixture updates, scheduler readiness changes, and UI/constant tweaks. ChangesDerived Vault Deltas Read-Model
Scheduler readiness
UI & constants
Sequence DiagramsequenceDiagram
participant ApplyPipeline
participant UpsertBatch as upsert_derived_vault_deltas_batch
participant DB
participant VaultBalances as UpsertVaultBalances
ApplyPipeline->>UpsertBatch: build_batch(start_block,target_block,raindex)
UpsertBatch->>DB: DELETE FROM derived_vault_deltas WHERE block BETWEEN start..end
UpsertBatch->>DB: INSERT OR REPLACE INTO derived_vault_deltas SELECT ... FROM vault_deltas vd WHERE block BETWEEN start..end
ApplyPipeline->>VaultBalances: run vault balance upsert (reads derived_vault_deltas)
VaultBalances->>DB: SELECT from derived_vault_deltas for running_balance computation
DB->>VaultBalances: rows for balance aggregation
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
findolor
force-pushed
the
arda/derived-vault-deltas-standalone
branch
2 times, most recently
from
d36008f to
fbd5d6b
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/common/src/local_db/query/upsert_derived_vault_deltas/mod.rs`:
- Around line 8-17: The batch builder upsert_derived_vault_deltas_batch must
guard against an inverted block window (start_block > end_block) so it doesn't
silently produce empty BETWEEN predicates; update
upsert_derived_vault_deltas_batch to check the window and either normalize it
(swap start/end) or return an explicit no-op SqlStatementBatch (empty) when
start_block > end_block, and ensure this check happens before calling
delete_derived_vault_deltas_stmt and insert_derived_vault_deltas_stmt to avoid
generating statements with empty ranges.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 8605a942-d4e7-4348-8327-e65562aae98f
📒 Files selected for processing (15)
crates/cli/src/commands/local_db/pipeline/runner/manifest.rscrates/common/src/local_db/pipeline/adapters/apply.rscrates/common/src/local_db/pipeline/runner/environment.rscrates/common/src/local_db/pipeline/runner/remotes.rscrates/common/src/local_db/query/clear_raindex_data/query.sqlcrates/common/src/local_db/query/clear_tables/query.sqlcrates/common/src/local_db/query/create_tables/mod.rscrates/common/src/local_db/query/create_tables/query.sqlcrates/common/src/local_db/query/mod.rscrates/common/src/local_db/query/upsert_derived_vault_deltas/mod.rscrates/common/src/local_db/query/upsert_derived_vault_deltas/query.sqlcrates/common/src/local_db/query/upsert_vault_balances/insert_balance_changes.sqlcrates/common/src/local_db/query/upsert_vault_balances/insert_running_balances.sqlcrates/settings/src/local_db_manifest.rscrates/settings/src/remote/manifest.rs
✅ Files skipped from review due to trivial changes (4)
- crates/common/src/local_db/query/clear_raindex_data/query.sql
- crates/common/src/local_db/query/mod.rs
- crates/common/src/local_db/query/clear_tables/query.sql
- crates/settings/src/remote/manifest.rs
Comment on lines
+8
to
+17
| pub fn upsert_derived_vault_deltas_batch( | ||
| raindex_id: &RaindexIdentifier, | ||
| start_block: u64, | ||
| end_block: u64, | ||
| ) -> SqlStatementBatch { | ||
| SqlStatementBatch::from(vec![ | ||
| delete_derived_vault_deltas_stmt(raindex_id, start_block, end_block), | ||
| insert_derived_vault_deltas_stmt(raindex_id, start_block, end_block), | ||
| ]) | ||
| } |
Contributor
There was a problem hiding this comment.
Guard against inverted block windows in batch construction.
If start_block > end_block, both BETWEEN ?3 AND ?4 predicates become empty and the rebuild silently does nothing, which can leave stale rows in derived_vault_deltas.
Suggested fix
pub fn upsert_derived_vault_deltas_batch(
raindex_id: &RaindexIdentifier,
start_block: u64,
end_block: u64,
) -> SqlStatementBatch {
+ let (start_block, end_block) = if start_block <= end_block {
+ (start_block, end_block)
+ } else {
+ (end_block, start_block)
+ };
+
SqlStatementBatch::from(vec![
delete_derived_vault_deltas_stmt(raindex_id, start_block, end_block),
insert_derived_vault_deltas_stmt(raindex_id, start_block, end_block),
])
}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/common/src/local_db/query/upsert_derived_vault_deltas/mod.rs` around
lines 8 - 17, The batch builder upsert_derived_vault_deltas_batch must guard
against an inverted block window (start_block > end_block) so it doesn't
silently produce empty BETWEEN predicates; update
upsert_derived_vault_deltas_batch to check the window and either normalize it
(swap start/end) or return an explicit no-op SqlStatementBatch (empty) when
start_block > end_block, and ensure this check happens before calling
delete_derived_vault_deltas_stmt and insert_derived_vault_deltas_stmt to avoid
generating statements with empty ranges.
fbd5d6b to
087150a
Compare
findolor
changed the base branch from
graphite-base/2592
to
local-db-manifest-urls
This was referenced May 25, 2026
JuaniRios
approved these changes
May 25, 2026
4 tasks
0xgleb
approved these changes
May 25, 2026
graphite-app
Bot
changed the base branch from
local-db-manifest-urls
to
graphite-base/2592
findolor
force-pushed
the
arda/derived-vault-deltas-standalone
branch
from
087150a to
6f9e176
Compare
findolor
force-pushed
the
graphite-base/2592
branch
from
3f8691a to
7e32a38
Compare
findolor
changed the base branch from
graphite-base/2592
to
local-db-manifest-urls
findolor
force-pushed
the
arda/derived-vault-deltas-standalone
branch
from
bab7d79 to
d6970b6
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
crates/common/src/local_db/query/upsert_derived_vault_deltas/mod.rs (1)
8-16:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winGuard inverted block windows before constructing statements.
If
start_block > end_block, bothBETWEEN ?3 AND ?4clauses no-op and the targeted window is not refreshed.Suggested fix
pub fn upsert_derived_vault_deltas_batch( raindex_id: &RaindexIdentifier, start_block: u64, end_block: u64, ) -> SqlStatementBatch { + let (start_block, end_block) = if start_block <= end_block { + (start_block, end_block) + } else { + (end_block, start_block) + }; + SqlStatementBatch::from(vec![ delete_derived_vault_deltas_stmt(raindex_id, start_block, end_block), insert_derived_vault_deltas_stmt(raindex_id, start_block, end_block), ]) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/common/src/local_db/query/upsert_derived_vault_deltas/mod.rs` around lines 8 - 16, In upsert_derived_vault_deltas_batch, guard against inverted block windows by checking if start_block > end_block before building the statements (the calls to delete_derived_vault_deltas_stmt and insert_derived_vault_deltas_stmt); if the window is inverted, return an empty SqlStatementBatch (or otherwise no-op) instead of constructing statements with BETWEEN ?3 AND ?4 that will silently no-op, otherwise proceed to construct and return the batch as before.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/ui-components/src/lib/components/TanstackAppTable.svelte`:
- Around line 214-326: The PR is missing the required screenshot demonstrating
the frontend change in
packages/ui-components/src/lib/components/TanstackAppTable.svelte; please build
the app and capture a screenshot showing each visible state (loading skeleton -
element with data-testid="tableLoadingSkeleton", empty state -
data-testid="emptyMessage" showing emptyMessage text, populated table -
container data-testid="tanstackTableContainer" with rows rendered, and the "Load
More" button data-testid="loadMoreButton" in both enabled/disabled states as
applicable), then attach the image(s) to the pull request (or upload to the CI
artifacts for the build) so reviewers can verify the UI change.
---
Duplicate comments:
In `@crates/common/src/local_db/query/upsert_derived_vault_deltas/mod.rs`:
- Around line 8-16: In upsert_derived_vault_deltas_batch, guard against inverted
block windows by checking if start_block > end_block before building the
statements (the calls to delete_derived_vault_deltas_stmt and
insert_derived_vault_deltas_stmt); if the window is inverted, return an empty
SqlStatementBatch (or otherwise no-op) instead of constructing statements with
BETWEEN ?3 AND ?4 that will silently no-op, otherwise proceed to construct and
return the batch as before.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 917aa396-bf81-413f-b5e3-a72aff005179
📒 Files selected for processing (20)
crates/cli/src/commands/local_db/pipeline/runner/manifest.rscrates/common/src/local_db/pipeline/adapters/apply.rscrates/common/src/local_db/pipeline/runner/environment.rscrates/common/src/local_db/pipeline/runner/remotes.rscrates/common/src/local_db/query/clear_raindex_data/query.sqlcrates/common/src/local_db/query/clear_tables/query.sqlcrates/common/src/local_db/query/create_tables/mod.rscrates/common/src/local_db/query/create_tables/query.sqlcrates/common/src/local_db/query/mod.rscrates/common/src/local_db/query/upsert_derived_vault_deltas/mod.rscrates/common/src/local_db/query/upsert_derived_vault_deltas/query.sqlcrates/common/src/local_db/query/upsert_vault_balances/insert_balance_changes.sqlcrates/common/src/local_db/query/upsert_vault_balances/insert_running_balances.sqlcrates/common/src/raindex_client/local_db/pipeline/runner/scheduler/native.rscrates/common/src/raindex_client/local_db/pipeline/runner/scheduler/wasm.rscrates/settings/src/local_db_manifest.rscrates/settings/src/remote/manifest.rspackages/ui-components/src/__tests__/TanstackAppTable.test.tspackages/ui-components/src/lib/components/TanstackAppTable.sveltepackages/webapp/src/lib/constants.ts
✅ Files skipped from review due to trivial changes (2)
- crates/common/src/local_db/query/clear_tables/query.sql
- crates/cli/src/commands/local_db/pipeline/runner/manifest.rs
Merge activity
|
## Summary - add `derived_vault_deltas` as a pipeline-maintained read model for the existing `vault_deltas` view output - refresh derived vault deltas in the apply transaction before `vault_balance_changes` / `running_vault_balances` - switch vault balance maintenance SQL to read `derived_vault_deltas` instead of recomputing the view - include the table in required schema, clear/reset paths, export/import, and bump local DB schema version from 3 to 4 This PR is standalone on `main` and does not depend on the derived-trades stack. ## Benchmarks Measured with `nix develop` on a copied Base ST0x local DB, not the live DB. Before this PR, the same balance maintenance path measured: - recent 10k window: ~13.38s combined vault upsert batch - busy window: `upsert running_vault_balances` alone hit ~184.54s With `derived_vault_deltas` backfilled on the copy: - one-time full derived backfill: ~6.65s for 29,682 rows - recent 10k window: insert changes ~2.19ms, update running balances ~4.53ms - densest 10k window found: 644 derived rows, insert changes ~91.33ms, update running balances ~56.28ms ## Tests - `COMMIT_SHA=local nix develop -c cargo test -p raindex_common local_db::query --lib` - `COMMIT_SHA=local nix develop -c cargo test -p raindex_common local_db::pipeline::adapters::apply --lib` - `COMMIT_SHA=local nix develop -c cargo test -p raindex_common local_db::export --lib` - `COMMIT_SHA=local nix develop -c cargo test -p raindex_common raindex_client::local_db::pipeline::bootstrap --lib` - `COMMIT_SHA=local nix develop -c cargo test -p raindex_app_settings local_db_manifest --lib` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a derived vault-deltas read-model with windowed refreshes; pipeline now refreshes it during processing. * **Bug Fixes / Behavior** * Balance upserts now use derived deltas for consistent running balances and correct incremental updates. * Clear/reset operations now include derived deltas. * Sync scheduler marks chains as ready even when not leader. * **Chores** * Bumped DB schema to v4; tests/fixtures updated. * UI table now shows an initial loading skeleton. * Updated registry URL. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/rainlanguage/raindex/pull/2592?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai -->
](https://app.coderabbit.ai/change-stack/rainlanguage/raindex/pull/2592?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack){kind=link}
graphite-app
Bot
force-pushed
the
arda/derived-vault-deltas-standalone
branch
from
ab3eb52 to
691e0ff
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
derived_vault_deltasas a pipeline-maintained read model for the existingvault_deltasview outputvault_balance_changes/running_vault_balancesderived_vault_deltasinstead of recomputing the viewThis PR is standalone on
mainand does not depend on the derived-trades stack.Benchmarks
Measured with
nix developon a copied Base ST0x local DB, not the live DB.Before this PR, the same balance maintenance path measured:
upsert running_vault_balancesalone hit ~184.54sWith
derived_vault_deltasbackfilled on the copy:Tests
COMMIT_SHA=local nix develop -c cargo test -p raindex_common local_db::query --libCOMMIT_SHA=local nix develop -c cargo test -p raindex_common local_db::pipeline::adapters::apply --libCOMMIT_SHA=local nix develop -c cargo test -p raindex_common local_db::export --libCOMMIT_SHA=local nix develop -c cargo test -p raindex_common raindex_client::local_db::pipeline::bootstrap --libCOMMIT_SHA=local nix develop -c cargo test -p raindex_app_settings local_db_manifest --libSummary by CodeRabbit
New Features
Bug Fixes / Behavior
Chores