Skip to content

Read trades from derived table#2589

Closed
findolor wants to merge 1 commit into
arda/derived-trades-maintenancefrom
arda/derived-trades-read-path
Closed

Read trades from derived table#2589
findolor wants to merge 1 commit into
arda/derived-trades-maintenancefrom
arda/derived-trades-read-path

Conversation

@findolor
Copy link
Copy Markdown
Collaborator

@findolor findolor commented May 22, 2026

Chained PRs

Depends on #2588. Stack order: #2587 -> #2588 -> #2589 -> #2590

Motivation

The slow trade endpoints come from reconstructing trades from event tables on every read. Once the sync pipeline maintains derived_trades, the SDK read path should query that table directly.

Solution

  • Rewrite generic fetch_trades and fetch_trades_count to read from derived_trades.
  • Update owner, order-hash, and transaction trade query paths to use the derived table.
  • Preserve token metadata joins only for returned page rows instead of joining event tables across the full candidate set.
  • Keep existing filters for token input/output, owner, taker, order hash, transaction hash, time range, and pagination.

Local performance check

On the locally synced Base ST0x DB for 0xe522cB4a5fCb2eb31a52Ff41a4653d85A4fd7C9D, USDC matched 9,813 derived rows. The SDK local DB path averaged about 8.7ms for page + exact count over 100 iterations.

Checks

  • COMMIT_SHA=local nix develop -c cargo test -p raindex_common local_db::query --lib
  • COMMIT_SHA=local nix develop -c cargo test -p raindex_common local_db::pipeline::adapters::apply --lib
  • COMMIT_SHA=local nix develop -c cargo test -p raindex_common local_db::export --lib
  • COMMIT_SHA=local nix develop -c cargo test -p raindex_common raindex_client::trades --lib

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 22, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: b0fefc6b-998a-47d6-b8e6-28a1efcb9cd8

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch arda/derived-trades-read-path

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Collaborator Author

findolor commented May 22, 2026

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more


How to use the Graphite Merge Queue

Add the label Raindex-queue to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

@findolor findolor changed the base branch from arda/derived-trades-maintenance to graphite-base/2589 May 23, 2026 13:46
@findolor findolor force-pushed the graphite-base/2589 branch from 3124557 to 055e02d Compare May 23, 2026 13:47
@findolor findolor force-pushed the arda/derived-trades-read-path branch from 1abc55d to aac028c Compare May 23, 2026 13:47
@findolor findolor changed the base branch from graphite-base/2589 to arda/derived-trades-maintenance May 23, 2026 13:47
@findolor
Copy link
Copy Markdown
Collaborator Author

Folded into #2587 so the derived-trades local DB schema generation lands atomically.

@findolor findolor closed this May 23, 2026
graphite-app Bot pushed a commit that referenced this pull request May 26, 2026
## Motivation

Trade-list reads should use a pipeline-maintained local DB read model instead of reconstructing trades from source event tables on every request. This keeps producer bootstrap DBs and client incremental syncs on the same schema generation and avoids landing partial local DB protocol changes on `main`.

## Solution

- Add `derived_trades` as the derived/read-model table for local DB trades.
- Maintain `derived_trades` in the shared local DB apply pipeline in the same transaction as source event/state writes.
- Rebuild derived rows by sync window while allowing order resolution to look back to older active `AddOrderV3` events.
- Include the table in required-table checks, clear/reset paths, bootstrap export/import, and schema versioning.
- Update trade local DB read/count queries to read from `derived_trades` instead of reconstructing from event tables.
- Cover maintenance, incremental sync, bootstrap/export, clear/reset, and fetch equivalence with tests.

## Review note

This was folded from the previous split stack (#2588, #2589, #2590) so the local DB schema generation lands atomically.

## Checks

- `COMMIT_SHA=local nix develop -c cargo test -p raindex_app_settings --lib`
- `COMMIT_SHA=local nix develop -c cargo test -p raindex_common local_db::query --lib`
- `COMMIT_SHA=local nix develop -c cargo fmt --all`
- `git diff --check`

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

* **New Features**
  * Added a pipeline-maintained "derived trades" table with multiple indexes and an upsert refresh to speed and simplify trade queries.
  * Queries and exports now read from the derived-trades read model for faster, more consistent results.

* **Chores**
  * Database schema version bumped to 4; local DBs must be migrated/reinitialized to match the new schema.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant