Optimize live orders queries based on confirmed_valid_to column

[m-sz]

Description

Second part of #4047 which introduced optimized queries based on the introduced confirmed_valid_to column

It is crucial for the database to be already migrated manually as described in previous PR before applying this one.

Changes

• Adapt user_orders_with_quote query to use new column
• Adapt solvable_orders query to use new column

How to test

Tested on a test-db created by @MartinquaXD which contains a snapshot of prod data. The optimized queries run significantly faster due to changes in orders table and new indices.

Related Issues

#4021

[github-actions]

Reminder: Please update the DB Readme and comment whether migrations are reversible (include rollback scripts if applicable).

• If creating new tables, update the tables list.
• When adding a new index, consider using CREATE INDEX CONCURRENTLY for tables involved in the critical execution path.
• For breaking changes, remember that during rollout k8s starts the new autopilot, runs the Flyway migration, and only then shuts down the old pod. That overlap means the previous version can still be processing requests on the migrated schema, so make it compatible first and ship the breaking DB change in the following release.

---

Caused by:

• database/sql/V095__add_confirmed_valid_to_to_orders.sql

[gemini-code-assist]

Code Review

This pull request optimizes order queries by using a new confirmed_valid_to column, which is a good performance improvement. However, I've found a few critical issues that need to be addressed. There's a bug in how confirmed_valid_to is populated for ethflow orders, which could lead to incorrect query results. Additionally, a refactoring of the solvable_orders query seems incomplete and will cause the query to fail at runtime. I've also included some suggestions for further query optimization and documentation improvements.

[gemini-code-assist]

The confirmed_valid_to field is being set to order_placement.order.validTo, which is the on-chain validTo value. However, for ethflow orders, this should be the user-specified valid_to to align with the query optimizations in crates/database/src/orders.rs.

The new query logic relies on confirmed_valid_to to correctly filter expired orders, and for ethflow orders, the user-specified valid_to (from ethflow_orders table) is the one that matters. The backfill logic in the migration file also confirms this expectation.

This is a critical issue as it will lead to incorrect filtering of ethflow orders. You'll need to refactor to pass the user valid_to (when available for ethflow orders) to this function and use it to populate confirmed_valid_to.

[gemini-code-assist]

Removing the trades_agg CTE here and the corresponding LEFT JOIN in a later change breaks the solvable_orders query. The SELECT list of the query (lines 757-759 in the file) still references the ta alias from this join to get sum_buy, sum_sell, and sum_fee.

Since ta will no longer be defined, the query will fail at runtime. To fix this, you should update the SELECT list to use subqueries to calculate these sums, similar to how it's done in the WHERE clause, or use a LATERAL join to compute the aggregates.

[gemini-code-assist]

This test sets confirmed_valid_to to the same value as valid_to. While this is correct for non-ethflow orders, it doesn't cover the case for ethflow orders where the user-specified valid_to can be different from the on-chain validTo.

To prevent regressions related to the bug I mentioned about populating confirmed_valid_to, I recommend extending this test or adding a new one to specifically cover the ethflow order scenario where these two valid_to values differ.

[gemini-code-assist]

The LEFT JOIN ethflow_orders e ON e.uid = o.uid seems to be redundant now. The e alias is not used in the WHERE clause of the live_orders CTE anymore, and the necessary data from ethflow_orders is fetched later in a LATERAL join (lines 795-800). Removing this join would make the query slightly more efficient.

[gemini-code-assist]

This LEFT JOIN on ethflow_orders appears to be unnecessary. The alias e is not used anywhere in the rest of the user_orders_with_quote query. Removing this join will improve query performance.

[gemini-code-assist]

The documentation for the new column and index has a few inaccuracies:

1. The type for confirmed_valid_to is listed as timestamptz, but the migration file defines it as bigint.
2. The orders table cannot have two PRIMARY KEYs. The new index should be documented as a regular index.
3. The new index is on (owner, confirmed_valid_to), but the migration file creates it only on (confirmed_valid_to). The documentation should match the implementation.

Please update the documentation to reflect the actual schema changes.

[gemini-code-assist]

For better query performance, consider creating a composite index on (owner, confirmed_valid_to). The user_orders_with_quote query filters by both owner and confirmed_valid_to. A composite index on both columns would be more efficient for this query than the current index on just confirmed_valid_to.

CREATE INDEX orders_owner_live_limit ON orders USING btree (owner, confirmed_valid_to)