Skip to content

perf: flat SQL for single-op pipelines and group_by+summarise#50

Merged
cigrainger merged 1 commit into
elixir-dux:mainfrom
hugobarauna:perf/flat-sql-single-op
Apr 5, 2026
Merged

perf: flat SQL for single-op pipelines and group_by+summarise#50
cigrainger merged 1 commit into
elixir-dux:mainfrom
hugobarauna:perf/flat-sql-single-op

Conversation

@hugobarauna
Copy link
Copy Markdown
Contributor

Summary

Skip CTE wrapping for common pipeline shapes:

  • Single-op pipelines (e.g. filter or mutate alone) emit flat SQL instead of WITH __s0 AS (...) SELECT * FROM __s0
  • group_by + summarise (the most common aggregation) emits a single SELECT ... GROUP BY without CTEs
  • Table sources are referenced by name directly instead of wrapping in (SELECT * FROM "table") __src

Benchmark (1M rows, Apple M2 Pro, Benchee)

Operation main this PR Change
compute: filter 0.98ms (1018 ips) 0.94ms (1066 ips) +5%
compute: mutate 0.66ms (1507 ips) 0.59ms (1701 ips) +13%
e2e: summarise→to_rows 7.79ms (128 ips) 6.11ms (164 ips) +28%
e2e: filter→to_rows 361ms (2.77 ips) 349ms (2.87 ips) +4%

Benchmark script: https://gist.github.com/hugobarauna/369f68acf4292f27309782ba774b8988

Example

Before:

WITH
  __s0 AS (SELECT * FROM (SELECT * FROM ("__dux_123") __src) __src WHERE quantity > 50)
SELECT * FROM __s0

After:

SELECT * FROM "__dux_123" WHERE quantity > 50

Three QueryBuilder optimizations:

1. Single-op pipelines emit flat SQL without CTE wrapping
2. group_by+summarise pattern emits a single SELECT...GROUP BY
3. Table sources referenced directly by name instead of subquery
@cigrainger
Copy link
Copy Markdown
Contributor

Hey @hugobarauna thank you for this. Looks like a pretty straightforward win! Crazy few days over here but I'll have a proper look Monday.

@cigrainger
Copy link
Copy Markdown
Contributor

Thank you! <3

@cigrainger cigrainger merged commit 3bda19c into elixir-dux:main Apr 5, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants