Skip to content

feat: over() helper for window functions in Query macro#49

Merged
cigrainger merged 1 commit into
mainfrom
feat/window-over
Apr 2, 2026
Merged

feat: over() helper for window functions in Query macro#49
cigrainger merged 1 commit into
mainfrom
feat/window-over

Conversation

@cigrainger
Copy link
Copy Markdown
Contributor

@cigrainger cigrainger commented Apr 1, 2026

Summary

Adds over() for window functions inside mutate expressions with a full tuple-based frame syntax. Addresses feedback from @billylanchantin about making frames feel native rather than stringly-typed.

require Dux

df
|> Dux.mutate(
  rank: over(row_number(), partition_by: :dept, order_by: [desc: :salary]),
  running: over(sum(amount), order_by: :date, frame: {:rows, :unbounded, :current}),
  moving_avg: over(avg(price), order_by: :date, frame: {:rows, -2, 2}),
  prev: over(lag(amount, 1), order_by: :date),
  pct: salary * 100.0 / over(sum(salary), partition_by: :dept),
  others: over(sum(x), frame: {:rows, :unbounded, :unbounded, exclude: :current})
)

Frame tuple syntax

`{type, start, end}` where:

  • type: `:rows`, `:range`, or `:groups`
  • start: negative int (PRECEDING), `:unbounded`, `:current`, or `0`
  • end: positive int (FOLLOWING), `:unbounded`, `:current`, or `0`
Frame SQL
`{:rows, -2, :current}` `ROWS BETWEEN 2 PRECEDING AND CURRENT ROW`
`{:rows, :unbounded, :current}` `ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW`
`{:rows, -2, 2}` `ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING`
`{:rows, :unbounded, :unbounded}` `ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING`
`{:rows, :current, :unbounded}` `ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING`
`{:range, :unbounded, :current}` `RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW`

Add `exclude:` as a 4th element:

Frame SQL
`{:rows, :unbounded, :unbounded, exclude: :current}` `... EXCLUDE CURRENT ROW`
`{:rows, :unbounded, :unbounded, exclude: :ties}` `... EXCLUDE TIES`
`{:rows, :unbounded, :unbounded, exclude: :group}` `... EXCLUDE GROUP`

Raw SQL strings still work as a fallback for anything the tuple syntax doesn't cover.

Implementation

`over()` is a macro transformation in `Dux.Query`. Compiles to `FUNC() OVER (PARTITION BY ... ORDER BY ... frame)`. Frame tuples are resolved at macro expansion time (handles Elixir AST representation of negative integers and 3+ element tuples).

Test plan (35 tests: 33 unit + 2 property-based)

746 total tests pass, 0 credo issues.

  • Happy path (8): row_number, running sum, bare window, lag/lead, multi-partition, arithmetic, first/last_value, multiple windows
  • Tuple frames (10): rows preceding+current, cumulative, centered, unbounded, range, current-to-following, 0-as-current, exclude current, exclude ties, string fallback
  • Ranking (3): dense_rank ties, rank gaps, ntile
  • Sad path (3): invalid frame, bad partition col, bad order col
  • Adversarial (4): empty dataset, single row, NULLs, interpolated values
  • Scale/wicked (3): 2000-row top-1, 500-row top-N, chained compute
  • Property-based (2): running sum invariant, row_number 1..n
  • Distributed (2): single worker, local-vs-distributed equivalence

🤖 Generated with Claude Code

Adds composable over() helper for window functions inside mutate:

  Dux.mutate(df,
    rank: over(row_number(), partition_by: :dept, order_by: [desc: :salary]),
    running: over(sum(amount), order_by: :date, frame: {:rows, :unbounded, :current}),
    pct: salary * 100.0 / over(sum(salary), partition_by: :dept)
  )

Options:
  - partition_by: atom, list of atoms, or column expression
  - order_by: atom, or keyword list with :asc/:desc
  - frame: tuple {type, start, end} or raw SQL string

Frame tuple syntax:
  - Type: :rows, :range, or :groups
  - Start: negative int (PRECEDING), :unbounded, :current, or 0
  - End: positive int (FOLLOWING), :unbounded, :current, or 0
  - Optional 4th element: [exclude: :current | :group | :ties | :no_others]

Examples:
  {:rows, -2, :current}              # 3-row moving window
  {:rows, :unbounded, :current}      # cumulative
  {:rows, -2, 2}                     # centered 5-row window
  {:rows, :unbounded, :unbounded}    # entire partition
  {:rows, :unbounded, :unbounded, exclude: :current}  # all except self

Raw SQL strings still work as fallback:
  frame: "ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING"

35 tests (33 unit + 2 property-based) covering all categories:
happy path, sad path, adversarial, scale/wicked, property-based,
distributed, frame tuples, frame strings, exclude clauses.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@cigrainger cigrainger merged commit 20af083 into main Apr 2, 2026
5 checks passed
@cigrainger cigrainger deleted the feat/window-over branch April 2, 2026 05:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant