Skip to content

Comments

feat(Duckdb): Add transpilation support for ARRAY_GENERATE_RANGE function#7107

Open
fivetran-amrutabhimsenayachit wants to merge 1 commit intomainfrom
RD-1147618-array-generate-range
Open

feat(Duckdb): Add transpilation support for ARRAY_GENERATE_RANGE function#7107
fivetran-amrutabhimsenayachit wants to merge 1 commit intomainfrom
RD-1147618-array-generate-range

Conversation

@fivetran-amrutabhimsenayachit
Copy link
Collaborator

When ARRAY_GENERATE_RANGE has a negative step value, the current transpilation produces incorrect results.
For example:

Snowflake: ARRAY_GENERATE_RANGE(-5, -25, -10) -> [-5, -15]
Current DuckDB: GENERATE_SERIES(-5, -25 - 1, -10) -> [-5, -15, -25] (WRONG)

Another example:

Snowflake: ARRAY_GENERATE_RANGE(5, 1, -1) -> [5, 4, 3, 2]
Current DuckDB: GENERATE_SERIES(5, 1 - 1, -1) -> [5, 4, 3, 2, 1, 0] (WRONG)

Fix:
Added pattern detection in the generateseries_sql function in. DuckDB:

  • Detects when end argument is Sub(stop, 1) (Snowflake's pattern)
  • Extracts the original stop value and uses RANGE(start, stop, step) instead
  • RANGE has exclusive-end semantics matching Snowflake perfectly

DuckDb after transpilation:

"SELECT RANGE(2, 5) AS basic_range, RANGE(5, 25, 10) AS with_step, RANGE(-5, -25, -10) AS negative_step, RANGE(10, 10) AS empty_start_equals_stop, RANGE(0, 10, 2) AS even_numbers, RANGE(NULL, 5) AS null_start, RANGE(1, NULL) AS null_stop, RANGE(1, 5, NULL) AS null_step, RANGE(5, 1, -1) AS descending, RANGE(CAST(1.5 AS INT), CAST(5.8 AS INT)) AS from_float, RANGE(10, 5) AS empty_wrong_direction_positive, RANGE(-10, -5, -1) AS empty_wrong_direction_negative, RANGE(-5, 5, 2) AS negative_to_positive, RANGE(0, -10, -3) AS zero_to_negative"
┌─────────────┬───────────┬───────────────┬─────────────────────────┬─────────────────┬────────────┬───────────┬───────────┬──────────────┬──────────────┬────────────────────────────────┬────────────────────────────────┬──────────────────────┬──────────────────┐
│ basic_range │ with_step │ negative_step │ empty_start_equals_stop │  even_numbers   │ null_start │ null_stop │ null_step │  descending  │  from_float  │ empty_wrong_direction_positive │ empty_wrong_direction_negative │ negative_to_positive │ zero_to_negative │
│   int64[]   │  int64[]  │    int64[]    │         int64[]         │     int64[]     │  int64[]   │  int64[]  │  int64[]  │   int64[]    │   int64[]    │            int64[]             │            int64[]             │       int64[]        │     int64[]      │
├─────────────┼───────────┼───────────────┼─────────────────────────┼─────────────────┼────────────┼───────────┼───────────┼──────────────┼──────────────┼────────────────────────────────┼────────────────────────────────┼──────────────────────┼──────────────────┤
│ [2, 3, 4]   │ [5, 15]   │ [-5, -15]     │ []                      │ [0, 2, 4, 6, 8] │ NULL       │ NULL      │ NULL      │ [5, 4, 3, 2] │ [2, 3, 4, 5] │ []                             │ []                             │ [-5, -3, -1, 1, 3]   │ [0, -3, -6, -9]  │
└─────────────┴───────────┴───────────────┴─────────────────────────┴─────────────────┴────────────┴───────────┴───────────┴──────────────┴──────────────┴────────────────────────────────┴────────────────────────────────┴──────────────────────┴──────────────────┘

@github-actions
Copy link
Contributor

SQLGlot Integration Test Results

Comparing:

  • this branch (sqlglot:RD-1147618-array-generate-range, sqlglot version: RD-1147618-array-generate-range)
  • baseline (main, sqlglot version: 28.10.1.dev116)

⚠️ Limited to dialects: duckdb, snowflake

By Dialect

dialect main sqlglot:RD-1147618-array-generate-range transitions links
duckdb -> duckdb 4003/4003 passed (100.0%) 4003/4003 passed (100.0%) No change full result / delta
snowflake -> duckdb 1579/2483 passed (63.6%) 1579/2483 passed (63.6%) No change full result / delta
snowflake -> snowflake 2703/2703 passed (100.0%) 2703/2703 passed (100.0%) No change full result / delta

Overall

main: 9189 total, 8285 passed (pass rate: 90.2%), sqlglot version: 28.10.1.dev116

sqlglot:RD-1147618-array-generate-range: 9189 total, 8285 passed (pass rate: 90.2%), sqlglot version: RD-1147618-array-generate-range

Transitions:
No change

Comment on lines +3131 to +3132
if isinstance(end, exp.Sub) and isinstance(end.expression, exp.Literal):
if end.expression.this == "1":
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we hard-coding checks for literals? This isn't a good signal. What are you trying to check here? Should we be using an explicit arg instead?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants