chore: bump sqlglot pin to sqlglot[c]==30.7.0#101
Merged
suryaiyer95 merged 1 commit intomainfrom May 7, 2026
Merged
Conversation
Match the version being adopted in altimate-dags PR #1024 so all downstream consumers (`altimate-dags`, airflow workers via `altimate-datapilot-cli`) resolve to the same C-accelerated sqlglot release. The 6.8x parsing speedup landing in altimate-dags benefits any DAG path that reaches into datapilot-cli's `SqlCheck`. `SqlCheck` (`src/datapilot/core/platforms/dbt/insights/sql/sql_check.py`) is the only place that imports `sqlglot`. Its optimizer-rule pipeline (`pushdown_projections`, `normalize`, `unnest_subqueries`, `eliminate_subqueries`, `eliminate_joins`, `eliminate_ctes`) inspects each rule via `inspect.getfullargspec(rule).args`. Verified that sqlglot[c] 30.7.0 still exposes proper signatures via mypyc — no compat shim required. Non-breaking validation: - Existing test suite: **77/77 pass** on both 25.30.0 (old pin) and 30.7.0 (new pin). - SqlCheck output is byte-identical between 25.30.0 and 30.7.0 on a 7-query fixture covering filter pushdown, CTE chains, unused joins, IN-subqueries, DISTINCT dedup, OR/AND filters, and a no-op query. Same 6 insights, same recommended optimized SQL, same rule names. Refs: AltimateAI/altimate-dags#1024
mdesmet
approved these changes
May 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bump the
sqlglotinstall requirement fromsqlglot~=25.30.0tosqlglot[c]==30.7.0to align with the pin being adopted inAltimateAI/altimate-dags#1024 (sqlglot[c] v30 — 6.8x parsing speedup).
SqlCheck(src/datapilot/core/platforms/dbt/insights/sql/sql_check.py)is the only place in datapilot-cli that imports
sqlglot. It uses theoptimizer rules (
pushdown_projections,normalize,unnest_subqueries,eliminate_subqueries,eliminate_joins,eliminate_ctes) and inspectseach rule's signature via
inspect.getfullargspec(rule).args. Verifiedthat sqlglot[c] 30.7.0 still exposes proper signatures via mypyc — no
compat shim is needed.
Non-breaking validation
pytest tests/inspect.getfullargspecon optimizer rulesThe 7-query fixture covered filter pushdown, CTE chains, unused joins,
IN-subqueries,
SELECT DISTINCTdedup, complex OR/AND filters, and ano-optimization-needed baseline. Output recommendations are character-
identical between versions.
Why exact pin (
==30.7.0) and not a rangeMatches the version validated in altimate-dags#1024 against 1,986,940
production queries (and re-validated today on 214,259 rakuten queries:
99.9991 % byte-identical SQL fingerprints, 0 false negatives, 0 false
positives). Holding the pin tight ensures DAG workers and the
datapilot-cli installed alongside them resolve to the same sqlglot
release; we can relax to a range later if/when 30.x minor bumps are
similarly validated.
Refs: AltimateAI/altimate-dags#1024
Test plan
pinned in altimate-infra airflow
extraPipPackages(
apps/{prod,staging/airflow-mi,freemium}/airflow/values.yml).🤖 Generated with Claude Code