18 tool issues found across SQL Tools, Core Tools, Schema Tools, and Lineage

[mhallida]

Summary

Ran a comprehensive docs-as-spec test against every tool documented on docs.altimate.sh. Tested 46 tools using the exact examples from the docs plus edge cases. 34 passed, 18 issues found.

A full detailed report with reproduction steps, expected vs actual output, impact analysis, and suggested fixes for every issue is attached below.

---

Priority Breakdown

P1 — Critical (3 issues)

• sql_diff — Reports clearly different queries as "identical" (docs example fails)
• altimate_core_compare — Same comparison bug, likely shared root cause with sql_diff
• sql_fix — Returns the original unfixed SQL instead of applying the fix

P2 — Medium (10 issues)

• sql_rewrite — "No rewritable anti-patterns" on a textbook YEAR() non-sargable predicate
• sql_optimize — Detects anti-patterns but returns unmodified SQL (acts as second sql_analyze)
• sql_translate — 5 Snowflake→BigQuery constructs fail: TRY_TO_NUMBER, ARRAY_AGG WITHIN GROUP, IFF→IF, QUALIFY (invalid subquery wrapping), FLATTEN→UNNEST
• altimate_core_fix — Can't fix SELCT→SELECT keyword typos
• altimate_core_correct — Same fix failure + [object Object] serialization bug in output
• altimate_core_equivalence — = 'x' vs IN ('x') reported as non-equivalent
• altimate_core_track_lineage — Returns empty for multi-query CREATE TABLE AS SELECT lineage
• altimate_core_migration — Misses FLOAT→DECIMAL type narrowing (schema_diff catches it)
• altimate_core_classify_pii — Misses credit_card_number column as PII (0 confidence)
• schema_diff (SQL-level) — Returns empty diff on clearly different DDL ("? → ? columns")

P3 — Low (4 issues)

• sql_format — Outputs single-line SQL instead of multi-line indented format
• altimate_core_semantics — Returns empty for implicit cross joins
• sql_analyze — 2 of 19 documented rules don't fire: WINDOW_WITHOUT_PARTITION, LARGE_IN_LIST
• altimate_core_policy — Rejects valid DML before checking policies

---

What Works Well (28+ tools passing)

sql_analyze, altimate_core_check, altimate_core_column_lineage (simple queries), altimate_core_classify_pii (email/phone/SSN/IP/name/DOB), altimate_core_query_pii, altimate_core_import_ddl, altimate_core_export_ddl, altimate_core_testgen (24 test cases generated), altimate_core_complete, altimate_core_prune_schema, altimate_core_optimize_context, altimate_core_resolve_term, altimate_core_introspection_sql, altimate_core_fingerprint, altimate_core_grade, altimate_core_schema_diff (via schema_context), lineage_check, all Memory tools, all Training tools, all Warehouse tools.

---

Full Report

The complete report with all 18 tickets, reproduction steps, expected vs actual outputs, and suggested fixes is in the attached file: altimate-code-test-tickets.md

Each ticket includes:

• Priority and component
• Exact input used (matching docs examples where possible)
• Expected output (per docs)
• Actual output (verbatim)
• Impact analysis
• Suggested fix

Proposed Approach

Fix all 18 issues in a single PR rather than 18 separate ones, since many share root causes:

• sql_diff + altimate_core_compare likely share comparison logic
• sql_fix + altimate_core_fix + altimate_core_correct all relate to the fix/correct pipeline
• sql_optimize + sql_rewrite both fail at the AST rewrite stage

---

Metadata

Field	Value
CLI Version	0.5.7
Platform	darwin
Architecture	arm64
OS Release	25.3.0
Category	bug
Working Directory	AI-Tools
Session ID	ses_2e4130cb2ffeCqOgttOduwkspe

[mhallida]

Altimate Code - Test Results & Tickets

Test Date: 2026-03-23
Test Methodology: Executed every documented tool from docs.altimate.sh using the exact example inputs from the docs, plus additional edge cases. Compared actual output to documented expected behavior.
Repo: AltimateAI/altimate-code

---

Summary

Category	Tools Tested	Passed	Issues Found
SQL Tools	7	3	7 issues across 4 tools
Core Tools - Analysis	8	5	5 issues across 3 tools
Core Tools - Lineage	4	2	2 issues across 2 tools
Core Tools - PII/Safety	3	2	1 issue
Core Tools - Schema Ops	6	5	1 issue
Core Tools - Context/Gen	4	4	0 issues
Schema Tools (high-level)	3	2	1 issue
Lineage Tools	1	1	1 issue (edge case)
Warehouse Tools	3	3	0 issues
Memory Tools	4	4	0 issues
Training Tools	3	3	0 issues
Total	46	34	18 issues

---

TICKET-001: sql_fix does not actually fix the typo it diagnoses

Priority: P1 - High
Component: SQL Tools > sql_fix
Type: Bug
Docs Reference: SQL Tools - sql_fix

Description

The docs show sql_fix diagnosing a typo in analytics.public.userss and returning fixed SQL with analytics.public.users. In practice, the tool returns the original unfixed SQL as the "Auto-Fixed SQL".

Steps to Reproduce

Input:
  sql: "SELECT * FROM analytics.public.userss"
  error_message: "SQL compilation error: Object 'ANALYTICS.PUBLIC.USERSS' does not exist"
  dialect: snowflake

Expected (per docs):
  Diagnosis: Typo in table name. 'userss' should be 'users'
  Fixed SQL: SELECT * FROM analytics.public.users

Actual:
  === Auto-Fixed SQL ===
  SELECT * FROM analytics.public.userss   <-- UNCHANGED, not fixed

Impact

Users relying on sql_fix to auto-correct SQL errors will get back the same broken SQL, defeating the purpose of the tool. The docs promise auto-correction behavior that doesn't work.

Suggested Fix

The fuzzy-matching logic needs to actually resolve the closest table name match and substitute it. The tool correctly identifies the error context but fails to apply the fix.

---

TICKET-002: sql_diff reports identical queries as "identical" when they clearly differ

Priority: P1 - High
Component: SQL Tools > sql_diff
Type: Bug
Docs Reference: SQL Tools - sql_diff

Description

Using the exact example from the docs, sql_diff returns "The two SQL queries are identical" when they clearly have different columns and WHERE clauses.

Steps to Reproduce

Input:
  original: "SELECT id, name FROM users WHERE active = true"
  modified: "SELECT id, name, email FROM users WHERE active = true AND created_at > '2026-01-01'"

Expected (per docs):
  Changes:
    + Added column: email
    + Added filter: created_at > '2026-01-01'
    Additions: 2 | Deletions: 0 | Modifications: 0

Actual:
  "The two SQL queries are identical."

Impact

This tool is documented as a key part of the code review workflow. If it can't detect differences between two clearly different queries, it's broken at its core function.

Suggested Fix

The comparison engine likely has a normalization step that's too aggressive (possibly stripping differences before comparison, or a bug in the diff algorithm).

---

TICKET-003: sql_rewrite fails to rewrite the exact example from the docs

Priority: P2 - Medium
Component: SQL Tools > sql_rewrite
Type: Bug
Docs Reference: SQL Tools - sql_rewrite

Description

The docs show sql_rewrite converting YEAR(order_date) = 2026 into a range predicate. In practice, the tool returns "No rewritable anti-patterns detected."

Steps to Reproduce

Input:
  sql: "SELECT * FROM orders WHERE YEAR(order_date) = 2026 AND status IN ('shipped', 'delivered') ORDER BY order_date"
  dialect: snowflake

Expected (per docs):
  Rewrites applied:
    1. Predicate pushdown: YEAR(order_date) = 2026 -> range predicate
  Rewritten SQL with order_date >= '2026-01-01' AND order_date < '2027-01-01'

Actual:
  "No rewritable anti-patterns detected."

Impact

The docs list 6 rewrite strategies (predicate pushdown, SELECT pruning, function elimination, JOIN reordering, subquery flattening, UNION ALL promotion). The tool doesn't apply any of them on a textbook case. Meanwhile, sql_optimize and sql_analyze do detect this same pattern as non_sargable_predicate / FUNCTION_IN_FILTER.

Suggested Fix

The rewrite rules need to be connected to the anti-pattern detection. The detection layer works (confirmed via sql_analyze), but the AST transformation layer in sql_rewrite appears disconnected or not implemented for these cases.

---

TICKET-004: sql_optimize detects problems but doesn't actually optimize the SQL

Priority: P2 - Medium
Component: SQL Tools > sql_optimize
Type: Bug
Docs Reference: SQL Tools - sql_optimize

Description

The docs show sql_optimize returning optimized SQL with concrete rewrites (e.g., replacing YEAR() with range predicates, replacing SELECT * with explicit columns). In practice, the "Optimized SQL" is identical to the input.

Steps to Reproduce

Input:
  sql: "SELECT * FROM orders o JOIN customers c ON o.customer_id = c.id WHERE YEAR(o.order_date) = 2026"
  dialect: snowflake

Expected (per docs):
  Optimized SQL with explicit columns, range predicate for YEAR()
  "Estimated improvement: 70-80% reduction in bytes scanned"

Actual:
  === Optimized SQL ===
  SELECT * FROM orders o JOIN customers c ON o.customer_id = c.id WHERE YEAR(o.order_date) = 2026
  (unchanged — just lists anti-patterns without fixing)

Also tested with correlated subquery — same behavior: detects the anti-pattern but doesn't rewrite.

Impact

The tool functions as a second sql_analyze (detection-only) rather than as an optimizer. The docs promise concrete rewritten SQL that would save significant warehouse compute. Users following the docs will expect optimized output and get none.

Suggested Fix

Connect the anti-pattern detection to the AST rewrite engine. The tool already knows what's wrong — it just needs to apply the fixes.

---

TICKET-005: sql_translate fails to translate several Snowflake constructs to BigQuery

Priority: P2 - Medium
Component: SQL Tools > sql_translate
Type: Bug
Docs Reference: SQL Tools - sql_translate

Description

Several Snowflake-specific constructs fail to translate properly to BigQuery:

Issue 5a: TRY_TO_NUMBER not translated

Input (Snowflake): TRY_TO_NUMBER(amount_str) AS amount
Expected (per docs): SAFE_CAST(amount_str AS NUMERIC)
Actual: TRY_TO_NUMBER(amount_str) AS amount  <-- left as Snowflake syntax

Issue 5b: ARRAY_AGG WITHIN GROUP not translated

Input (Snowflake): ARRAY_AGG(tag) WITHIN GROUP (ORDER BY tag)
Expected (per docs): ARRAY_AGG(tag ORDER BY tag)
Actual: ARRAY_AGG(tag) WITHIN GROUP (ORDER BY tag)  <-- left as Snowflake syntax

Issue 5c: IFF() translated to CASE WHEN instead of IF()

Input (Snowflake): IFF(status = 'active', 1, 0)
Expected (per docs): IF(status = 'active', 1, 0)
Actual: CASE WHEN status = 'active' THEN 1 ELSE 0 END

While CASE WHEN is semantically correct, BigQuery natively supports IF() and the docs specifically show IF() as the expected output. This is a quality-of-translation issue.

Issue 5d: QUALIFY converted to subquery wrapping

Input (Snowflake): QUALIFY ROW_NUMBER() OVER (...) = 1
Expected (per docs): QUALIFY ROW_NUMBER() OVER (...) = 1  (BigQuery supports QUALIFY natively)
Actual: SELECT * FROM (...) AS _qualify WHERE ROW_NUMBER() OVER (...) = 1

BigQuery has supported QUALIFY since 2021. The translation unnecessarily wraps in a subquery, and worse, the resulting SQL is invalid because ROW_NUMBER() can't be used directly in a WHERE clause without a window alias.

Issue 5e: FLATTEN() not translated to BigQuery UNNEST()

Input (Snowflake): FLATTEN(input => tags) AS f
Expected: UNNEST(tags) AS f  (or similar BigQuery array expansion)
Actual: FLATTEN(input => tags) AS f  <-- left as Snowflake syntax

Impact

SQL translation is a core selling point ("Cross-Dialect Translation" is highlighted on the homepage). Multiple untranslated constructs in a single test query means users migrating from Snowflake to BigQuery will get SQL that doesn't run.

Suggested Fix

Add translation rules for: TRY_TO_NUMBER -> SAFE_CAST, ARRAY_AGG WITHIN GROUP -> ARRAY_AGG ... ORDER BY, IFF -> IF, QUALIFY -> native (not subquery wrapping for BQ), FLATTEN -> UNNEST.

---

TICKET-006: sql_format does not produce multi-line formatted output

Priority: P3 - Low
Component: SQL Tools > sql_format
Type: Bug
Docs Reference: SQL Tools - sql_format

Description

The docs show sql_format producing beautifully indented, multi-line SQL. In practice, the output is single-line with no line breaks.

Steps to Reproduce

Input:
  sql: "select a.id,b.name,count(*) as cnt from table_a a join table_b b on a.id=b.a_id where a.status='active' group by 1,2 order by 3 desc"
  dialect: snowflake, indent: 2

Expected (per docs):
  SELECT
    a.id,
    b.name,
    COUNT(*) AS cnt
  FROM table_a a
  JOIN table_b b
    ON a.id = b.a_id
  WHERE a.status = 'active'
  GROUP BY 1, 2
  ORDER BY 3 DESC

Actual:
  SELECT a.id, b.name, COUNT(*) AS cnt FROM table_a AS a JOIN table_b AS b ON a.id = b.a_id WHERE a.status = 'active' GROUP BY 1, 2 ORDER BY 3 DESC

The output is cleaner (adds spaces, uppercases keywords, adds AS for aliases) but it's all on one line. The indent parameter appears to have no effect.

Impact

SQL formatting is a basic hygiene tool. Single-line SQL output defeats the purpose of a formatter. The docs set expectations for multi-line, indented output that users will want.

Suggested Fix

The formatter likely uses sqlglot's pretty option. Ensure pretty=True is passed along with the indent parameter.

---

TICKET-007: altimate_core_fix cannot fix obvious syntax errors

Priority: P2 - Medium
Component: Core Tools > altimate_core_fix
Type: Bug
Docs Reference: Core Tools - altimate_core_fix

Description

The docs describe altimate_core_fix as using "fuzzy matching and iterative re-validation to correct syntax errors, typos, and schema reference issues." In practice, it cannot fix SELCT -> SELECT or WERE -> WHERE.

Steps to Reproduce

Input:
  sql: "SELCT id, namee FROM userss WERE active = true"

Expected: Fixed SQL with corrected keywords

Actual:
  "Could not auto-fix the SQL. Errors found: No automatic fix available for E000"

Impact

This is a fundamental keyword typo that any SQL auto-fixer should handle. The tool name is literally "fix" but it can't correct basic misspellings.

---

TICKET-008: altimate_core_correct also cannot fix obvious syntax errors

Priority: P2 - Medium
Component: Core Tools > altimate_core_correct
Type: Bug
Docs Reference: Core Tools - altimate_core_correct

Description

Described as "more thorough than fix" with "multiple correction rounds," altimate_core_correct also fails on the same basic input. Additionally, the output format is poor — it shows Iterations: [object Object] instead of meaningful iteration details.

Steps to Reproduce

Input:
  sql: "SELCT id, namee FROM userss WERE active = true"

Expected: Corrected SQL after multiple iterations

Actual:
  "Iterations: [object Object]
   Could not correct the SQL after maximum iterations."

Impact

1. Same fix failure as altimate_core_fix — can't handle basic keyword typos
2. [object Object] in output indicates a serialization bug (JS object not stringified)

Suggested Fix

1. Add keyword fuzzy matching (SELCT -> SELECT, WERE -> WHERE, etc.)
2. Fix the [object Object] serialization issue — likely needs JSON.stringify() or proper formatting

---

TICKET-009: altimate_core_compare reports structurally different queries as "identical"

Priority: P1 - High
Component: Core Tools > altimate_core_compare
Type: Bug
Docs Reference: Core Tools - altimate_core_compare

Description

The docs say this tool "identifies differences in table references, join conditions, filters, projections, and aggregations." In practice, it reports queries with different projections and filters as "structurally identical."

Steps to Reproduce

Input:
  left_sql:  "SELECT id, name FROM users WHERE active = true ORDER BY name"
  right_sql: "SELECT id, name, email FROM users WHERE active = true AND created_at > '2026-01-01' ORDER BY name"

Expected: Differences reported:
  + Added projection: email
  + Added filter: created_at > '2026-01-01'

Actual:
  "Queries are structurally identical."

Impact

This is the same fundamental issue as TICKET-002 (sql_diff). The comparison/diff engine has a normalization bug that ignores real structural differences. This affects the entire diff/compare pipeline.

Root Cause Hypothesis

Both sql_diff and altimate_core_compare likely share the same underlying comparison logic, and it's either (a) normalizing queries too aggressively before comparison, or (b) only comparing table references/FROM clauses and ignoring SELECT/WHERE changes.

---

TICKET-010: altimate_core_equivalence reports equivalent queries as "different"

Priority: P2 - Medium
Component: Core Tools > altimate_core_equivalence
Type: Bug
Docs Reference: Core Tools - altimate_core_equivalence

Description

WHERE status = 'active' and WHERE status IN ('active') are semantically equivalent, but the tool reports them as producing different results.

Steps to Reproduce

Input:
  sql1: "SELECT id FROM users WHERE status = 'active'"
  sql2: "SELECT id FROM users WHERE status IN ('active')"

Expected: Queries produce the same result set (equivalent)

Actual: "Queries produce different results."

Impact

This tool is critical for the /query-optimize skill, which uses equivalence checking to "verify rewrites preserve semantics." If it can't recognize that = 'x' and IN ('x') are equivalent, it will falsely flag valid optimizations as breaking.

---

TICKET-011: altimate_core_track_lineage returns empty results for multi-query lineage

Priority: P2 - Medium
Component: Core Tools > altimate_core_track_lineage
Type: Bug
Docs Reference: Core Tools - altimate_core_track_lineage

Description

The docs describe this tool as building "a combined lineage graph from a sequence of queries." In practice, it returns no lineage edges.

Steps to Reproduce

Input:
  queries: [
    "CREATE TABLE staging AS SELECT id, name, email FROM raw_users",
    "CREATE TABLE enriched AS SELECT s.id, s.name, s.email, p.plan_name FROM staging s JOIN plans p ON s.plan_id = p.id"
  ]

Expected: Lineage showing raw_users -> staging -> enriched column flow

Actual: "No lineage edges found across queries."

Impact

Multi-query lineage tracking is essential for understanding data pipeline flows. If it can't trace across CREATE TABLE ... AS SELECT statements, it misses a major use case (ETL pipeline lineage).

---

TICKET-012: altimate_core_semantics returns empty results for implicit cross join

Priority: P3 - Low
Component: Core Tools > altimate_core_semantics
Type: Bug
Docs Reference: Core Tools - altimate_core_semantics

Description

The docs describe this tool as detecting "logical issues like cartesian products, wrong JOIN conditions, NULL misuse, and type mismatches." It returns no findings on a query with a comma-join and a WHERE clause that could indicate a missing join condition.

Steps to Reproduce

Input:
  sql: "SELECT * FROM orders o, customers c WHERE o.customer_id = c.id AND o.amount > 0"

Expected: At least some semantic analysis (implicit join pattern noted)

Actual: "Semantic issues: " (empty)

Impact

Low priority since sql_analyze does catch the implicit cross join pattern. But the tool-specific docs imply deeper semantic analysis that isn't delivered.

---

TICKET-013: altimate_core_migration misses FLOAT -> DECIMAL type narrowing risk

Priority: P2 - Medium
Component: Core Tools > altimate_core_migration
Type: Bug
Docs Reference: Core Tools - altimate_core_migration

Description

altimate_core_schema_diff correctly flags the FLOAT -> DECIMAL(12,2) type change as a BREAKING change. However, altimate_core_migration on the same DDLs reports "Migration appears safe. No risks detected."

Steps to Reproduce

Input:
  old_ddl: "CREATE TABLE orders (id INT, amount FLOAT, status TEXT)"
  new_ddl: "CREATE TABLE orders (id INT, amount DECIMAL(12,2), status TEXT, created_at TIMESTAMP)"

altimate_core_schema_diff result:
  [BREAKING] Column 'orders.amount' type changed: FLOAT -> DECIMAL(12,2)

altimate_core_migration result:
  "Migration appears safe. No risks detected."

Impact

FLOAT -> DECIMAL(12,2) is a type narrowing that can cause data loss (FLOAT has higher precision than DECIMAL(12,2) for large values). The schema diff tool catches it but the migration safety tool misses it. Users relying on migration safety analysis before applying DDL will get a false "safe" signal.

Suggested Fix

The migration tool should flag type narrowing (FLOAT -> DECIMAL, VARCHAR(255) -> VARCHAR(50), etc.) as potential data loss risks, consistent with what schema_diff already detects.

---

TICKET-014: altimate_core_column_lineage misses user-sourced columns in complex CTEs

Priority: P3 - Low
Component: Core Tools > altimate_core_column_lineage
Type: Bug

Description

In a multi-CTE query with users and orders tables, the lineage only traces back to orders columns, completely missing the users.id and users.name columns that flow through CTEs to the output.

Steps to Reproduce

Input SQL:
  WITH active_users AS (
    SELECT id, name, email FROM users WHERE active = true
  ),
  user_orders AS (
    SELECT u.id, u.name, COUNT(o.id) as order_count, SUM(o.amount) as total_spent
    FROM active_users u LEFT JOIN orders o ON u.id = o.user_id
    GROUP BY u.id, u.name
  )
  SELECT id, name, order_count, total_spent,
    CASE WHEN total_spent > 1000 THEN 'VIP' ELSE 'Regular' END as tier
  FROM user_orders

Expected: Lineage for id <- users.id, name <- users.name, order_count <- orders.id, total_spent <- orders.amount, tier <- orders.amount

Actual:
  ORDER_COUNT <- "ORDERS"."ID"
  TIER <- "ORDERS"."AMOUNT"
  TOTAL_SPENT <- "ORDERS"."AMOUNT"
  (users.id and users.name completely missing)
  All marked as "Not sure" confidence

Impact

The homepage claims "100% edge match across 500 queries with complex joins, CTEs, and subqueries." This test shows CTE lineage is incomplete for multi-hop flows. The id and name columns flow through active_users CTE -> user_orders CTE -> output, but lineage doesn't trace them.

---

TICKET-015: altimate_core_classify_pii misses credit_card_number column

Priority: P2 - Medium
Component: Core Tools > altimate_core_classify_pii
Type: Bug
Docs Reference: Schema Tools - schema_detect_pii

Description

The PII classifier misses credit_card_number as PII. The docs list "credit card" as a detection category.

Steps to Reproduce

Input schema:
  users.credit_card_number (VARCHAR) -> None (0 confidence)
  users.card_last_four (VARCHAR) -> None (0 confidence)
  users.card_expiry (VARCHAR) -> None (0 confidence)

All three are clearly credit-card-related PII but all return 0 confidence.

Contrast with the first PII test:

  users.email -> Email (0.95) PASS
  users.phone_number -> Phone (0.95) PASS
  users.ssn -> SSN (0.98) PASS
  users.ip_address -> IPAddress (0.8) PASS
  users.first_name -> Name (0.85) PASS
  users.date_of_birth -> DateOfBirth (0.9) PASS

Impact

The docs and homepage both list credit card detection as a supported PII category. False negatives on PII detection can have compliance (PCI-DSS) implications.

Suggested Fix

Add patterns for credit card columns: credit_card, card_number, cc_number, card_last_four, card_expiry, cvv, card_holder.

---

TICKET-016: schema_diff (SQL-level tool) returns empty diff for clearly different schemas

Priority: P2 - Medium
Component: Schema Tools > schema_diff
Type: Bug
Docs Reference: Schema Tools - schema_diff

Description

The SQL-level schema_diff tool (distinct from altimate_core_schema_diff) returns an empty comparison when given DDL with clear changes.

Steps to Reproduce

Input:
  old_sql: "CREATE TABLE orders (id INT, amount FLOAT, status TEXT)"
  new_sql: "CREATE TABLE orders (id INT, amount DECIMAL(12,2), status TEXT, created_at TIMESTAMP)"
  dialect: snowflake

Expected (per docs):
  ~ Modified: amount (FLOAT -> DECIMAL(12,2)), severity: medium
  + Added: created_at (TIMESTAMP), severity: low

Actual:
  "Schema comparison: ? -> ? columns
   Dropped: 0 | Added: 0 | Type Changed: 0 | Renamed: 0"

Note: altimate_core_schema_diff (the Rust-based tool) correctly detects these changes using schema_context input. The issue is with the SQL-level schema_diff tool that takes DDL strings.

Impact

The docs show this as a straightforward DDL comparison tool. The "? -> ? columns" in the output suggests the DDL parsing is failing silently.

---

TICKET-017: sql_analyze missing documented rules: WINDOW_WITHOUT_PARTITION and LARGE_IN_LIST

Priority: P3 - Low
Component: SQL Tools > sql_analyze
Type: Bug
Docs Reference: SQL Tools - sql_analyze

Description

Two of the 19 documented rules don't appear to fire when tested with appropriate inputs:

17a: WINDOW_WITHOUT_PARTITION not detected

Input: "SELECT SUM(amount) OVER () as running_total FROM orders"
Expected: Warning about window function without PARTITION BY
Actual: Only "SELECT without LIMIT" detected. No WINDOW_WITHOUT_PARTITION.

17b: LARGE_IN_LIST not detected

Input: "SELECT id FROM users WHERE id IN (1,2,3,...,21)"  (21 values)
Expected: Info about large IN list
Actual: Only "SELECT without LIMIT" detected. No LARGE_IN_LIST.

Rules that DO work (confirmed):

• SELECT_STAR: PASS
• SELECT_STAR_IN_SUBQUERY: PASS
• CARTESIAN_PRODUCT: PASS
• IMPLICIT_CARTESIAN: PASS
• NOT_IN_WITH_SUBQUERY: PASS
• FUNCTION_IN_FILTER: PASS
• FUNCTION_IN_JOIN: PASS
• LIKE_LEADING_WILDCARD: PASS
• UNUSED_CTE: PASS
• ORDER_BY_IN_SUBQUERY: PASS
• ORDER_BY_WITHOUT_LIMIT: PASS (detected as ORDER_BY_IN_SUBQUERY)
• MISSING_LIMIT: PASS (but overly aggressive - fires on every query)

Impact

The docs claim 19 rules with "100% accuracy across 1,077 benchmark queries." Two rules appear non-functional, which means 17/19 confirmed (89%).

---

TICKET-018: altimate_core_policy rejects valid SQL before checking policies

Priority: P3 - Low
Component: Core Tools > altimate_core_policy
Type: Bug
Docs Reference: Core Tools - altimate_core_policy

Description

The policy checker rejects valid SQL with a validation error before even checking the policy rules.

Steps to Reproduce

Input:
  sql: "DELETE FROM users WHERE id = 1"
  policy_json: '{"rules": [{"type": "forbidden_operations", "operations": ["DELETE","DROP","TRUNCATE"]}]}'

Expected: Policy violation: "DELETE is a forbidden operation"

Actual:
  "[error] validation_failed: Query failed validation and cannot be checked against policies"

The SQL is syntactically valid. The tool should evaluate it against the policy (and report the DELETE violation), not reject it at validation.

Impact

Policy governance is a key feature for enterprise users. If the tool can't evaluate DML statements against policies, the forbidden_operations policy type is useless.

Suggested Fix

DML statements (DELETE, UPDATE, INSERT, MERGE) should pass validation even without schema context, since the policy check is the whole point.

---

Overall Assessment

What Works Well

• sql_analyze: Solid anti-pattern detection. Caught 15+ of 19 documented rules in testing. Consistent, reliable outputs.
• altimate_core_check: Full pipeline (validate + lint + safety + PII) works as documented.
• altimate_core_column_lineage: Excellent for simple to moderate queries. The lineage output format with lens types (Original, Alias, Transformation) is informative.
• altimate_core_classify_pii: Strong detection for email, phone, SSN, IP, name, and DOB. Confidence scores are well-calibrated.
• altimate_core_query_pii: Correctly identifies PII exposure in queries and provides masking suggestions.
• altimate_core_import_ddl / export_ddl: Clean round-trip between DDL and YAML schema format.
• altimate_core_testgen: Impressive - generates 24 well-categorized test cases covering boundary values, NULL handling, type edges, and empty tables.
• altimate_core_complete: Schema-aware autocomplete works correctly.
• altimate_core_prune_schema: Correctly prunes irrelevant tables from schema context.
• altimate_core_optimize_context: Progressive disclosure with compression ratios works well.
• altimate_core_resolve_term: Fuzzy matching against schema for business terms works.
• altimate_core_introspection_sql: Generates correct INFORMATION_SCHEMA queries.
• altimate_core_fingerprint: Deterministic SHA-256 fingerprinting works.
• Memory Tools: Full write/read/delete/audit cycle works perfectly. Markdown format is clean.
• Training Tools: Save/list/remove cycle works. Applied counts and budget tracking work.
• Warehouse Tools: warehouse_list, warehouse_discover, project_scan all work as documented.
• altimate_core_grade: Grading works with category breakdowns.
• altimate_core_schema_diff (via schema_context): Correctly detects breaking changes.

Top Priorities for Fix

1. P1: sql_diff + altimate_core_compare — Comparison is fundamentally broken (TICKET-002, TICKET-009)
2. P1: sql_fix — Core auto-fix doesn't fix anything (TICKET-001)
3. P2: sql_optimize + sql_rewrite — Detection works but rewriting doesn't (TICKET-003, TICKET-004)
4. P2: sql_translate — Multiple Snowflake constructs untranslated for BigQuery (TICKET-005)
5. P2: altimate_core_fix + altimate_core_correct — Can't fix basic keyword typos (TICKET-007, TICKET-008)