Skip to content

[WIP] Add preliminary Clickhouse formatter #922

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mattbasta wants to merge 10 commits into
base: master
Choose a base branch
from
Open

[WIP] Add preliminary Clickhouse formatter #922

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
80f6e1e
Add preliminary Clickhouse formatter
mattbasta Nov 14, 2025
d457a98
Get formatter working with existing features
mattbasta Nov 17, 2025
510997c
Checkpoint tests
mattbasta Nov 18, 2025
be7a20d
Breaking tests for consistency within Clickhouse formatter
mattbasta Nov 19, 2025
7c4f798
Finish getting the tests together
mattbasta Dec 5, 2025
1c19d1a
Cleanup
mattbasta Dec 5, 2025
28dd0f0
Cleanup from review feedback, WIP
mattbasta Dec 10, 2025
ae7ee87
Clean up keyword vs operator for in/any
mattbasta Dec 10, 2025
c4bef88
Reduce intersection between functions and keywords
mattbasta Dec 10, 2025
b912e09
Remove special casing for in()
mattbasta Dec 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,6 @@ We provide **JSON Schema** for `.sql-formatter.json` configuration file, enablin
- [Using the schema in VSCode](https://code.visualstudio.com/docs/languages/json#_mapping-in-the-user-settings)
- [Using the schema in Zed](https://zed.dev/docs/languages/json#schema-specification-via-settings)


### Usage as ESLint plugin

- Inside `eslint-plugin-sql` by using the rule [eslint-plugin-sql#format](https://github.com/gajus/eslint-plugin-sql#format).
Expand Down
1 change: 1 addition & 0 deletions src/allDialects.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
export { bigquery } from './languages/bigquery/bigquery.formatter.js';
export { clickhouse } from './languages/clickhouse/clickhouse.formatter.js';
export { db2 } from './languages/db2/db2.formatter.js';
export { db2i } from './languages/db2i/db2i.formatter.js';
export { duckdb } from './languages/duckdb/duckdb.formatter.js';
Expand Down
1 change: 1 addition & 0 deletions src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ export { ConfigError } from './validateConfig.js';

// When adding a new dialect, be sure to add it to the list of exports below.
export { bigquery } from './languages/bigquery/bigquery.formatter.js';
export { clickhouse } from './languages/clickhouse/clickhouse.formatter.js';
export { db2 } from './languages/db2/db2.formatter.js';
export { db2i } from './languages/db2i/db2i.formatter.js';
export { duckdb } from './languages/duckdb/duckdb.formatter.js';
Expand Down
332 changes: 332 additions & 0 deletions src/languages/clickhouse/clickhouse.formatter.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,332 @@
import { DialectOptions } from '../../dialect.js';
import { expandPhrases } from '../../expandPhrases.js';
import { EOF_TOKEN, isToken, Token, TokenType } from '../../lexer/token.js';
import { functions } from './clickhouse.functions.js';
import { dataTypes, keywords } from './clickhouse.keywords.js';

const reservedSelect = expandPhrases([
'SELECT [DISTINCT]',
// https://clickhouse.com/docs/sql-reference/statements/alter/view
'MODIFY QUERY SELECT [DISTINCT]',
]);

const reservedClauses = expandPhrases([
'SET',
// https://clickhouse.com/docs/sql-reference/statements/select
'WITH',
'FROM',
'SAMPLE',
'PREWHERE',
'WHERE',
'GROUP BY',
'HAVING',
'QUALIFY',
'ORDER BY',
'LIMIT', // Note: Clickhouse has no OFFSET clause
'SETTINGS',
'INTO OUTFILE',
'FORMAT',
// https://clickhouse.com/docs/sql-reference/window-functions
'WINDOW',
'PARTITION BY',
// https://clickhouse.com/docs/sql-reference/statements/insert-into
'INSERT INTO',
'VALUES',
// https://clickhouse.com/docs/sql-reference/statements/create/view#refreshable-materialized-view
'DEPENDS ON',
// https://clickhouse.com/docs/sql-reference/statements/move
'MOVE {USER | ROLE | QUOTA | SETTINGS PROFILE | ROW POLICY}',
// https://clickhouse.com/docs/sql-reference/statements/grant
'GRANT',
// https://clickhouse.com/docs/sql-reference/statements/revoke
'REVOKE',
// https://clickhouse.com/docs/sql-reference/statements/check-grant
'CHECK GRANT',
// https://clickhouse.com/docs/sql-reference/statements/set-role
'SET [DEFAULT] ROLE [NONE | ALL | ALL EXCEPT]',
// https://clickhouse.com/docs/sql-reference/statements/optimize
'DEDUPLICATE BY',
// https://clickhouse.com/docs/sql-reference/statements/alter/statistics
'MODIFY STATISTICS',
// Used for ALTER INDEX ... TYPE and ALTER STATISTICS ... TYPE
'TYPE',
// https://clickhouse.com/docs/sql-reference/statements/alter
'ALTER USER [IF EXISTS]',
'ALTER [ROW] POLICY [IF EXISTS]',
// https://clickhouse.com/docs/sql-reference/statements/drop
'DROP {USER | ROLE | QUOTA | PROFILE | SETTINGS PROFILE | ROW POLICY | POLICY} [IF EXISTS]',
]);

const standardOnelineClauses = expandPhrases([
// https://clickhouse.com/docs/sql-reference/statements/create
'CREATE [OR REPLACE] [TEMPORARY] TABLE [IF NOT EXISTS]',
]);
const tabularOnelineClauses = expandPhrases([
'ALL EXCEPT',
'ON CLUSTER',
// https://clickhouse.com/docs/sql-reference/statements/update
'UPDATE',
// https://clickhouse.com/docs/sql-reference/statements/system
'SYSTEM RELOAD {DICTIONARIES | DICTIONARY | FUNCTIONS | FUNCTION | ASYNCHRONOUS METRICS}',
'SYSTEM DROP {DNS CACHE | MARK CACHE | ICEBERG METADATA CACHE | TEXT INDEX DICTIONARY CACHE | TEXT INDEX HEADER CACHE | TEXT INDEX POSTINGS CACHE | REPLICA | DATABASE REPLICA | UNCOMPRESSED CACHE | COMPILED EXPRESSION CACHE | QUERY CONDITION CACHE | QUERY CACHE | FORMAT SCHEMA CACHE | FILESYSTEM CACHE}',
'SYSTEM FLUSH LOGS',
'SYSTEM RELOAD {CONFIG | USERS}',
'SYSTEM SHUTDOWN',
'SYSTEM KILL',
'SYSTEM FLUSH DISTRIBUTED',
'SYSTEM START DISTRIBUTED SENDS',
'SYSTEM {STOP | START} {LISTEN | MERGES | TTL MERGES | MOVES | FETCHES | REPLICATED SENDS | REPLICATION QUEUES | PULLING REPLICATION LOG}',
'SYSTEM {SYNC | RESTART | RESTORE} REPLICA',
'SYSTEM {SYNC | RESTORE} DATABASE REPLICA',
'SYSTEM RESTART REPLICAS',
'SYSTEM UNFREEZE',
'SYSTEM WAIT LOADING PARTS',
'SYSTEM {LOAD | UNLOAD} PRIMARY KEY',
'SYSTEM {STOP | START} [REPLICATED] VIEW',
'SYSTEM {STOP | START} VIEWS',
'SYSTEM {REFRESH | CANCEL | WAIT} VIEW',
'WITH NAME',
// https://clickhouse.com/docs/sql-reference/statements/show
'SHOW [CREATE] {TABLE | TEMPORARY TABLE | DICTIONARY | VIEW | DATABASE}',
'SHOW DATABASES [[NOT] {LIKE | ILIKE}]',
'SHOW [FULL] [TEMPORARY] TABLES [FROM | IN]',
'SHOW [EXTENDED] [FULL] COLUMNS {FROM | IN}',
// https://clickhouse.com/docs/sql-reference/statements/attach
'ATTACH {TABLE | DICTIONARY | DATABASE} [IF NOT EXISTS]',
// https://clickhouse.com/docs/sql-reference/statements/detach
'DETACH {TABLE | DICTIONARY | DATABASE} [IF EXISTS]',
'PERMANENTLY',
'SYNC',
// https://clickhouse.com/docs/sql-reference/statements/drop
'DROP {DICTIONARY | DATABASE | PROFILE | VIEW | FUNCTION | NAMED COLLECTION} [IF EXISTS]',
'DROP [TEMPORARY] TABLE [IF EXISTS] [IF EMPTY]',
// https://clickhouse.com/docs/sql-reference/statements/alter/table#rename
'RENAME TO',
// https://clickhouse.com/docs/sql-reference/statements/exists
'EXISTS [TEMPORARY] {TABLE | DICTIONARY | DATABASE}',
// https://clickhouse.com/docs/sql-reference/statements/kill
'KILL QUERY',
// https://clickhouse.com/docs/sql-reference/statements/optimize
'OPTIMIZE TABLE',
// https://clickhouse.com/docs/sql-reference/statements/rename
'RENAME {TABLE | DICTIONARY | DATABASE}',
// https://clickhouse.com/docs/sql-reference/statements/exchange
'EXCHANGE {TABLES | DICTIONARIES}',
// https://clickhouse.com/docs/sql-reference/statements/truncate
'TRUNCATE TABLE [IF EXISTS]',
// https://clickhouse.com/docs/sql-reference/statements/execute_as
'EXECUTE AS',
// https://clickhouse.com/docs/sql-reference/statements/use
'USE',
'TO',
// https://clickhouse.com/docs/sql-reference/statements/undrop
'UNDROP TABLE',
// https://clickhouse.com/docs/sql-reference/statements/create
'CREATE {DATABASE | NAMED COLLECTION} [IF NOT EXISTS]',
'CREATE [OR REPLACE] {VIEW | DICTIONARY} [IF NOT EXISTS]',
'CREATE MATERIALIZED VIEW [IF NOT EXISTS]',
'CREATE FUNCTION',
'CREATE {USER | ROLE | QUOTA | SETTINGS PROFILE} [IF NOT EXISTS | OR REPLACE]',
'CREATE [ROW] POLICY [IF NOT EXISTS | OR REPLACE]',
// https://clickhouse.com/docs/sql-reference/statements/create/table#replace-table
'REPLACE [TEMPORARY] TABLE [IF NOT EXISTS]',
// https://clickhouse.com/docs/sql-reference/statements/alter
'ALTER {ROLE | QUOTA | SETTINGS PROFILE} [IF EXISTS]',
'ALTER [TEMPORARY] TABLE',
'ALTER NAMED COLLECTION [IF EXISTS]',
// https://clickhouse.com/docs/sql-reference/statements/alter/user
'GRANTEES',
'NOT IDENTIFIED',
'RESET AUTHENTICATION METHODS TO NEW',
'{IDENTIFIED | ADD IDENTIFIED} [WITH | BY]',
'[ADD | DROP] HOST {LOCAL | NAME | REGEXP | IP | LIKE}',
'VALID UNTIL',
'DROP [ALL] {PROFILES | SETTINGS}',
'{ADD | MODIFY} SETTINGS',
'ADD PROFILES',
// https://clickhouse.com/docs/sql-reference/statements/alter/apply-deleted-mask
'APPLY DELETED MASK',
'IN PARTITION',
// https://clickhouse.com/docs/sql-reference/statements/alter/column
'{ADD | DROP | RENAME | CLEAR | COMMENT | MODIFY | ALTER | MATERIALIZE} COLUMN',
// https://clickhouse.com/docs/sql-reference/statements/alter/partition
'{DETACH | DROP | ATTACH | FETCH | MOVE} {PART | PARTITION}',
'DROP DETACHED {PART | PARTITION}',
'{FORGET | REPLACE} PARTITION',
'CLEAR COLUMN',
'{FREEZE | UNFREEZE} [PARTITION]',
'CLEAR INDEX',
'TO {DISK | VOLUME}',
'[DELETE | REWRITE PARTS] IN PARTITION',
// https://clickhouse.com/docs/sql-reference/statements/alter/setting
'{MODIFY | RESET} SETTING',
// https://clickhouse.com/docs/sql-reference/statements/alter/delete
'DELETE WHERE',
// https://clickhouse.com/docs/sql-reference/statements/alter/order-by
'MODIFY ORDER BY',
// https://clickhouse.com/docs/sql-reference/statements/alter/sample-by
'{MODIFY | REMOVE} SAMPLE BY',
// https://clickhouse.com/docs/sql-reference/statements/alter/skipping-index
'{ADD | MATERIALIZE | CLEAR} INDEX [IF NOT EXISTS]',
'DROP INDEX [IF EXISTS]',
'GRANULARITY',
'AFTER',
'FIRST',

// https://clickhouse.com/docs/sql-reference/statements/alter/constraint
'ADD CONSTRAINT [IF NOT EXISTS]',
'DROP CONSTRAINT [IF EXISTS]',
// https://clickhouse.com/docs/sql-reference/statements/alter/ttl
'MODIFY TTL',
'REMOVE TTL',
// https://clickhouse.com/docs/sql-reference/statements/alter/statistics
'ADD STATISTICS [IF NOT EXISTS]',
'{DROP | CLEAR} STATISTICS [IF EXISTS]',
'MATERIALIZE STATISTICS [ALL | IF EXISTS]',
// https://clickhouse.com/docs/sql-reference/statements/alter/quota
'KEYED BY',
'NOT KEYED',
'FOR [RANDOMIZED] INTERVAL',
// https://clickhouse.com/docs/sql-reference/statements/alter/row-policy
'AS {PERMISSIVE | RESTRICTIVE}',
'FOR SELECT',
// https://clickhouse.com/docs/sql-reference/statements/alter/projection
'ADD PROJECTION [IF NOT EXISTS]',
'{DROP | MATERIALIZE | CLEAR} PROJECTION [IF EXISTS]',
// https://clickhouse.com/docs/sql-reference/statements/create/view#refreshable-materialized-view
'REFRESH {EVERY | AFTER}',
'RANDOMIZE FOR',
'APPEND',
'APPEND TO',
// https://clickhouse.com/docs/sql-reference/statements/delete
'DELETE FROM',
// https://clickhouse.com/docs/sql-reference/statements/explain
'EXPLAIN [AST | SYNTAX | QUERY TREE | PLAN | PIPELINE | ESTIMATE | TABLE OVERRIDE]',
// https://clickhouse.com/docs/sql-reference/statements/grant
'GRANT ON CLUSTER',
'GRANT CURRENT GRANTS',
'WITH GRANT OPTION',
// https://clickhouse.com/docs/sql-reference/statements/revoke
'REVOKE ON CLUSTER',
'ADMIN OPTION FOR',
// https://clickhouse.com/docs/sql-reference/statements/check-table
'CHECK TABLE',
'PARTITION ID',
// https://clickhouse.com/docs/sql-reference/statements/describe-table
'{DESC | DESCRIBE} TABLE',
]);

const reservedSetOperations = expandPhrases([
// https://clickhouse.com/docs/sql-reference/statements/select/union
'UNION [ALL | DISTINCT]',
// https://clickhouse.com/docs/sql-reference/statements/parallel_with
'PARALLEL WITH',
]);

const reservedJoins = expandPhrases([
// https://clickhouse.com/docs/sql-reference/statements/select/join
'[GLOBAL] [INNER|LEFT|RIGHT|FULL|CROSS] [OUTER|SEMI|ANTI|ANY|ALL|ASOF] JOIN',
// https://clickhouse.com/docs/sql-reference/statements/select/array-join
'[LEFT] ARRAY JOIN',
]);

const reservedKeywordPhrases = expandPhrases([
'{ROWS | RANGE} BETWEEN',
'ALTER MATERIALIZE STATISTICS',
]);

// https://clickhouse.com/docs/sql-reference/syntax
export const clickhouse: DialectOptions = {
name: 'clickhouse',
tokenizerOptions: {
reservedSelect,
reservedClauses: [...reservedClauses, ...standardOnelineClauses, ...tabularOnelineClauses],
reservedSetOperations,
reservedJoins,
reservedKeywordPhrases,

reservedKeywords: keywords,
reservedDataTypes: dataTypes,
reservedFunctionNames: functions,
extraParens: ['[]'],
Copy link
Collaborator

@nene nene Dec 13, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ClickHouse also supports map literals, so we should include '{}' in here.

But unfortunately that syntax conflicts with parameter syntax. It seems to me that in map literals the keys have to be quoted. I think we could distinguish these two based on that.

An example test which currently fails:

  it('supports map literals', () => {
    expect(format(`SELECT {'foo':1,'bar':10,'baz':2,'zap':8};`)).toBe(dedent`
      SELECT
        {'foo': 1, 'bar': 10, 'baz': 2, 'zap': 8};
    `);
  });

lineCommentTypes: ['#', '--'],
nestedBlockComments: false,
underscoresInNumbers: true,
stringTypes: ['$$', "''-qq-bs"],
identTypes: ['""-qq-bs', '``'],
paramTypes: {
// https://clickhouse.com/docs/sql-reference/syntax#defining-and-using-query-parameters
custom: [
{
regex: String.raw`\{\s*[^:]+:[^}]+\}`,
key: v => {
const match = /\{([^:]+):/.exec(v);
Comment on lines +261 to +263
Copy link
Collaborator

@nene nene Dec 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor inconsistency: the \s* is not necessary in the first regex.

return match ? match[1].trim() : v;
},
},
],
},
operators: [
// Arithmetic
'%', // modulo

// Ternary
'?',
':',

// Lambda creation
'->',
Comment on lines +269 to +278
Copy link
Collaborator

@nene nene Dec 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are missing some operators: ==, <=> and ||.

],
postProcess,
},
formatOptions: {
onelineClauses: [...standardOnelineClauses, ...tabularOnelineClauses],
tabularOnelineClauses,
},
};

/**
* Converts IN and ANY from RESERVED_FUNCTION_NAME to RESERVED_KEYWORD
* when they are used as operators/modifiers (not function calls).
*
* IN operator: foo IN (1, 2, 3) - IN comes after an identifier/expression
* IN function: IN(foo, 1, 2, 3) - IN comes at start or after operators/keywords
Copy link
Collaborator

@nene nene Dec 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The IN-function/operator logic was removed, but the comments still talk about it.

*
* ANY join modifier: ANY LEFT JOIN, ANY JOIN - ANY comes after an operator
* any() aggregate function: any(column) - selects first encountered value
*/
function postProcess(tokens: Token[]): Token[] {
return tokens.map((token, i) => {
const nextToken = tokens[i + 1] || EOF_TOKEN;
const prevToken = tokens[i - 1] || EOF_TOKEN;

// If we have queries like
// > GRANT SELECT, INSERT ON db.table TO john
// > GRANT SELECT(a, b), SELECT(c) ON db.table TO john
// we want to format them as
// > GRANT
// > SELECT,
// > INSERT ON db.table
// > TO john
// > GRANT
// > SELECT(a, b),
// > SELECT(c) ON db.table
// > TO john
// To do this we need to convert the SELECT keyword to a RESERVED_KEYWORD.
if (
token.type === TokenType.RESERVED_SELECT &&
(nextToken.type === TokenType.COMMA ||
prevToken.type === TokenType.RESERVED_CLAUSE ||
prevToken.type === TokenType.COMMA)
) {
return { ...token, type: TokenType.RESERVED_KEYWORD };
}

// We should format `set(100)` as-is rather than `SET (100)`
if (isToken.SET(token) && nextToken.type === TokenType.OPEN_PAREN) {
return { ...token, type: TokenType.RESERVED_FUNCTION_NAME };
}

return token;
});
}
Loading