Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
c236caa
New syntax with automatic type and range inference
thomasp85 Jan 22, 2026
0a00f90
Merged upstream/main into scale-syntax
thomasp85 Jan 22, 2026
fe6cb5b
Fix for new polars version
thomasp85 Jan 22, 2026
b0884a7
Clear out domain<->range use and allow null in input range
thomasp85 Jan 23, 2026
5900450
Move scale logic to scale module
thomasp85 Jan 23, 2026
56e21f0
Prepare for default output range
thomasp85 Jan 25, 2026
615f370
add color interpolation
thomasp85 Jan 26, 2026
337bf79
default output and palettes
thomasp85 Jan 26, 2026
5f697d1
Merge commit '3ab1ba2409039dcfb763b8a1a9f7a41311067c3e'
thomasp85 Jan 26, 2026
d022183
update to match changes in PR
thomasp85 Jan 26, 2026
2a4b3ef
Implement scale expansion for positional aesthetics
thomasp85 Jan 26, 2026
9f3e42f
add oob support
thomasp85 Jan 26, 2026
b0e5ab3
Finalise transforms
thomasp85 Jan 26, 2026
5833126
refactor transforms
thomasp85 Jan 26, 2026
c6207dc
sort of log naming
thomasp85 Jan 27, 2026
a338b0e
Add minor breaks functionality for a future modern backend
thomasp85 Jan 27, 2026
dcb454f
add range to schema
thomasp85 Jan 27, 2026
c2f1da0
Move temporal scales to transform
thomasp85 Jan 27, 2026
61e24fb
Add implicit scales
thomasp85 Jan 27, 2026
cad464a
Use scale type to determine grouping
thomasp85 Jan 27, 2026
cae386a
fix clippy warnings
thomasp85 Jan 27, 2026
6c7e940
reformat
thomasp85 Jan 27, 2026
8ee7b94
implement binning scale behaviour
thomasp85 Jan 28, 2026
98910b9
Clean up the whole colour->color->fill/stroke pipeline
thomasp85 Jan 28, 2026
bd1927e
Fix some clippy warnings
thomasp85 Jan 28, 2026
d73b531
support binning in vegalite
thomasp85 Jan 28, 2026
cd61fe3
Add renaming clause
thomasp85 Jan 28, 2026
71f3f41
Add renaming formatting
thomasp85 Jan 29, 2026
6079eaa
reformat
thomasp85 Jan 29, 2026
4357d2d
Handle temporal data correctly all throughout scales
thomasp85 Jan 29, 2026
b3b928e
Better handle boolean in discrete scale
thomasp85 Jan 29, 2026
81689a9
Make binning work with type casting
thomasp85 Jan 29, 2026
b7066a7
update examples
thomasp85 Jan 29, 2026
9c111df
New flow that inserts column casting early so all data has the correc…
thomasp85 Jan 30, 2026
96d4324
Add scale resolution of remapped scales
thomasp85 Feb 1, 2026
d49559f
Fix parsing for time
thomasp85 Feb 2, 2026
f6e2a31
fix a clippy warning
thomasp85 Feb 2, 2026
557cb71
initial scales doc
thomasp85 Feb 2, 2026
a1a6efb
reformat
thomasp85 Feb 2, 2026
04b0e67
Properly interpolate colour from scale in binned scale
thomasp85 Feb 2, 2026
54f1653
Add tz support when parsing datetime and upgraded breaks to use wilki…
thomasp85 Feb 2, 2026
e312ea9
update highlighing
thomasp85 Feb 2, 2026
4685aee
ensure data is clipped to coordinate system
thomasp85 Feb 2, 2026
4d44389
bunch of fixes to palette expansion
thomasp85 Feb 2, 2026
7454aca
Ensure breaks and input are properly clipped
thomasp85 Feb 2, 2026
eafbb21
Remove any notion of GUIDE
thomasp85 Feb 3, 2026
c6682de
remove guide. Clarify size and linewidth units
thomasp85 Feb 3, 2026
979c1cf
add inverse transforms of sqrt and logs
thomasp85 Feb 3, 2026
040e45b
Add continuous docs
thomasp85 Feb 3, 2026
1855323
fix interpolation of numeric output ranges
thomasp85 Feb 3, 2026
646320a
Only expand implicit range limits
thomasp85 Feb 3, 2026
a17187d
Add ordinal scale
thomasp85 Feb 3, 2026
6ae939d
ensure censoring works correctly
thomasp85 Feb 3, 2026
53872ff
Add squish support for binned
thomasp85 Feb 3, 2026
67f4b95
Better bin resolving for binned
thomasp85 Feb 3, 2026
f4a6d58
refactor scale resolution
thomasp85 Feb 3, 2026
20399e9
reformat
thomasp85 Feb 3, 2026
7df7b92
fix clippy warnings
thomasp85 Feb 3, 2026
555a34d
fix formatting
thomasp85 Feb 3, 2026
27372c1
Add discrete doc
thomasp85 Feb 3, 2026
34ebe6b
various small fixes
thomasp85 Feb 4, 2026
f1eeb38
limit oob types in binned
thomasp85 Feb 4, 2026
fc1093c
properly support identity
thomasp85 Feb 4, 2026
d6fdb33
Add identity doc
thomasp85 Feb 4, 2026
fe62576
small fixes to breaks
thomasp85 Feb 4, 2026
13a305e
Ensure binned range is properly converted
thomasp85 Feb 4, 2026
8dbd5e8
typos
thomasp85 Feb 4, 2026
3ae0e45
more refactor
thomasp85 Feb 4, 2026
8a9804b
try to cut down a bit on the tests
thomasp85 Feb 5, 2026
694566f
Fix input range for ordinal
thomasp85 Feb 5, 2026
82f15f9
Add ordinal doc
thomasp85 Feb 5, 2026
cb62281
add binned doc
thomasp85 Feb 5, 2026
db14696
update examples
thomasp85 Feb 5, 2026
1cf5e08
fix grouping
thomasp85 Feb 5, 2026
4fb9912
reformat
thomasp85 Feb 5, 2026
a8b53cf
remove last warnings
thomasp85 Feb 5, 2026
18f3c26
Self-review part 1
thomasp85 Feb 6, 2026
665f30b
Merge commit '640ce779fd3bc716249620ab5a1966be9770fbf7'
thomasp85 Feb 6, 2026
9ab73e9
reformat
thomasp85 Feb 6, 2026
5d34b9c
fix treesitter tests
thomasp85 Feb 6, 2026
2574493
refactor execute.rs into its own module
thomasp85 Feb 6, 2026
cb9ec83
Last fix for boxplots
thomasp85 Feb 6, 2026
1928cd7
reformat
thomasp85 Feb 6, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
130 changes: 72 additions & 58 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
SELECT date, revenue, region FROM sales WHERE year = 2024
VISUALISE date AS x, revenue AS y, region AS color
DRAW line
SCALE x SETTING type => 'date'
SCALE x VIA date
COORD cartesian SETTING ylim => [0, 100000]
LABEL title => 'Sales by Region', x => 'Date', y => 'Revenue'
THEME minimal
Expand Down Expand Up @@ -243,7 +243,7 @@ For detailed API documentation, see [`src/doc/API.md`](src/doc/API.md).

- Uses `tree-sitter-ggsql` grammar (507 lines, simplified approach)
- Parses **full query** (SQL + VISUALISE) into concrete syntax tree (CST)
- Grammar supports: PLOT/TABLE/MAP types, DRAW/SCALE/FACET/COORD/LABEL/GUIDE/THEME clauses
- Grammar supports: PLOT/TABLE/MAP types, DRAW/SCALE/FACET/COORD/LABEL/THEME clauses
- British and American spellings: `VISUALISE` / `VISUALIZE`
- **SQL portion parsing**: Basic SQL structure (SELECT, WITH, CREATE, INSERT, subqueries)
- **Recursive subquery support**: Fully recursive grammar for complex SQL
Expand Down Expand Up @@ -288,7 +288,6 @@ pub struct Plot {
pub facet: Option<Facet>, // FACET clause
pub coord: Option<Coord>, // COORD clause
pub labels: Option<Labels>, // LABEL clause
pub guides: Vec<Guide>, // GUIDE clauses
pub theme: Option<Theme>, // THEME clause
}

Expand Down Expand Up @@ -395,19 +394,6 @@ pub struct Labels {
pub labels: HashMap<String, String>, // label type → text
}

pub struct Guide {
pub aesthetic: String,
pub guide_type: Option<GuideType>,
pub properties: HashMap<String, ParameterValue>,
}

pub enum GuideType {
Legend,
ColorBar,
Axis,
None,
}

pub struct Theme {
pub style: Option<String>,
pub properties: HashMap<String, ParameterValue>,
Expand All @@ -421,7 +407,6 @@ pub struct Theme {
- `Plot::new()` - Create a new empty Plot
- `Plot::with_global_mapping(mapping)` - Create Plot with a global mapping
- `Plot::find_scale(aesthetic)` - Look up scale specification for an aesthetic
- `Plot::find_guide(aesthetic)` - Find a guide specification for an aesthetic
- `Plot::has_layers()` - Check if Plot has any layers
- `Plot::layer_count()` - Get the number of layers

Expand Down Expand Up @@ -781,7 +766,7 @@ SELECT * FROM (VALUES
SELECT * FROM sales
VISUALISE
DRAW line MAPPING date AS x, revenue AS y, region AS color
SCALE x SETTING type => 'date'
SCALE x VIA date
LABEL title => 'Sales Trends'
```

Expand Down Expand Up @@ -1093,16 +1078,15 @@ Where `<global_mapping>` can be:

### Clause Types

| Clause | Repeatable | Purpose | Example |
| ----------- | ---------- | ------------------ | ----------------------------------------- |
| `VISUALISE` | ✅ Yes | Entry point | `VISUALISE date AS x, revenue AS y` |
| `DRAW` | ✅ Yes | Define layers | `DRAW line MAPPING date AS x, value AS y` |
| `SCALE` | ✅ Yes | Configure scales | `SCALE x SETTING type => 'date'` |
| `FACET` | ❌ No | Small multiples | `FACET WRAP region` |
| `COORD` | ❌ No | Coordinate system | `COORD cartesian SETTING xlim => [0,100]` |
| `LABEL` | ❌ No | Text labels | `LABEL title => 'My Chart', x => 'Date'` |
| `GUIDE` | ✅ Yes | Legend/axis config | `GUIDE color SETTING position => 'right'` |
| `THEME` | ❌ No | Visual styling | `THEME minimal` |
| Clause | Repeatable | Purpose | Example |
| -------------- | ---------- | ------------------ | ------------------------------------ |
| `VISUALISE` | ✅ Yes | Entry point | `VISUALISE date AS x, revenue AS y` |
| `DRAW` | ✅ Yes | Define layers | `DRAW line MAPPING date AS x, value AS y` |
| `SCALE` | ✅ Yes | Configure scales | `SCALE x VIA date` |
| `FACET` | ❌ No | Small multiples | `FACET WRAP region` |
| `COORD` | ❌ No | Coordinate system | `COORD cartesian SETTING xlim => [0,100]` |
| `LABEL` | ❌ No | Text labels | `LABEL title => 'My Chart', x => 'Date'` |
| `THEME` | ❌ No | Visual styling | `THEME minimal` |

### DRAW Clause (Layers)

Expand Down Expand Up @@ -1214,49 +1198,79 @@ DRAW line
**Syntax**:

```sql
SCALE <aesthetic> SETTING
[type => <scale_type>]
[limits => [min, max]]
[breaks => <array | interval>]
[palette => <name>]
[domain => [values...]]
SCALE [TYPE] <aesthetic> [FROM <input>] [TO <output>] [VIA <transform>] [SETTING <properties>]
```

**Scale Types**:
**Type Modifiers** (optional, placed before aesthetic):

- **Continuous**: `linear`, `log10`, `log`, `log2`, `sqrt`, `reverse`
- **Discrete**: `categorical`, `ordinal`
- **Temporal**: `date`, `datetime`, `time`
- **Color Palettes**: `viridis`, `plasma`, `magma`, `inferno`, `cividis`, `diverging`, `sequential`
- **`CONTINUOUS`** - Continuous numeric data
- **`DISCRETE`** - Categorical/discrete data
- **`BINNED`** - Binned/bucketed data
- **`DATE`** - Date data (maps to Vega-Lite temporal type)
- **`DATETIME`** - Datetime data (maps to Vega-Lite temporal type)

**Subclauses**:

- **`FROM [...]`** - Input range specification (maps to Vega-Lite `scale.domain`)
- **`TO [...]`** or **`TO palette`** - Output range as array or named palette (maps to Vega-Lite `scale.range` or `scale.scheme`)
- **`VIA transform`** - Transformation method (reserved for future use)
- **`SETTING ...`** - Additional properties (e.g., `breaks`)

**Named Palettes** (used with `TO`):

- `viridis`, `plasma`, `magma`, `inferno`, `cividis`, `diverging`, `sequential`

**Critical for Date Formatting**:

```sql
SCALE x SETTING type => 'date'
SCALE x VIA date
-- Maps to Vega-Lite field type = "temporal"
-- Enables proper date axis formatting
```

**Domain Property**:
**Input Range Specification** (FROM clause):

The `domain` property explicitly sets the input domain for a scale:
The `FROM` clause explicitly sets the input range for a scale:

```sql
-- Set domain for discrete scale
SCALE color SETTING domain => ['red', 'green', 'blue']
-- Set range for discrete scale
SCALE DISCRETE color FROM ['A', 'B', 'C']

-- Set domain for continuous scale
SCALE x SETTING domain => [0, 100]
-- Set range for continuous scale
SCALE CONTINUOUS x FROM [0, 100]
```

**Note**: Cannot specify domain in both SCALE and COORD for the same aesthetic (will error).
**Range Specification** (TO clause):

**Example**:
The `TO` clause sets the output range - either explicit values or a named palette:

```sql
SCALE x SETTING type => 'date', breaks => '2 months'
SCALE y SETTING type => 'log10', limits => [1, 1000]
SCALE color SETTING palette => 'viridis', domain => ['A', 'B', 'C']
-- Explicit color values
SCALE color FROM ['A', 'B'] TO ['red', 'blue']

-- Named palette
SCALE color TO viridis
```

**Note**: Cannot specify range in both SCALE and COORD for the same aesthetic (will error).

**Examples**:

```sql
-- Date scale
SCALE x VIA date

-- Continuous scale with input range
SCALE CONTINUOUS y FROM [0, 100]

-- Discrete color scale with input range and output range
SCALE DISCRETE color FROM ['A', 'B', 'C'] TO ['red', 'green', 'blue']

-- Color scale with named palette
SCALE color TO viridis

-- Scale with input range and additional settings
SCALE x VIA date FROM ['2024-01-01', '2024-12-31'] SETTING breaks => '1 month'
```

### FACET Clause
Expand Down Expand Up @@ -1313,22 +1327,22 @@ COORD SETTING <properties>

- `xlim => [min, max]` - Set x-axis limits
- `ylim => [min, max]` - Set y-axis limits
- `<aesthetic> => [values...]` - Set domain for any aesthetic (color, fill, size, etc.)
- `<aesthetic> => [values...]` - Set range for any aesthetic (color, fill, size, etc.)

**Flip**:

- `<aesthetic> => [values...]` - Set domain for any aesthetic
- `<aesthetic> => [values...]` - Set range for any aesthetic

**Polar**:

- `theta => <aesthetic>` - Which aesthetic maps to angle (defaults to `y`)
- `<aesthetic> => [values...]` - Set domain for any aesthetic
- `<aesthetic> => [values...]` - Set range for any aesthetic

**Important Notes**:

1. **Axis limits auto-swap**: `xlim => [100, 0]` automatically becomes `[0, 100]`
2. **ggplot2 compatibility**: `coord_flip` preserves axis label names (labels stay with aesthetic names, not visual position)
3. **Domain conflicts**: Error if same aesthetic has domain in both SCALE and COORD
3. **Range conflicts**: Error if same aesthetic has input range in both SCALE and COORD
4. **Multi-layer support**: All coordinate transforms apply to all layers

**Status**:
Expand All @@ -1344,7 +1358,7 @@ COORD SETTING <properties>
-- Cartesian with axis limits
COORD cartesian SETTING xlim => [0, 100], ylim => [0, 50]

-- Cartesian with aesthetic domain
-- Cartesian with aesthetic range
COORD cartesian SETTING color => O ['red', 'green', 'blue']

-- Cartesian shorthand (type optional when using SETTING)
Expand All @@ -1353,7 +1367,7 @@ COORD SETTING xlim => [0, 100]
-- Flip coordinates for horizontal bar chart
COORD flip

-- Flip with aesthetic domain
-- Flip with aesthetic range
COORD flip SETTING color => ['A', 'B', 'C']

-- Polar for pie chart (theta defaults to y)
Expand Down Expand Up @@ -1427,7 +1441,7 @@ DRAW line
MAPPING sale_date AS x, total AS y, region AS color
DRAW point
MAPPING sale_date AS x, total AS y, region AS color
SCALE x SETTING type => 'date'
SCALE x VIA date
FACET WRAP region
LABEL title => 'Sales Trends by Region', x => 'Date', y => 'Total Quantity'
THEME minimal
Expand Down
3 changes: 3 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,9 @@ pyo3 = "0.26"
# Testing
proptest = "1.4"

# Color interpolation
palette = "0.7"

# Utilities
regex = "1.10"
chrono = "0.4"
Expand Down
18 changes: 9 additions & 9 deletions EXAMPLES.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,16 +111,16 @@ SCALE y SETTING type => 'log10'
SELECT date, temperature, station FROM weather
VISUALISE date AS x, temperature AS y, station AS color
DRAW line
SCALE color SETTING palette => 'viridis'
SCALE color TO viridis
```

### Custom Domain
### Custom input range

```sql
SELECT category, value FROM data
VISUALISE category AS x, value AS y, category AS fill
DRAW bar
SCALE fill SETTING domain => ['A', 'B', 'C', 'D']
SCALE DISCRETE fill FROM ['A', 'B', 'C', 'D']
```

---
Expand Down Expand Up @@ -470,7 +470,7 @@ WHERE timestamp >= NOW() - INTERVAL '7 days'
VISUALISE timestamp AS x, temperature AS y, station AS color, station AS linetype
DRAW line
SCALE x SETTING type => 'datetime'
SCALE color SETTING palette => 'viridis'
SCALE color TO viridis
LABEL title => 'Temperature Trends',
x => 'Time',
y => 'Temperature (°C)'
Expand All @@ -496,7 +496,7 @@ LABEL title => 'Top 10 Products by Revenue',
THEME classic
```

### Distribution with Custom Domain
### Distribution with Custom range

```sql
SELECT
Expand All @@ -508,7 +508,7 @@ WHERE category IN ('A', 'B', 'C')
VISUALISE date AS x, value AS y, category AS color, value AS size
DRAW point
SCALE x SETTING type => 'date'
SCALE color SETTING domain => ['A', 'B', 'C']
SCALE DISCRETE color FROM ['A', 'B', 'C']
SCALE size SETTING limits => [0, 100]
COORD cartesian SETTING ylim => [0, 150]
LABEL title => 'Measurement Distribution',
Expand All @@ -528,7 +528,7 @@ FROM data_points
VISUALISE x, y, category AS color
DRAW point SETTING size => 5
DRAW text MAPPING label AS label
SCALE color SETTING palette => 'viridis'
SCALE color TO viridis
COORD cartesian SETTING xlim => [0, 100], ylim => [0, 100]
LABEL title => 'Annotated Scatter Plot',
x => 'X Axis',
Expand All @@ -545,7 +545,7 @@ GROUP BY cyl
ORDER BY cyl
VISUALISE cyl AS x, vehicle_count AS y
DRAW bar
SCALE x SETTING domain => [4, 6, 8]
SCALE DISCRETE x FROM [4, 6, 8]
LABEL title => 'Distribution of Vehicles by Number of Cylinders',
x => 'Number of Cylinders',
y => 'Number of Vehicles'
Expand Down Expand Up @@ -640,7 +640,7 @@ Draw Line

8. **Labels**: Always provide meaningful titles and axis labels for clarity.

9. **Domain Specification**: Use either SCALE or COORD for domain/limit specification, but not both for the same aesthetic.
9. **Range Specification**: Use either SCALE or COORD for range/limit specification, but not both for the same aesthetic.

---

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ Key grammar elements:
- `SCALE <aesthetic> SETTING` - Configure data-to-visual mappings
- `FACET` - Create small multiples (WRAP for flowing layout, BY for grid)
- `COORD` - Coordinate transformations (cartesian, flip, polar)
- `LABEL`, `THEME`, `GUIDE` - Styling and annotation
- `LABEL`, `THEME` - Styling and annotation

## Jupyter Kernel

Expand Down
10 changes: 9 additions & 1 deletion doc/_quarto.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ website:
announcement:
icon: info-circle
dismissable: true
content: "ggsql is still in early development and all functionality is subject to change"
content: "ggsql is still in early development and all functionality are subject to change"
type: primary
position: below-navbar
twitter-card:
Expand Down Expand Up @@ -80,6 +80,14 @@ website:
- section: Layers
contents:
- auto: syntax/layer/*
- section: Scales
contents:
- section: Types
contents:
- auto: syntax/scale/type/*
- section: Palettes
contents:
- auto: syntax/scale/palette/*


format:
Expand Down
Loading