Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 79 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
name: Release

on:
workflow_dispatch:
inputs:
version:
description: 'Version (e.g. v0.1.0)'
required: true
type: string

jobs:
build:
strategy:
matrix:
include:
- os: ubuntu-latest
platform: linux
- os: macos-latest
platform: mac
- os: windows-latest
platform: win

runs-on: ${{ matrix.os }}

steps:
- uses: actions/checkout@v4

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '22'
cache: 'npm'

- name: Install dependencies
run: npm ci

- name: Build web + Electron
run: npx vite build
env:
ELECTRON: 'true'

- name: Package desktop app
run: npx electron-builder --${{ matrix.platform }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: desktop-${{ matrix.platform }}
path: |
release/*.exe
release/*.dmg
release/*.AppImage
release/*.deb
if-no-files-found: ignore

github_release:
runs-on: ubuntu-latest
needs: [build]
name: Create GitHub release
permissions:
contents: write
steps:
- name: Download all artifacts
uses: actions/download-artifact@v4
with:
path: artifacts
merge-multiple: true

- name: Create Release
uses: ncipollo/release-action@v1
with:
artifacts: "artifacts/*"
tag: ${{ inputs.version }}
commit: ${{ github.sha }}
generateReleaseNotes: true
draft: false
prerelease: false
17 changes: 16 additions & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,5 +25,20 @@ jobs:
- name: Run tests
run: npm test

- name: Build
- name: Build (web)
run: npm run build

- name: Build (Electron)
run: ELECTRON=true npx vite build

- name: Install Playwright browsers
run: npx playwright install --with-deps chromium

- name: Electron e2e tests
run: xvfb-run --auto-servernum npx playwright test

- uses: actions/upload-artifact@v4
if: failure()
with:
name: playwright-traces
path: test-results/
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@ node_modules/

# Build output
dist/
dist-electron/
release/

# Local config (template is config.default.json)
config.json

# Vite
*.local
Expand Down
70 changes: 57 additions & 13 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# CLAUDE.md - RowBinary Visualizer
# CLAUDE.md - ClickHouse Format Explorer

## Project Overview

A web-based tool for visualizing ClickHouse RowBinary wire format data. Features an interactive hex viewer with AST-based type visualization, similar to ImHex. The tool queries a local ClickHouse database and presents the raw binary data alongside a decoded AST tree with bidirectional highlighting.
A tool for visualizing ClickHouse RowBinary and Native wire format data. Features an interactive hex viewer with AST-based type visualization, similar to ImHex. Available as a web app (Docker) or an Electron desktop app that connects to your existing ClickHouse server.

**Current scope**: RowBinaryWithNamesAndTypes and Native formats.

Expand All @@ -11,16 +11,22 @@ A web-based tool for visualizing ClickHouse RowBinary wire format data. Features
- **Frontend**: React 18 + TypeScript + Vite
- **State**: Zustand
- **UI**: react-window (virtualized hex viewer), react-resizable-panels (split panes)
- **Testing**: Vitest + testcontainers (ClickHouse integration tests)
- **Deployment**: Docker (bundles ClickHouse + nginx + built frontend)
- **Desktop**: Electron (optional, connects to user's ClickHouse)
- **Testing**: Vitest + testcontainers (integration), Playwright (Electron e2e)
- **Deployment**: Docker (bundles ClickHouse + nginx) or Electron desktop app

## Commands

```bash
npm run dev # Start dev server (requires ClickHouse at localhost:8123)
npm run build # Build for production
npm run test # Run integration tests (uses testcontainers)
npm run lint # ESLint check
npm run dev # Start web dev server (requires ClickHouse at localhost:8123)
npm run build # Build web app for production
npm run test # Run integration tests (uses testcontainers)
npm run lint # ESLint check
npm run test:e2e # Build Electron + run Playwright e2e tests

# Electron desktop app
npm run electron:dev # Dev mode with hot reload
npm run electron:build # Package desktop installer for current platform

# Docker (self-contained with bundled ClickHouse)
docker build -t rowbinary-explorer .
Expand All @@ -33,7 +39,7 @@ docker run -d -p 8080:80 rowbinary-explorer
src/
├── components/ # React components
│ ├── App.tsx # Main layout with resizable panels
│ ├── QueryInput.tsx # SQL query input + run button
│ ├── QueryInput.tsx # SQL query input + run button + connection settings
│ ├── HexViewer/ # Virtualized hex viewer with highlighting
│ └── AstTree/ # Collapsible AST tree view
├── core/
Expand All @@ -52,10 +58,15 @@ src/
│ │ ├── type-lexer.ts # Tokenizer for type strings
│ │ └── type-parser.ts # Parser: string -> ClickHouseType
│ └── clickhouse/
│ └── client.ts # HTTP client for ClickHouse queries
│ └── client.ts # HTTP client (fetch for web, IPC for Electron)
├── store/
│ └── store.ts # Zustand store (query, parsed data, UI state)
└── styles/ # CSS files
electron/
├── main.ts # Electron main process (window, IPC handlers)
└── preload.ts # Preload script (contextBridge → electronAPI)
e2e/
└── electron.spec.ts # Playwright Electron e2e tests
docs/
├── rowbinaryspec.md # RowBinary wire format specification
├── nativespec.md # Native wire format specification
Expand Down Expand Up @@ -94,14 +105,34 @@ A discriminated union representing all ClickHouse types (`src/core/types/clickho

### Decoding Flow
1. User enters SQL query, clicks "Run Query"
2. `ClickHouseClient` (`src/core/clickhouse/client.ts`) POSTs query with selected format
2. `ClickHouseClient` (`src/core/clickhouse/client.ts`) sends query:
- **Web mode**: `fetch()` via Vite proxy or nginx
- **Electron mode**: IPC to main process → `fetch()` to ClickHouse (no CORS)
3. Decoder parses the binary response:
- **RowBinary** (`rowbinary-decoder.ts`): Row-oriented, header + rows
- **Native** (`native-decoder.ts`): Column-oriented with blocks
4. Type strings parsed via `parseType()` into `ClickHouseType`
5. Each decoded value returns an `AstNode` with byte tracking
6. UI renders hex view (left) and AST tree (right)

### Electron Architecture
```
Renderer (React) Main Process (Node.js)
│ │
├─ window.electronAPI │
│ .executeQuery(opts) ────────►├─ fetch(clickhouseUrl + query)
│ │ → ArrayBuffer
│◄── IPC response ──────────────┤
│ │
├─ Uint8Array → decoders │
└─ render hex view + AST tree │
```

- Runtime detection: `window.electronAPI` exists → IPC path, otherwise → `fetch()`
- `vite-plugin-electron` activates only when `ELECTRON=true` env var is set
- Connection config in `config.json` (project root in dev, next to executable in prod)
- Experimental ClickHouse settings (Variant, Dynamic, JSON, etc.) sent as query params

### Interactive Highlighting
- Click a node in AST tree → highlights corresponding bytes in hex view
- Click a byte in hex view → selects the deepest AST node containing that byte
Expand Down Expand Up @@ -133,12 +164,13 @@ A discriminated union representing all ClickHouse types (`src/core/types/clickho

## Testing

Integration tests use testcontainers to spin up a real ClickHouse instance:
### Integration Tests (Vitest + testcontainers)

Tests use testcontainers to spin up a real ClickHouse instance:
```bash
npm run test # Runs all integration tests
```

### Test Structure
Tests are organized into three categories with shared test case definitions:

1. **Smoke Tests** (`smoke.integration.test.ts`)
Expand All @@ -155,6 +187,18 @@ Tests are organized into three categories with shared test case definitions:
- Analyze byte coverage of AST leaf nodes
- Report uncovered byte ranges

### Electron e2e Tests (Playwright)

```bash
npm run test:e2e # Builds Electron app + runs Playwright tests
```

Tests in `e2e/electron.spec.ts` launch the actual Electron app and verify:
- App window opens and UI renders
- Host input is visible (Electron mode) and Share button is hidden
- Connection settings can be edited
- Upload button is present and functional

### Test Case Interface
```typescript
interface ValidationTestCase {
Expand Down
41 changes: 37 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ClickHouse Format Explorer

A web-based tool for visualizing ClickHouse RowBinary and Native format data. Features an interactive hex viewer with AST-based type visualization.
A tool for visualizing ClickHouse RowBinary and Native format data. Features an interactive hex viewer with AST-based type visualization. Available as a web app or Electron desktop app.

![Screenshot](.static/screenshot.png)

Expand All @@ -11,6 +11,7 @@ A web-based tool for visualizing ClickHouse RowBinary and Native format data. Fe
- **AST Tree**: Collapsible tree view showing decoded structure
- **Interactive Highlighting**: Selecting a node in the tree highlights corresponding bytes in the hex view (and vice versa)
- **Full Type Support**: All ClickHouse types including Variant, Dynamic, JSON, Geo types, Nested, etc.
- **Desktop App**: Electron app that connects to your existing ClickHouse server (no bundled DB)

## Quick Start (Docker)

Expand All @@ -23,9 +24,39 @@ docker run -d -p 8080:80 rowbinary-explorer

Open http://localhost:8080

## Development Setup
## Desktop App

For local development (requires ClickHouse at `localhost:8123`):
For developers who already run ClickHouse locally. Download the latest release for your platform from the [Releases](../../releases) page:

| Platform | Format |
|----------|--------|
| Windows | `.exe` (NSIS installer) |
| macOS | `.dmg` |
| Linux | `.AppImage` / `.deb` |

### Configuration

The app looks for a `config.json` file next to the executable:

```json
{
"host": "http://localhost:8123"
}
```

You can also change the host from the **Host** field in the toolbar. Changes are saved back to `config.json`.

### Building from source

```bash
npm install
npm run electron:dev # Dev mode with hot reload
npm run electron:build # Package installer for current platform
```

## Web Development Setup

For local web development (requires ClickHouse at `localhost:8123`):

```bash
npm install
Expand All @@ -37,7 +68,7 @@ Open http://localhost:5173
## Usage

1. Enter a SQL query in the input box
2. Click "Run Query" to fetch data from ClickHouse (expects local instance at `localhost:8123`)
2. Click "Execute" to fetch data from ClickHouse
3. Explore the parsed data:
- Click nodes in the AST tree to highlight bytes
- Click bytes in the hex viewer to select the corresponding node
Expand Down Expand Up @@ -66,3 +97,5 @@ SELECT '{"user": {"id": 123}}'::JSON(`user.id` UInt32)
- Zustand (state management)
- react-window (virtualized hex viewer)
- react-resizable-panels (split pane layout)
- Electron (desktop app, optional)
- Playwright (e2e testing)
Binary file added build/icon.icns
Binary file not shown.
Binary file added build/icon.ico
Binary file not shown.
Binary file added build/icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions config.default.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"host": "http://localhost:8123"
}
Loading