Feat/perf and structural diff

[spiicez21]

feat: Structural Diffing, Row Virtualization, and Windows Stability

Overview

This pull request introduces a major upgrade to Hunk's diffing engine and rendering pipeline. By integrating tree-sitter based structural analysis and viewport virtualization, Hunk can now handle massive changesets with high-intelligence markers. Additionally, this PR marks the official rollout of Windows support, featuring a hardened core and a cross-platform stabilized test suite.

---

Key Enhancements

1. Structural Diffing (AST-Aware)

Implemented an optional AST-aware structural diffing mode using tree-sitter to distinguish meaningful code changes from formatting noise.

• Intelligence: Uses web-tree-sitter to identify structural modifications (additions, moves, or logic changes).
• UI: Adds a new 'S' indicator in the diff gutter for structurally significant rows.
• Controls: Documentation and support for new --structural and --no-structural CLI flags.
• Tech: New logic in src/core/structural-diff.ts and updated dependencies in package.json.

2. Performance: Viewport Row Virtualization

Optimized rendering for files with 10,000+ lines to ensure zero lag during high-speed scrolling.

• Mechanism: Implemented row virtualization in DiffPane; only visible rows are rendered to the terminal.
• Validation: Added new benchmarks in benchmarks/ to track and verify scrolling performance on huge files.

3. Official Windows Support & Stability

Full architectural hardening for Windows-based development environments.

• Packaging: Added hunkdiff-windows-x64 to the prebuilt package matrix.
• Path Logic: Fixed cross-platform path resolution in loaders and utilities to handle \ and / separators seamlessly.
• Test Robustness: Hardened Git loader tests against platform-specific branch naming conflicts and reserved character restrictions (e.g., quotes in filenames).

---

Commit-by-Commit Changes

Hash	Type	Description
c1916f2	docs	Update changelog for structural diffing and virtualization
7338d33	fix	Resolve Windows-specific path and Git branch issues in tests
9bd8dad	feat	Add windows-x64 support to prebuilt package helpers
5f07b74	test	Fix paths test path mismatches on Windows
c859dd8	feat	Stabilize structural diffing and implement row virtualization

---

Validation & Quality

• Typecheck: bun run typecheck Passed.
• Unit Tests: All 114 tests passed across all modules.
• Smoke Test: bun run test:tty-smoke verified .
• Real-world Verification: Confirmed virtualization performance on a 20,000-line diff and validated 'S' markers on complex TypeScript refactors.

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	tree-sitter-javascript@​0.25.0
	tree-sitter-typescript@​0.23.2
	web-tree-sitter@​0.26.8

View full report

[greptile-apps]

Greptile Summary

This PR introduces three major capabilities: AST-aware structural diffing (via the TypeScript compiler API, not web-tree-sitter as the description states), viewport-based row virtualization in DiffPane and PierreDiffView for large changesets, and official Windows support with cross-platform path and test fixes.

• Structural diffing: A new compareStructural function analyses TypeScript/JavaScript ASTs to tag diff rows with an 'S' marker. However, the feature is only wired up in the file-pair loader (hunk diff <left> <right>); all git/show/stash/patch modes silently ignore --structural because normalizePatchChangeset never calls compareStructural. The type definition also includes a \"move\" variant that is never emitted or matched.
• Row virtualization: Both the file-level windowing in DiffPane (visibleWindowedFileIds) and the row-level culling in PierreDiffView (visiblePlannedRows) look correct; the per-row structural-change lookup is a linear scan that scales poorly as structural changes and row counts grow.
• Windows support: Path normalization, branch-naming guards in tests, and addition of hunkdiff-windows-x64 to the prebuilt matrix appear solid.

Confidence Score: 3/5

Safe to merge for the virtualization and Windows stability work; the structural diffing feature is effectively broken for the most common workflows and should not be shipped without fixing the loader gap or adding a user-facing warning.

The virtualization and Windows hardening changes are well-contained and look correct. The structural diffing feature only works when comparing two concrete files on disk — the primary git diff and hunk show flows silently ignore --structural. The implementation also uses the TypeScript compiler API rather than the documented web-tree-sitter, and produces no analysis for non-JS/TS files without informing the user.

src/core/structural-diff.ts and src/core/loaders.ts need the most attention: the former lacks a file-type guard and has a dead move variant, and the latter never calls compareStructural for git/show/stash/patch inputs. src/ui/diff/PierreDiffView.tsx also deserves a second look for the per-row linear search.

Important Files Changed

Filename	Overview
src/core/structural-diff.ts	New structural-diffing module using the TypeScript compiler API (not web-tree-sitter as documented). Missing file-type guard, emits only three of four declared change types, and "move" is never produced or matched anywhere.
src/core/loaders.ts	compareStructural is only wired up in loadFileDiffChangeset (file-pair diffs); all git/show/stash/patch loaders silently ignore --structural, so S-markers never appear for those modes.
src/ui/diff/PierreDiffView.tsx	Adds per-row structural-change matching with a linear scan; also adds row virtualization (visiblePlannedRows filtering). The virtualization logic is sound but the structural lookup is O(rows×changes).
src/ui/components/panes/DiffPane.tsx	Adds showStructural prop threading, onScrollViewportChange for viewport tracking, and file-level windowing (visibleWindowedFileIds). The @ts-expect-error on onScroll is expected given the untyped API.
src/core/cli.ts	Adds --structural / --no-structural flag pair to applyCommonOptions and buildCommonOptions. Flag handling is consistent with other paired boolean flags.
src/ui/diff/renderRows.tsx	Adds structuralChange parameter to renderRow; renders 'S' prefix for structurally-significant rows in both split and stack layouts. "move" type is never checked here.
src/ui/components/panes/DiffSection.tsx	Adds showStructural, viewportTop, and viewportHeight props; memo comparator correctly updated to include the new fields.
src/ui/lib/appMenus.ts	Adds "Structural diff" toggle entry (hint "x") to the view menu; correctly wired to toggleStructural / showStructural.
src/core/types.ts	StructuralChange type added and DiffFile gains structuralChanges field. "move" variant is present but unused.
benchmarks/single-huge-file.ts	New benchmark for split-mode virtualization on large files.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    CLI["parseCli\n(--structural flag)"] --> KIND{Input kind?}
    KIND -->|"diff / difftool\n(file pair)"| FILELOADER["loadFileDiffChangeset\nreads leftText + rightText"]
    KIND -->|"git / show / stash\n/ patch"| GITLOADER["normalizePatchChangeset\n(patch text only)"]
    FILELOADER -->|"options.structural=true"| STRUCT["compareStructural\nts.createSourceFile → AST diff"]
    FILELOADER -->|"options.structural=false"| NOSTRUCT["structuralChanges = undefined"]
    GITLOADER --> NOGIT["structuralChanges = undefined\n⚠️ always, even with --structural"]
    STRUCT --> BUILDFILE["buildDiffFile\nDiffFile.structuralChanges = changes"]
    NOSTRUCT --> BUILDFILE
    NOGIT --> BUILDFILE2["buildDiffFile\nDiffFile.structuralChanges = undefined"]
    BUILDFILE --> APP["App\nshowStructural state"]
    BUILDFILE2 --> APP
    APP -->|showStructural=true| PIERRE["PierreDiffView\nper-row: structuralChanges.find()\nO(rows × changes)"]
    PIERRE -->|match found| SMARKER["DiffRowView\nPrefix = 'S'"]
    PIERRE -->|no match| NORMAL["DiffRowView\nNormal rail marker"]

Loading

Comments Outside Diff (1)

1. src/core/loaders.ts, line 456-498 (link)

   --structural silently does nothing for git/show/stash/patch modes

   compareStructural is only called inside loadFileDiffChangeset, so hunk diff <ref>, hunk show, hunk stash show, and hunk patch --structural all parse and accept the flag but never compute structural changes. Every DiffFile produced by those loaders will have structuralChanges: undefined, so the 'S' markers never appear — with no warning to the user that the feature is inactive.

   The normalizePatchChangeset path (used by git, show, stash, patch) only has the unified diff text and not the raw before/after file content, so a different approach (e.g., reading file contents via git show / git diff-tree) would be needed. At minimum, the CLI should warn when --structural is passed in a mode where it has no effect.

   Prompt To Fix With AI

   This is a comment left during a code review.
   Path: src/core/loaders.ts
   Line: 456-498
   
   Comment:
   **`--structural` silently does nothing for git/show/stash/patch modes**
   
   `compareStructural` is only called inside `loadFileDiffChangeset`, so `hunk diff <ref>`, `hunk show`, `hunk stash show`, and `hunk patch --structural` all parse and accept the flag but never compute structural changes. Every `DiffFile` produced by those loaders will have `structuralChanges: undefined`, so the 'S' markers never appear — with no warning to the user that the feature is inactive.
   
   The `normalizePatchChangeset` path (used by git, show, stash, patch) only has the unified diff text and not the raw before/after file content, so a different approach (e.g., reading file contents via `git show` / `git diff-tree`) would be needed. At minimum, the CLI should warn when `--structural` is passed in a mode where it has no effect.
   
   How can I resolve this? If you propose a fix, please make it concise.

Prompt To Fix All With AI

Fix the following 4 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 4
src/core/loaders.ts:456-498
**`--structural` silently does nothing for git/show/stash/patch modes**

`compareStructural` is only called inside `loadFileDiffChangeset`, so `hunk diff <ref>`, `hunk show`, `hunk stash show`, and `hunk patch --structural` all parse and accept the flag but never compute structural changes. Every `DiffFile` produced by those loaders will have `structuralChanges: undefined`, so the 'S' markers never appear — with no warning to the user that the feature is inactive.

The `normalizePatchChangeset` path (used by git, show, stash, patch) only has the unified diff text and not the raw before/after file content, so a different approach (e.g., reading file contents via `git show` / `git diff-tree`) would be needed. At minimum, the CLI should warn when `--structural` is passed in a mode where it has no effect.

### Issue 2 of 4
src/core/structural-diff.ts:1-6
**TypeScript compiler API used on all file types without an extension guard**

`ts.createSourceFile` is called regardless of the file's language. For non-TypeScript/JavaScript files (`.py`, `.go`, `.rs`, `.css`, JSON, etc.) the TypeScript parser produces a meaningless AST: the `collectTopLevel` function only recognises `FunctionDeclaration`, `ClassDeclaration`, and `VariableStatement` nodes, so those patterns never match for non-JS/TS syntax, and `compareStructural` silently returns an empty array. The PR description also states the implementation uses `web-tree-sitter`, but the code imports from the `typescript` package instead — the documentation and the implementation differ.

### Issue 3 of 4
src/core/structural-diff.ts:12-17
**`"move"` change type is declared but never emitted or matched**

`StructuralChange.type` includes `"move"`, but `compareStructural` never produces it — only `"addition"`, `"deletion"`, and `"modification"` are generated. Furthermore, the row-matching logic in `PierreDiffView.tsx` only checks for `"deletion"`, `"addition"`, and `"modification"` when deciding whether to show an 'S' marker, so even if a move were produced it would never be displayed. This dead variant should either be removed from the union or implemented.

### Issue 4 of 4
src/ui/diff/PierreDiffView.tsx:166-196
**O(rows × changes) linear scan per rendered row**

For each rendered row, `file.structuralChanges?.find(...)` iterates over all structural changes. On a large file with many structural nodes and thousands of diff rows this is quadratic. A pre-computed `Set` of affected line numbers (built once outside the render loop with a `useMemo`) would reduce per-row lookup to O(1).

_{Reviews (1): Last reviewed commit: "fix(diff): render structural change mark..." | Re-trigger Greptile}

[benvinegar]

Very cool. This is going to take me some time to get my head around though. Will report back.

Is it possible to split out the Windows changes as a separate PR?

[clabby]

Would also love structural diff support.

If we want to offload this to difftastic rather than having custom structural diff logic, the JSON output of the CLI paired with Wilfred/difftastic#936 would do the trick.