Skip to content

BE-360: HashQL: Rework compiletest UI#8340

Open
indietyp wants to merge 19 commits intobm/be-302-hashql-statement-coloring-based-on-capabilitiesfrom
bm/be-360-hashql-rework-compiletest
Open

BE-360: HashQL: Rework compiletest UI#8340
indietyp wants to merge 19 commits intobm/be-302-hashql-statement-coloring-based-on-capabilitiesfrom
bm/be-360-hashql-rework-compiletest

Conversation

@indietyp
Copy link
Member

@indietyp indietyp commented Jan 30, 2026

Modernize the HashQL Compiletest UI

🌟 What is the purpose of this PR?

This PR modernizes the HashQL compiletest UI by replacing the indicatif-based progress reporting with a more robust and feature-rich TUI (Text User Interface) using ratatui. It also improves the overall architecture of the test harness by separating concerns and providing multiple output formats.

🔍 What does this change?

  • Replaces indicatif and tracing-indicatif with ratatui for a more interactive terminal UI
  • Adds three output modes: interactive TUI, human-readable text, and JSON
  • Restructures the codebase with a cleaner architecture separating test discovery, execution, and reporting
  • Improves error reporting and diagnostic verification
  • Adds detailed statistics for test runs including timing breakdowns, I/O operations, and slowest tests
  • Enhances the CLI interface with better documentation and more options

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

🛡 What tests cover this?

The changes are to the test harness itself, which is used to run tests for HashQL.

❓ How to test this?

  1. Run the compiletest CLI with different output formats:
    • cargo run -p hashql-compiletest -- run (interactive TUI)
    • cargo run -p hashql-compiletest -- run --format human
    • cargo run -p hashql-compiletest -- run --format json
  2. Try listing tests: cargo run -p hashql-compiletest -- list
  3. Try listing suites: cargo run -p hashql-compiletest -- suites

screenshot.2026-01-31T00-21-13.mp4 (uploaded via Graphite)

@cursor
Copy link

cursor bot commented Jan 30, 2026

PR Summary

Medium Risk
Large refactor of the compiletest harness/CLI plus new TUI rendering and dependency changes; risk is mainly regressions in test discovery/execution output expectations and terminal/CI behavior across formats.

Overview
Replaces the hashql-compiletest execution/reporting stack with a new runner + harness architecture and adds three output modes: interactive TUI, human text, and newline-delimited JSON.

Swaps out indicatif/tracing-indicatif (and related deps like radix_trie) for ratatui + ansi-to-tui, updates the CLI entrypoint, and changes logging setup (including enabling JSON tracing output when requested).

Updates annotation verification to return structured Report<[AnnotationError]> results and enhances trial execution with per-phase timing + I/O statistics that power the new UIs; dependency-diagram docs are updated accordingly.

Written by Cursor Bugbot for commit 864f436. This will update automatically on new commits. Configure here.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 1, 2026

Benchmark results

@rust/hash-graph-benches – Integrations

policy_resolution_large

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2002 $$27.4 \mathrm{ms} \pm 459 \mathrm{μs}\left({\color{gray}1.76 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$3.16 \mathrm{ms} \pm 12.7 \mathrm{μs}\left({\color{gray}-1.431 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 1001 $$11.8 \mathrm{ms} \pm 102 \mathrm{μs}\left({\color{gray}-1.774 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 3314 $$41.7 \mathrm{ms} \pm 324 \mathrm{μs}\left({\color{gray}0.813 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$13.5 \mathrm{ms} \pm 73.6 \mathrm{μs}\left({\color{gray}0.225 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 1526 $$22.7 \mathrm{ms} \pm 164 \mathrm{μs}\left({\color{gray}-0.574 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 2078 $$42.3 \mathrm{ms} \pm 169 \mathrm{μs}\left({\color{gray}-1.224 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$19.7 \mathrm{ms} \pm 111 \mathrm{μs}\left({\color{gray}-0.480 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 1033 $$27.1 \mathrm{ms} \pm 181 \mathrm{μs}\left({\color{gray}-0.623 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_medium

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 102 $$3.58 \mathrm{ms} \pm 20.3 \mathrm{μs}\left({\color{gray}0.341 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.77 \mathrm{ms} \pm 14.7 \mathrm{μs}\left({\color{gray}-0.533 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 51 $$3.14 \mathrm{ms} \pm 18.0 \mathrm{μs}\left({\color{gray}0.351 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 269 $$4.92 \mathrm{ms} \pm 27.7 \mathrm{μs}\left({\color{gray}0.003 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$3.32 \mathrm{ms} \pm 17.9 \mathrm{μs}\left({\color{gray}-0.722 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 107 $$3.85 \mathrm{ms} \pm 17.6 \mathrm{μs}\left({\color{gray}-2.277 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 133 $$4.00 \mathrm{ms} \pm 30.5 \mathrm{μs}\left({\color{gray}-4.754 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.18 \mathrm{ms} \pm 16.6 \mathrm{μs}\left({\color{gray}-1.166 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 63 $$3.79 \mathrm{ms} \pm 20.6 \mathrm{μs}\left({\color{gray}-0.819 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_none

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2 $$2.33 \mathrm{ms} \pm 10.0 \mathrm{μs}\left({\color{gray}-0.458 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.30 \mathrm{ms} \pm 14.5 \mathrm{μs}\left({\color{gray}0.356 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 1 $$2.38 \mathrm{ms} \pm 12.7 \mathrm{μs}\left({\color{gray}-0.313 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 8 $$2.56 \mathrm{ms} \pm 10.5 \mathrm{μs}\left({\color{gray}-0.334 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.46 \mathrm{ms} \pm 10.6 \mathrm{μs}\left({\color{gray}-0.310 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 3 $$2.65 \mathrm{ms} \pm 12.5 \mathrm{μs}\left({\color{gray}0.018 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_small

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 52 $$2.71 \mathrm{ms} \pm 13.4 \mathrm{μs}\left({\color{gray}-0.279 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.39 \mathrm{ms} \pm 8.60 \mathrm{μs}\left({\color{gray}0.670 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 25 $$2.53 \mathrm{ms} \pm 10.2 \mathrm{μs}\left({\color{gray}-0.633 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 94 $$3.04 \mathrm{ms} \pm 13.2 \mathrm{μs}\left({\color{gray}-1.601 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$2.60 \mathrm{ms} \pm 10.4 \mathrm{μs}\left({\color{gray}-0.680 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 26 $$2.84 \mathrm{ms} \pm 11.2 \mathrm{μs}\left({\color{gray}-0.057 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 66 $$2.96 \mathrm{ms} \pm 13.0 \mathrm{μs}\left({\color{gray}0.394 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.58 \mathrm{ms} \pm 10.9 \mathrm{μs}\left({\color{gray}-0.269 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 29 $$2.81 \mathrm{ms} \pm 10.4 \mathrm{μs}\left({\color{gray}0.159 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_complete

Function Value Mean Flame graphs
entity_by_id;one_depth 1 entities $$38.4 \mathrm{ms} \pm 152 \mathrm{μs}\left({\color{gray}2.28 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 10 entities $$57.6 \mathrm{ms} \pm 325 \mathrm{μs}\left({\color{lightgreen}-22.783 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 25 entities $$42.6 \mathrm{ms} \pm 169 \mathrm{μs}\left({\color{gray}0.400 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 5 entities $$45.1 \mathrm{ms} \pm 160 \mathrm{μs}\left({\color{gray}2.10 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 50 entities $$53.7 \mathrm{ms} \pm 325 \mathrm{μs}\left({\color{gray}3.78 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 1 entities $$39.8 \mathrm{ms} \pm 197 \mathrm{μs}\left({\color{gray}-0.242 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 10 entities $$240 \mathrm{ms} \pm 744 \mathrm{μs}\left({\color{lightgreen}-41.943 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 25 entities $$94.9 \mathrm{ms} \pm 405 \mathrm{μs}\left({\color{gray}2.66 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 5 entities $$83.0 \mathrm{ms} \pm 361 \mathrm{μs}\left({\color{gray}-0.659 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 50 entities $$274 \mathrm{ms} \pm 812 \mathrm{μs}\left({\color{gray}-1.258 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 1 entities $$14.2 \mathrm{ms} \pm 67.7 \mathrm{μs}\left({\color{gray}0.194 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 10 entities $$14.5 \mathrm{ms} \pm 69.6 \mathrm{μs}\left({\color{gray}2.18 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 25 entities $$14.7 \mathrm{ms} \pm 70.5 \mathrm{μs}\left({\color{gray}0.405 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 5 entities $$14.3 \mathrm{ms} \pm 62.2 \mathrm{μs}\left({\color{gray}-0.652 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 50 entities $$17.6 \mathrm{ms} \pm 106 \mathrm{μs}\left({\color{gray}3.57 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_linkless

Function Value Mean Flame graphs
entity_by_id 1 entities $$14.5 \mathrm{ms} \pm 69.9 \mathrm{μs}\left({\color{gray}0.442 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$14.3 \mathrm{ms} \pm 67.6 \mathrm{μs}\left({\color{gray}0.945 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$14.2 \mathrm{ms} \pm 67.4 \mathrm{μs}\left({\color{gray}0.138 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$14.9 \mathrm{ms} \pm 81.2 \mathrm{μs}\left({\color{gray}1.28 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$21.7 \mathrm{ms} \pm 167 \mathrm{μs}\left({\color{gray}0.545 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$28.5 \mathrm{ms} \pm 260 \mathrm{μs}\left({\color{gray}-1.118 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$29.0 \mathrm{ms} \pm 289 \mathrm{μs}\left({\color{gray}-2.445 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$29.5 \mathrm{ms} \pm 281 \mathrm{μs}\left({\color{gray}2.64 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$28.6 \mathrm{ms} \pm 366 \mathrm{μs}\left({\color{lightgreen}-6.792 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$29.1 \mathrm{ms} \pm 266 \mathrm{μs}\left({\color{gray}-0.706 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$28.6 \mathrm{ms} \pm 325 \mathrm{μs}\left({\color{gray}0.996 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$29.2 \mathrm{ms} \pm 279 \mathrm{μs}\left({\color{gray}1.45 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$28.8 \mathrm{ms} \pm 240 \mathrm{μs}\left({\color{gray}-2.074 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$27.9 \mathrm{ms} \pm 262 \mathrm{μs}\left({\color{gray}-1.891 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba $$7.82 \mathrm{ms} \pm 31.3 \mathrm{μs}\left({\color{gray}0.018 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
entity_by_property traversal_paths=0 0 $$44.6 \mathrm{ms} \pm 230 \mathrm{μs}\left({\color{gray}-0.047 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$91.4 \mathrm{ms} \pm 312 \mathrm{μs}\left({\color{gray}0.618 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$50.9 \mathrm{ms} \pm 348 \mathrm{μs}\left({\color{gray}1.05 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$58.9 \mathrm{ms} \pm 366 \mathrm{μs}\left({\color{gray}1.03 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$67.2 \mathrm{ms} \pm 376 \mathrm{μs}\left({\color{gray}0.720 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$72.9 \mathrm{ms} \pm 358 \mathrm{μs}\left({\color{gray}1.12 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=0 0 $$48.5 \mathrm{ms} \pm 241 \mathrm{μs}\left({\color{gray}-0.616 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$75.2 \mathrm{ms} \pm 351 \mathrm{μs}\left({\color{gray}0.529 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$55.1 \mathrm{ms} \pm 315 \mathrm{μs}\left({\color{gray}0.105 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$62.7 \mathrm{ms} \pm 364 \mathrm{μs}\left({\color{gray}0.954 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$64.7 \mathrm{ms} \pm 400 \mathrm{μs}\left({\color{gray}0.382 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$65.3 \mathrm{ms} \pm 454 \mathrm{μs}\left({\color{gray}1.68 \mathrm{\%}}\right) $$

scenarios

Function Value Mean Flame graphs
full_test query-limited $$135 \mathrm{ms} \pm 542 \mathrm{μs}\left({\color{gray}1.59 \mathrm{\%}}\right) $$ Flame Graph
full_test query-unlimited $$135 \mathrm{ms} \pm 375 \mathrm{μs}\left({\color{gray}1.02 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-limited $$102 \mathrm{ms} \pm 503 \mathrm{μs}\left({\color{gray}1.14 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-unlimited $$596 \mathrm{ms} \pm 1.92 \mathrm{ms}\left({\color{gray}4.48 \mathrm{\%}}\right) $$ Flame Graph

@indietyp indietyp force-pushed the bm/be-302-hashql-statement-coloring-based-on-capabilities branch from e803a60 to cc929a8 Compare February 2, 2026 16:36
@indietyp indietyp force-pushed the bm/be-360-hashql-rework-compiletest branch from d7d7b06 to 864f436 Compare February 2, 2026 16:36
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/apps > hash* Affects HASH (a `hash-*` app) area/apps > hash-graph area/apps area/deps Relates to third-party dependencies (area) area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > backend Owned by the @backend team

Development

Successfully merging this pull request may close these issues.

1 participant