Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 15 additions & 13 deletions .brunch-fixtures/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,27 +9,29 @@ This directory is the on-disk home of the fixture strategy described in

```
.brunch-fixtures/
├── briefs/ # Curated product briefs (YAML)
│ ├── offline-kanban.yaml
│ ├── role-based-doc-sharing.yaml
├── briefs/ # Curated product briefs (JSON)
│ ├── brief-001-identity-reference.json
│ ├── brief-002-state-lifecycle.json
│ ├── brief-003-derived-views.json
│ └── ...
└── <brief-id>/
└── <run-id>/
├── <run-id>.jsonl # Captured transcript
├── <run-id>.graph.json # Captured graph state
├── <run-id>.coherence.json # Captured coherence verdict + needs
└── <run-id>.meta.json # Brief id, persona dials, model, timestamps
├── <run-id>.meta.json # Brief id, driver mode, session, projection summary
├── <run-id>.graph.json # Deferred until the graph plane exists
└── <run-id>.coherence.json # Deferred until coherence is first-class
```

## Status

Empty by design until the `mode-shell-and-fixture-driver` frontier (M1) lands
the JSON-RPC stdio agent-as-user driver. Briefs may be authored ahead of that
under the `brief-library-curation` parallel frontier — see `memory/PLAN.md`.
The first M1 briefs live under `briefs/` as JSON files. Captured runs are added
under each brief id by the JSON-RPC stdio fixture driver.

## Conventions

- Briefs are short, human-readable YAML; the captured runs are the heavy data.
- Brief ids are kebab-case and stable; runs are timestamped or content-hashed.
- Property invariants from the fixture-strategy doc are checked on every
capture (replay regression, property regression, adversarial / generative).
- Briefs are short, human-readable JSON; the captured runs are the heavy data.
- Brief ids are kebab-case and stable; runs are timestamped, content-hashed, or
deterministic for reviewable scripted captures.
- Replay regression runs check transcript reproduction first. Property and
adversarial / generative checks come online as later milestones provide graph
and coherence artifacts.
4 changes: 4 additions & 0 deletions .brunch-fixtures/brief-001/scripted-001/scripted-001.jsonl
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{"type":"session","version":3,"id":"019e4a13-1a50-7eb3-a4f7-644d4bff42bd","timestamp":"2026-05-21T10:27:06.448Z","cwd":"/Users/lunelson/Code/hashintel/brunch-next"}
{"type":"custom","customType":"brunch.session_binding","data":{"schemaVersion":1,"sessionId":"019e4a13-1a50-7eb3-a4f7-644d4bff42bd","specId":"spec-a3e8371c-af78-45ee-8b1a-7752317caa35","specTitle":"Team knowledge cards"},"id":"fc95cd0b","parentId":null,"timestamp":"2026-05-21T10:27:06.449Z"}
{"type":"custom_message","customType":"brunch.elicitation_prompt","content":"Elicitation prompt for brief-001 — Team knowledge cards: A small team wants a shared workspace for knowledge cards. Each card has a stable identity, a human-readable title, and may link to other cards even when titles change.","display":true,"id":"101e1d31","parentId":"fc95cd0b","timestamp":"2026-05-21T10:27:06.451Z"}
{"type":"message","id":"71549d11","parentId":"101e1d31","timestamp":"2026-05-21T10:27:06.451Z","message":{"role":"user","content":"I care about renaming cards without breaking links.\nTwo cards can have similar titles, so titles cannot be the only reference.","timestamp":1779321600000}}
28 changes: 28 additions & 0 deletions .brunch-fixtures/brief-001/scripted-001/scripted-001.meta.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"schemaVersion": 1,
"briefId": "brief-001",
"runId": "scripted-001",
"timestamp": "2026-05-21T00:00:00.000Z",
"brunchVersion": "0.0.0",
"session": {
"id": "019e4a13-1a50-7eb3-a4f7-644d4bff42bd",
"sourceFile": "/Users/lunelson/Code/hashintel/brunch-next/.brunch/sessions/2026-05-21T10-27-06-448Z_019e4a13-1a50-7eb3-a4f7-644d4bff42bd.jsonl"
},
"driver": {
"mode": "scripted-deterministic"
},
"projectionSummary": {
"status": "ready",
"exchangeCount": 1,
"openPrompt": false
},
"artifacts": {
"jsonl": "scripted-001.jsonl",
"graph": {
"status": "deferred"
},
"coherence": {
"status": "deferred"
}
}
}
4 changes: 4 additions & 0 deletions .brunch-fixtures/brief-002/scripted-001/scripted-001.jsonl
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{"type":"session","version":3,"id":"019e4a13-1a57-77cd-8c77-43922404adca","timestamp":"2026-05-21T10:27:06.455Z","cwd":"/Users/lunelson/Code/hashintel/brunch-next"}
{"type":"custom","customType":"brunch.session_binding","data":{"schemaVersion":1,"sessionId":"019e4a13-1a57-77cd-8c77-43922404adca","specId":"spec-b37cabd4-12f5-4cea-bcbf-90a170e6f058","specTitle":"Approval workflow for vendor invoices"},"id":"99670cde","parentId":null,"timestamp":"2026-05-21T10:27:06.455Z"}
{"type":"custom_message","customType":"brunch.elicitation_prompt","content":"Elicitation prompt for brief-002 — Approval workflow for vendor invoices: A finance team needs invoices to move from draft to submitted to approved or rejected. Only budget owners can approve, and rejected invoices can be revised and resubmitted.","display":true,"id":"99dc94b3","parentId":"99670cde","timestamp":"2026-05-21T10:27:06.455Z"}
{"type":"message","id":"e0baa29d","parentId":"99dc94b3","timestamp":"2026-05-21T10:27:06.455Z","message":{"role":"user","content":"Rejected invoices are not terminal; they can go back to draft.\nApproved invoices should not be edited without reopening the workflow.","timestamp":1779321600000}}
28 changes: 28 additions & 0 deletions .brunch-fixtures/brief-002/scripted-001/scripted-001.meta.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"schemaVersion": 1,
"briefId": "brief-002",
"runId": "scripted-001",
"timestamp": "2026-05-21T00:00:00.000Z",
"brunchVersion": "0.0.0",
"session": {
"id": "019e4a13-1a57-77cd-8c77-43922404adca",
"sourceFile": "/Users/lunelson/Code/hashintel/brunch-next/.brunch/sessions/2026-05-21T10-27-06-455Z_019e4a13-1a57-77cd-8c77-43922404adca.jsonl"
},
"driver": {
"mode": "scripted-deterministic"
},
"projectionSummary": {
"status": "ready",
"exchangeCount": 1,
"openPrompt": false
},
"artifacts": {
"jsonl": "scripted-001.jsonl",
"graph": {
"status": "deferred"
},
"coherence": {
"status": "deferred"
}
}
}
4 changes: 4 additions & 0 deletions .brunch-fixtures/brief-003/scripted-001/scripted-001.jsonl
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{"type":"session","version":3,"id":"019e4a13-1a59-712e-82ff-78900f655901","timestamp":"2026-05-21T10:27:06.457Z","cwd":"/Users/lunelson/Code/hashintel/brunch-next"}
{"type":"custom","customType":"brunch.session_binding","data":{"schemaVersion":1,"sessionId":"019e4a13-1a59-712e-82ff-78900f655901","specId":"spec-b804de5f-762e-4d27-a538-8b8d38139bec","specTitle":"Project dashboard rollups"},"id":"74df4b55","parentId":null,"timestamp":"2026-05-21T10:27:06.457Z"}
{"type":"custom_message","customType":"brunch.elicitation_prompt","content":"Elicitation prompt for brief-003 — Project dashboard rollups: A product lead wants a dashboard that rolls task status, blockers, and recent decisions up from individual project notes into one current view.","display":true,"id":"72fa4dfe","parentId":"74df4b55","timestamp":"2026-05-21T10:27:06.457Z"}
{"type":"message","id":"d47729ba","parentId":"72fa4dfe","timestamp":"2026-05-21T10:27:06.457Z","message":{"role":"user","content":"If the source note changes, the dashboard should not silently stay stale.\nRecent decisions should show where they came from.","timestamp":1779321600000}}
28 changes: 28 additions & 0 deletions .brunch-fixtures/brief-003/scripted-001/scripted-001.meta.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"schemaVersion": 1,
"briefId": "brief-003",
"runId": "scripted-001",
"timestamp": "2026-05-21T00:00:00.000Z",
"brunchVersion": "0.0.0",
"session": {
"id": "019e4a13-1a59-712e-82ff-78900f655901",
"sourceFile": "/Users/lunelson/Code/hashintel/brunch-next/.brunch/sessions/2026-05-21T10-27-06-457Z_019e4a13-1a59-712e-82ff-78900f655901.jsonl"
},
"driver": {
"mode": "scripted-deterministic"
},
"projectionSummary": {
"status": "ready",
"exchangeCount": 1,
"openPrompt": false
},
"artifacts": {
"jsonl": "scripted-001.jsonl",
"graph": {
"status": "deferred"
},
"coherence": {
"status": "deferred"
}
}
}
19 changes: 19 additions & 0 deletions .brunch-fixtures/briefs/brief-001-identity-reference.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"schemaVersion": 1,
"id": "brief-001",
"title": "Team knowledge cards",
"kernelTags": ["identity-reference", "containment-topology"],
"productBrief": "A small team wants a shared workspace for knowledge cards. Each card has a stable identity, a human-readable title, and may link to other cards even when titles change.",
"expectedStructuralObservations": [
"Cards need stable IDs separate from mutable titles.",
"Links should target card identity rather than display text."
],
"scriptedUserNotes": [
"I care about renaming cards without breaking links.",
"Two cards can have similar titles, so titles cannot be the only reference."
],
"deferredExpectations": {
"graph": "Later graph fixtures should produce identity/reference nodes and link invariants.",
"coherence": "Later coherence checks should flag title-anchored references as weak evidence."
}
}
19 changes: 19 additions & 0 deletions .brunch-fixtures/briefs/brief-002-state-lifecycle.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"schemaVersion": 1,
"id": "brief-002",
"title": "Approval workflow for vendor invoices",
"kernelTags": ["state-lifecycle", "authority-capability"],
"productBrief": "A finance team needs invoices to move from draft to submitted to approved or rejected. Only budget owners can approve, and rejected invoices can be revised and resubmitted.",
"expectedStructuralObservations": [
"Invoice states and legal transitions must be explicit.",
"Approval authority depends on the budget owner role."
],
"scriptedUserNotes": [
"Rejected invoices are not terminal; they can go back to draft.",
"Approved invoices should not be edited without reopening the workflow."
],
"deferredExpectations": {
"graph": "Later graph fixtures should capture lifecycle states, transitions, and authority predicates.",
"coherence": "Later coherence checks should flag contradictory terminality claims."
}
}
19 changes: 19 additions & 0 deletions .brunch-fixtures/briefs/brief-003-derived-views.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"schemaVersion": 1,
"id": "brief-003",
"title": "Project dashboard rollups",
"kernelTags": ["derived-data-views", "temporal-history"],
"productBrief": "A product lead wants a dashboard that rolls task status, blockers, and recent decisions up from individual project notes into one current view.",
"expectedStructuralObservations": [
"Dashboard values are derived from underlying notes and decisions.",
"The system needs evidence for when a rollup was last refreshed."
],
"scriptedUserNotes": [
"If the source note changes, the dashboard should not silently stay stale.",
"Recent decisions should show where they came from."
],
"deferredExpectations": {
"graph": "Later graph fixtures should capture source-to-view derivation edges and evidence anchors.",
"coherence": "Later coherence checks should flag stale projections when source facts change."
}
}
60 changes: 28 additions & 32 deletions docs/architecture/fixture-strategy.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,47 +92,43 @@ Briefs are short, human-readable, and curated. The run artefacts are the heavy d

### Brief fixture format

```yaml
# .brunch-fixtures/briefs/offline-kanban.yaml
id: offline-kanban
title: Offline Kanban Editing
brief: |
We want to build a Kanban tool that engineering teams can use offline.
Multiple people edit the same board. Cards move through workflow states.
Some columns have WIP limits.
persona:
style: collaborative # terse | verbose | collaborative | indecisive
domain_literacy: high # low | medium | high
patience: medium # affects how many follow-ups before frustration
change_mind_probability: 0.1 # per-turn probability of revising an earlier answer
expected_kernels:
- state_lifecycle
- containment_topology
- concurrency_collaboration
- resource_accounting
- derived_data_views
- temporal_history
expected_entity_coverage:
intent: [requirement, assumption, invariant, decision, example]
oracle: [check, validation_method]
known_branch_points:
- "What should happen on offline-edit conflict?"
known_invalidations: []
`.brunch-fixtures/briefs/brief-002-state-lifecycle.json`:

```json
{
"schemaVersion": 1,
"id": "brief-002",
"title": "Approval workflow for vendor invoices",
"kernelTags": ["state-lifecycle", "authority-capability"],
"productBrief": "A finance team needs invoices to move from draft to submitted to approved or rejected. Only budget owners can approve, and rejected invoices can be revised and resubmitted.",
"expectedStructuralObservations": [
"Invoice states and legal transitions must be explicit.",
"Approval authority depends on the budget owner role."
],
"scriptedUserNotes": [
"Rejected invoices are not terminal; they can go back to draft.",
"Approved invoices should not be edited without reopening the workflow."
],
"deferredExpectations": {
"graph": "Later graph fixtures should capture lifecycle states, transitions, and authority predicates.",
"coherence": "Later coherence checks should flag contradictory terminality claims."
}
}
```

### Starter set (seven briefs)

| # | Brief | Active kernels (expected) | Stretches |
| --- | --- | --- | --- |
| 1 | **Offline Kanban** | State/lifecycle, containment, concurrency, resource accounting, derived data, temporal | Kernel doc's flagship; broad behavioral coverage |
| 2 | **Role-based document sharing** | Identity, authority, containment, temporal (revocation), observability, change/migration | Authority cascades; nested inheritance |
| 3 | **Subscription billing** | Resource accounting, state/lifecycle, transactions, external effects, error/recovery, temporal | Transaction + external-effects boundary; assurance-level pressure |
| 1 | **Team knowledge cards** | Identity/reference, containment topology | Stable identity versus mutable titles; links that must survive renames |
| 2 | **Approval workflow for vendor invoices** | State/lifecycle, authority/capability | Non-terminal rejection, reopening approved work, role-gated transitions |
| 3 | **Project dashboard rollups** | Derived-data views, temporal history | Stale projections, source evidence for rollups and decisions |
| 4 | **Calendar scheduling with notifications** | Concurrency (overlap), authority, external effects, error/recovery | External-effects + recovery semantics |
| 5 | **Knowledge-graph editor (meta)** | Identity, containment, change/migration, observability, validation | Brunch describing itself; sanity check on the modelling |
| 6 | **Verified sort algorithm** | Validation/normalization, formal properties only | Narrow but stretches `formal_property` requirements, `Obligation` nodes, `proof` / `model_check` validation methods, and assurance-level computation |
| 7 | **"Notion meets Linear meets Slack"** | Forces scope-boundary clarification before any kernel can engage | Adversarial; stresses offer-first interaction and scope-card affordance |

Briefs 1–3 are already worked out in [`BEHAVIORAL_KERNELS.md`](file:///Users/lunelson/Code/hashintel/brunch-next/docs/design/BEHAVIORAL_KERNELS.md); they should be the first three captured.
Briefs #1–#3 are the first curated M1 seeds under `.brunch-fixtures/briefs/`. They are intentionally thin, human-reviewed product briefs for transcript/projection replay, not final evidence that Brunch's elicitation interaction logic or knowledge-flow model is correct.

### Brief #7 expectations — "Notion meets Linear meets Slack"

Expand Down Expand Up @@ -217,8 +213,8 @@ The fixture harness threads through the existing milestone ladder; it does not n

| Milestone | Fixture work |
| --- | --- |
| **M0** (walking skeleton + TUI) | Begin capturing briefs as YAML. Manually-driven runs at the TUI produce first JSONL captures. Briefs cost nothing to write; the longer the library, the more leverage later. |
| **M1** (mode shell: print + rpc) | Stand up the agent-as-user harness against `brunch --mode rpc`. First **replay regression** fixtures land here, asserting transcript reproduction only. Graph plane does not yet exist; assertions are transcript-shaped. |
| **M0** (walking skeleton + TUI) | Begin curating briefs as JSON. Manually-driven runs at the TUI produce first JSONL captures. Briefs cost nothing to write; the longer the library, the more leverage later. |
| **M1** (mode shell: print + rpc) | Stand up the first fixture-capture path against `brunch --mode rpc`. First **replay regression** fixtures land here, asserting transcript reproduction/projection only. Graph plane does not yet exist; assertions are transcript-shaped, and scripted exchange shape should not be treated as final elicitation behavior. |
| **M2** (JSONL session viability) | The captured transcripts *are* the JSONL session files. The fixture library's reproducibility is part of M2's evidence. |
| **M3** (web shell) | The same offer-response fixtures drive the web client through its WebSocket; free coverage of the web shell against known-good runs. |
| **M4** (graph data plane) | Graph snapshots become part of the run-fixture bundle. The first **property regression** assertions land here. |
Expand Down
Loading
Loading