feat(qwen-story): 8-beat E2E narrative + pmat bug-hunt + daily cron#1875
Merged
Conversation
…ly cron
Adds an end-to-end "Qwen story" that exercises every core apr command
group against the Qwen scale ladder (0.5B → 1.5B → 7B → 30B-MoE). The
story is the single canonical demo in README.md AND a regression gate
via runnable script + falsification contract + nightly cron.
## Beats
1. **Discover** (Registry) — pull, list
2. **Trust** (QA) — qa, validate, lint
3. **Explore** (Inspection) — inspect, tensors, tree
4. **Adapt** (Model ops) — export, diff, convert/quantize
5. **Use** (Inference) — run, chat, code
6. **Serve** (REST) — serve run + curl /v1/chat/completions OpenAI-compat
7. **Operate** (Profiling) — profile, gpu, serve plan (7B Q4K GGUF)
8. **Scale** (MoE) — inspect, tensors on 30B-MoE qwen3moe
## Pmat bug-hunt layer
When run with `PMAT_HUNT=1` (default), each beat emits a structured
manifest of high-risk untested code in the command-handler modules it
just exercised:
-- pmat bug-hunt manifest (run chat code) --
gap crates/apr-cli/src/commands/run.rs:resolve_model_alias (impact=42.3)
churn crates/apr-cli/src/commands/code.rs:dispatch_agent (commits=11)
fault crates/aprender-serve/src/api/cuda_chat_backend.rs:try_qwen3_moe (unwrap)
The nightly cron uploads this manifest as an artifact, compares against
the previous successful run, and opens (or comments on) a tracking issue
when growth exceeds 5 lines — so untested branches in command handlers
can't accumulate quietly.
## Files
- `scripts/qwen-story.sh` (336 LOC) — runnable story with proper exit-code
capture (`OUT=$(cmd); EC=$?` everywhere; no pipe-then-`$?` per memory rule)
- `contracts/qwen-story-v1.yaml` — 3 equations + 8 falsifiers, all PASS
locally (script exists+executable, 8 beats, run_cmd helper, pmat_hunt
per beat, README link, daily cron file, bashrs clean, Beat 7 skips
`apr qa` on 7B Q4K due to #1864)
- `README.md` — new `## A Qwen story` section replacing the flat
`## CLI examples` block. Fixes two README bugs surfaced during dogfood:
`apr profile --roofline` (no such flag; just `apr profile <file>`)
and `apr bench --assert-tps` (flag is on `apr qa`, not `bench`).
- `.github/workflows/qwen-story-daily.yml` — self-hosted GPU runner,
04:17 UTC cron + workflow_dispatch, uploads pmat manifest + story log
artifacts, files tracking issue when story regresses or manifest grows.
## Verification
$ bash scripts/qwen-story.sh # local smoke
-- Beat 1: Discover (Registry) --
✓ PASS B1 list
-- Beat 2: Trust (QA gates) --
✓ PASS B2 apr qa
✗ FAIL B2 apr validate --quality - exit=5 (after #1866 fix this should be 0)
-- Beat 3: Explore (Inspection) --
✓ PASS B3 apr inspect --json (arch=qwen2)
✓ PASS B3 apr tensors --json (339 tensors)
✓ PASS B3 apr tree
-- Beat 4: Adapt (Model ops) --
✗ FAIL B4 apr export - PANIC (exit=101) - #1865 regression
-- Beat 5: Use (Inference) --
✓ PASS B5 apr run (Rust code completion)
✓ PASS B5 apr code -p
-- Beat 6: Serve (REST API) --
✓ PASS B6 apr serve run (port=22915)
✓ PASS B6 /v1/chat/completions (got OK...)
-- Beat 7: Operate (Profiling) --
✓ PASS B7 apr profile
✓ PASS B7 apr gpu --json
✓ PASS B7 apr serve plan -- 7B VRAM budget
-- Beat 8: Scale (MoE introspection) --
✓ PASS B8 apr inspect --json (arch=qwen3moe)
✓ PASS B8 apr tensors --json (579 tensors)
14 PASS / 2 FAIL / 0 SKIP
The 2 FAILs are EXPECTED until the in-flight fixes land:
- B2 validate --quality: closed by #1870
- B4 export panic: closed by #1868
Once those PRs merge, this story will be 16 PASS / 0 FAIL / 0 SKIP on a
host with all 4 Qwen models cached.
## Follow-up
A separate PR will add `/dogfood` Gate 18 that invokes this script (kept
separate to avoid conflict with PR #1872 which is already adding Gates
13-17 to the dogfood skill).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
noahgift
added a commit
that referenced
this pull request
May 22, 2026
…itstream-io) (#1878) `cargo deny check advisories` started failing on every PR (and on main) 2026-05-22 with: error[unmaintained]: core2 is unmaintained, all versions yanked ├ ID: RUSTSEC-2026-0105 ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0105 The dep is pulled in transitively via `bitstream-io` (image/media decoding stack — `cargo tree` shows `bitstream-io v4.9.0 → core2 v0.4.0`). No first-party use; no drop-in replacement until upstream `bitstream-io` migrates off core2. This commit unblocks the in-flight PR cascade (#1867 #1868 #1870 #1873 #1875 #1876) which all failed CI's `ci / lint` step on this advisory. The deny entry is structured per the existing pattern in this file (id + human reason mentioning the transitive path) so revisiting the ignore in 6-12 months is straightforward. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A canonical end-to-end "Qwen story" that doubles as a regression gate. Eight beats, one narrative, every core command group — anchored on the Qwen scale ladder (0.5B safetensors → 30B-MoE GGUF) so the story exercises real production scales, not toy fixtures.
What ships
scripts/qwen-story.sh(336 LOC)OUT=$(cmd); EC=$?(no pipe-then-$?)contracts/qwen-story-v1.yamlREADME.md§ "A Qwen story"## CLI examplesblock; fixes 2 broken README examples.github/workflows/qwen-story-daily.ymlThe 8 beats
apr pull,apr listapr qa,apr validate,apr lintapr inspect,apr tensors,apr treeapr export,apr diff,apr convertapr run,apr chat,apr code -papr serve run+ curl/v1/chat/completionsapr profile,apr gpu,apr serve plan(on 7B Q4K)apr inspect,apr tensors(on 30B-MoEqwen3moe)Pmat bug-hunt layer
When
PMAT_HUNT=1(default), each beat emits a structured manifest of high-risk untested code in the command-handler modules it just exercised:The nightly cron diffs this manifest against the prior successful run and opens (or comments on) a tracking issue when growth exceeds 5 lines — so untested branches in command handlers can't accumulate quietly.
README example fixes (caught while building the script)
The story script revealed two more broken examples in
README.md ## CLI examples:apr profile model.gguf --roofline→ there is no--rooflineflag; correct usage is justapr profile model.gguf(the description already says "Deep profiling with Roofline analysis")apr bench model.gguf --assert-tps 100→ the--assert-tpsflag lives onapr qa, notapr benchThe new
## A Qwen storysection replaces this block with verified invocations from the runnable script.Local smoke test
Both failures are expected until the in-flight fixes land. The story is wired correctly — it catches the panic, it catches the validate threshold, it catches each regression class. Once #1868 + #1870 merge, the story will be 16/0/0 on a host with all 4 Qwen models cached.
Falsifier sweep
All 8 falsifiers in
qwen-story-v1.yamlPASS:Follow-up
A small follow-up PR will add
/dogfoodGate 18 that invokesscripts/qwen-story.sh— kept separate to avoid conflict with #1872 which is already adding Gates 13-17 to the dogfood skill.Test plan
bashrs lint scripts/qwen-story.sh— 0 errorsfeedback_self_hosted_only.md🤖 Generated with Claude Code