[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-05-23 #34207

2026-05-23T12:48:56Z

github-actions[bot]
Bot May 23, 2026

Executive Summary

Reviewed gh-aw documentation as a Claude Code user (no Copilot, no Copilot CLI). The product does support Claude as a first-class engine, and the path from gh aw add-wizard through engine: claude is functional — but the docs continue to default to Copilot in tone, examples, and a misconfigured setup URL that sends Claude users to the wrong page. Adoption is possible with friction, not blocked.

Key Finding: Claude documentation grew this cycle (7 → 11 literal examples in docs), but Copilot examples grew faster (62 → 76). The bias ratio is roughly the same as last review: ~76% of engine examples are Copilot. Three previously-flagged issues remain unresolved (default engine, dispatcher file, Anthropic URL discrepancy).

Persona Context

Reviewed as a developer who:

✅ Uses GitHub for version control
✅ Uses Claude Code as primary AI assistant
❌ Does NOT use GitHub Copilot
❌ Does NOT use Copilot CLI
❌ Does NOT have a Copilot subscription

Question 1: Onboarding Experience

Can a Claude Code user understand and get started with gh-aw?

Mostly yes. The Quick Start (docs/src/content/docs/setup/quick-start.mdx) and Prerequisites section (line 29) clearly list all four primary engines (Copilot, Claude, Codex, Gemini) as alternative AI accounts. The add-wizard flow (line 70) explicitly prompts the user to pick between engines, and step 4 (line 127) demonstrates engine: claude as a customization. README.md line 40 also names all four engines plainly. This is a strong, neutral entry point.

Specific Issues Found:

Issue 1 — Claude API key URL points to wrong page. docs/src/content/docs/reference/auth.mdx:109 instructs Claude users to *"Create an API key at (platform.claude.com/redacted) That URL is a docs landing page, not a key-creation page. quick-start.mdx:84 and auth.mdx:199 correctly point to console.anthropic.com / console.anthropic.com/settings/keys. A Claude user following auth.mdx step-by-step will not find a key form.
Issue 2 — Default engine in How It Works is Copilot. docs/src/content/docs/introduction/how-they-work.mdx:26 reads "Workflows support GitHub Copilot (default)...". For someone choosing Claude, the "default" phrasing implies friction is required to opt out — when in fact they just set engine: claude.
Issue 3 — gh aw init silently provisions a Copilot-only artifact. cli.md:134 describes gh aw init as creating .github/agents/agentic-workflows.agent.md, and custom-agent-for-aw.mdx:10 explicitly labels these as "prompts that can be used with Copilot, Copilot CLI and VSCode Agent Mode". A Claude-only user runs gh aw init, gets a Copilot agent file checked in, and has no documented signal that this is unused on their setup. The doc page in question doesn't say "this is optional if you don't use Copilot."

Recommended Fixes:

Replace auth.mdx:109 URL with https://console.anthropic.com/settings/keys to match the rest of the docs.
Reframe how-they-work.mdx:26 from "GitHub Copilot (default)" to "GitHub Copilot, Claude by Anthropic, Codex, or Gemini — pick whichever AI account you have."
Add a one-liner to cli.md § init and custom-agent-for-aw.mdx clarifying that the dispatcher agent file is only consumed by Copilot Chat / VSCode Agent Mode and is safely ignored by Claude/Codex/Gemini engines.

Question 2: Inaccessible Features for Non-Copilot Users

What features don't work without Copilot?

Features That Require Copilot (per engines.md:34-46):

engine.agent — custom agent file references (Copilot only)
engine.harness — custom harness script (Copilot only)
max-continuations — autopilot multi-run mode (Copilot only)
BYOK via COPILOT_PROVIDER_* env vars (Copilot only; documented engines.md:201-269)
The agentic-workflows.agent.md dispatcher generated by gh aw init (Copilot Chat / VSCode Agent Mode only)

Features That Work Without Copilot (engine-agnostic):

All tools: edit, bash, github, web-fetch, playwright, cache-memory, repo-memory, qmd, agentic-workflows, mcp-servers, cli-proxy (per tools.md)
All triggers (on:), permissions, safe-outputs, threat-detection, integrity filtering
engine.api-target and engine.bare are explicitly listed as supported across all engines (engines.md:42-43)
tools.timeout / tools.startup-timeout (with engine-specific defaults documented)
max-turns is the only Claude-specific feature; Copilot lacks it
tools.web-search works on Claude via MCP (web-search.md describes this, though only with a Copilot example)

Missing Documentation:

No gh aw init --skip-copilot-agent or equivalent flag documented for users who don't want the dispatcher file.
No paragraph in engines.md or auth.mdx explaining the consequences of choosing each engine — e.g., "Choose Claude if you need max-turns control; choose Copilot if you need custom agents." The "Which engine should I choose?" section (engines.md:26-28) is one sentence and skews favorably toward Copilot's "broadest feature set."
Crush, OpenCode, and Pi appear in engines.md:20-22 but are missing from the auth table (auth.mdx:17-22). A Claude user choosing one of these alternatives can't find its secret docs in the main auth reference.

Question 3: Documentation Gaps and Assumptions

Where does the documentation assume Copilot usage?

Quantified bias (counts from grep -rE "^engine:\s*<id>|^\s*id:\s*<id>" docs/src/content/docs/):

Engine	Doc examples	Files with literal `engine: <id>`
copilot	76	30
claude	11	6
codex	5	—
gemini	1	—
custom	0	0

Copilot accounts for ~76% of engine examples — essentially unchanged from last review (76/93 vs 62/71). Claude grew (+4), but Copilot grew faster (+14).

Copilot-Centric Examples Found In:

docs/src/content/docs/reference/serena.md (4 Copilot, 0 Claude)
docs/src/content/docs/reference/audit.md (4 Copilot, 0 Claude)
docs/src/content/docs/reference/imports.md (12 Copilot, 0 Claude)
docs/src/content/docs/reference/threat-detection.md (5 Copilot, 0 Claude)
docs/src/content/docs/reference/templating.md (1 Copilot, 0 Claude)
docs/src/content/docs/reference/mcp-scripts.md (1 Copilot, 0 Claude)
docs/src/content/docs/reference/web-search.md (1 Copilot, 0 Claude) — despite the engine table noting Claude needs MCP for web-search, the example is Copilot
docs/src/content/docs/reference/qmd.md (1 Copilot, 0 Claude)
docs/src/content/docs/reference/frontmatter.md (1 Copilot, 0 Claude)
docs/src/content/docs/reference/inline-sub-agents.md (1 Copilot, 0 Claude)
docs/src/content/docs/introduction/architecture.mdx:228 (sole config example uses engine: copilot)

Missing Alternative Instructions:

No engine: claude example in imports.md, threat-detection.md, web-search.md, mcp-scripts.md, templating.md, qmd.md, inline-sub-agents.md. Claude users have to infer that those features work for them.
engine: custom exists in supported engine list (engines.md mentions engine.command for custom executables) but has 0 worked examples anywhere in docs or pkg.
Only docs/src/content/docs/reference/network.md shows all four engines side-by-side.

Severity-Categorized Findings

🚫 Critical Blockers (Score: 10/10)

No critical blockers — Claude Code users can complete the Quick Start path and reach a working engine: claude workflow. This has been stable across the last three review cycles.

⚠️ Major Obstacles (Score: 6/10)

Obstacle 1: Claude API key URL on auth.mdx is broken

Impact: Claude user following the canonical Authentication page lands on a docs site instead of a key-creation form. Forces detour to a search engine.

Current State: docs/src/content/docs/reference/auth.mdx:109:

Create an API key at (platform.claude.com/redacted)

Why It's Problematic: Other pages in the same docset (quick-start.mdx:84, auth.mdx:199 troubleshooting block) use console.anthropic.com[/settings/keys]. Inconsistency suggests two authors wrote different sections; the bad URL leaks credibility.

Suggested Fix: Replace with https://console.anthropic.com/settings/keys.

Affected Files: docs/src/content/docs/reference/auth.mdx:109

Obstacle 2: `gh aw init` creates a Copilot-only artifact without disclosure

Impact: Claude-only users get an unused .github/agents/agentic-workflows.agent.md committed to their repo with no documented signal that it's safe to delete.

Current State: cli.md:134 describes init as creating "the dispatcher agent file" without identifying it as Copilot-only. custom-agent-for-aw.mdx:10 is the only place that says these files are for "Copilot, Copilot CLI and VSCode Agent Mode".

Why It's Problematic: Strong opinion-shaped artifact added without consent. A Claude user investigating their .github/agents/ directory has no breadcrumb back to docs explaining what it is or whether they need it.

Suggested Fix: Either (a) make agent-file creation conditional on a chosen Copilot engine, or (b) add --skip-agent flag, or (c) prepend a one-line comment in the generated .agent.md saying "This file enables /agent agentic-workflows in Copilot Chat and VSCode Agent Mode; ignore it if you use a different engine."

Affected Files: docs/src/content/docs/setup/cli.md:134, docs/src/content/docs/reference/custom-agent-for-aw.mdx

Obstacle 3: 11 reference pages have zero Claude examples

Impact: Claude users reading imports.md, threat-detection.md, web-search.md, serena.md, audit.md, templating.md, mcp-scripts.md, qmd.md, frontmatter.md, inline-sub-agents.md, or sandbox.md see only Copilot examples and have to guess whether the feature works for them.

Current State: Counts above. web-search.md is the worst symptom — the engine matrix in engines.md:40 explicitly notes Claude uses MCP for web-search, but the reference page itself shows only Copilot.

Suggested Fix: Sprinkle one engine: claude (or engine-neutral) example into each of the high-traffic reference pages — at minimum imports.md, threat-detection.md, web-search.md, and frontmatter.md.

Affected Files: Listed above under Question 3.

Obstacle 4: Crush / OpenCode / Pi listed as engines but missing from auth table

Impact: Users choosing one of the experimental engines (engines.md:20-22) can't find which secret they need without scrolling back into engines.md itself.

Current State: auth.mdx:17-22 only documents Copilot, Claude, Codex, Gemini secrets. engines.md:20-22 says Crush/OpenCode/Pi default to COPILOT_GITHUB_TOKEN — that's surprising and should be called out in the auth table too.

Suggested Fix: Add three rows to auth.mdx "Which secret do I need?" with cross-reference to engines.md notes about provider-specific overrides.

Affected Files: docs/src/content/docs/reference/auth.mdx:17-22

💡 Minor Confusion Points (Score: 6/10)

Issue 1: how-they-work.mdx:26 calls Copilot "default" — frames Claude/Codex/Gemini as opt-outs rather than first-class choices.
Issue 2: engines.md:24 doubles down on Copilot as default. README.md:40 is more neutral ("pick whichever AI account you already have") — the docs should follow the README's framing.
Issue 3: Sole architecture-page config example is engine: copilot (architecture.mdx:228). A second example with engine: claude would balance the visual.
Issue 4: engine: custom is unreferenced (0 examples across docs and pkg). Either document a worked example or move it to a Coming Soon section.
Issue 5: "Which engine should I choose?" (engines.md:26-28) is one sentence and leads with Copilot's "broadest gh-aw feature set" — reads as marketing for Copilot.
Issue 6: No Claude-specific troubleshooting recipes beyond auth — e.g., what to do when max-turns is exhausted, when MCP gateway times out at Claude's default 60 s tool timeout (tools.md:155).

Engine Comparison Analysis

Available Engines

From docs/src/content/docs/reference/engines.md:14-22:

engine: copilot — strong docs, default, 76 examples
engine: claude — 11 doc examples (up from 7), API key URL inconsistency
engine: codex — 5 doc examples (up from 1)
engine: gemini — 1 doc example
engine: crush — experimental, listed but absent from auth table
engine: opencode — experimental, listed but absent from auth table
engine: pi — experimental, listed but absent from auth table
engine: custom — referenced in supported list, 0 worked examples

Documentation Quality by Engine

Engine	Setup Docs	Examples	Auth Docs	Overall
Copilot	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐
Claude	⭐⭐⭐⭐	⭐⭐⭐	⭐⭐⭐ (broken URL)	⭐⭐⭐
Codex	⭐⭐⭐	⭐⭐	⭐⭐⭐⭐	⭐⭐⭐
Gemini	⭐⭐⭐	⭐	⭐⭐⭐	⭐⭐
Crush/OpenCode/Pi	⭐⭐	⭐	⭐	⭐⭐
Custom	⭐	⭐ (zero)	n/a	⭐

Tool Availability Analysis

Engine-Agnostic Tools (per tools.md):

edit, bash, github, web-fetch, playwright, cache-memory, repo-memory, qmd, agentic-workflows, mcp-servers, cli-proxy, tools.timeout, tools.startup-timeout

Engine-Specific Behavior:

web-search — native on Codex (opt-in), MCP on Claude/Copilot/Gemini per engines.md:40
max-turns — Claude only
max-continuations — Copilot only
engine.agent (custom agent file) — Copilot only
engine.harness (custom harness) — Copilot only
BYOK via COPILOT_PROVIDER_* — Copilot only

Unclear / Undocumented:

Whether tools.cli-proxy (tools.md:129-149) behaves identically across engines — page does not call out per-engine differences.
Whether agentic-workflows: introspection tool (tools.md:116-127) is engine-agnostic — implied yes, not explicit.
Whether qmd: vector search works the same on Claude vs Copilot (no engine note in qmd.md).

Authentication Requirements

Current Documentation Coverage

Quick Start covers authentication for:

✅ Copilot — COPILOT_GITHUB_TOKEN (detailed instructions, link pre-fills PAT scopes)
✅ Claude — ANTHROPIC_API_KEY (covered, but with broken setup URL on auth.mdx:109)
✅ Codex — OPENAI_API_KEY / CODEX_API_KEY fallback (well documented)
✅ Gemini — GEMINI_API_KEY (basic but functional)
❌ Crush, OpenCode, Pi — absent from auth.mdx table
n/a Custom — depends on user's setup

Missing for Claude Users

Single canonical Anthropic Console URL. Currently three URLs in three places:
- auth.mdx:109 → `(platform.claude.com/redacted) (wrong page)
- auth.mdx:199 → https://console.anthropic.com/ (in troubleshooting block)
- quick-start.mdx:84 → https://console.anthropic.com/settings/keys (correct deep link)
A note on CLAUDE_CODE_OAUTH_TOKEN not being supported is good (auth.mdx:121) — keep this.

Secret Names

Engine	Secret	Status
Copilot	`COPILOT_GITHUB_TOKEN`	documented
Claude	`ANTHROPIC_API_KEY`	documented (URL issue)
Codex	`OPENAI_API_KEY` or `CODEX_API_KEY`	documented
Gemini	`GEMINI_API_KEY`	documented
Crush/OpenCode	`COPILOT_GITHUB_TOKEN` default	only in `engines.md`, missing from auth.mdx
Pi	`COPILOT_GITHUB_TOKEN` default, provider-specific if `model:` uses `provider/model`	only in `engines.md`, missing from auth.mdx

Example Workflow Analysis

Workflow Count by Engine (literal `engine: (id)` matches)

Docs:
  engine: copilot - 76 (was 62)
  engine: claude  - 11 (was 7)   ← +4 since last review
  engine: codex   - 5  (was 1)   ← +4 since last review
  engine: gemini  - 1  (was 1)
  engine: custom  - 0  (was 0)

pkg/ (test fixtures & internal workflows):
  engine: copilot - 127 (was 117)
  engine: claude  - 46  (was 41)
  engine: codex   - 15  (was 13)
  engine: gemini  - 1   (was 0)
  engine: custom  - 0   (was 0)

Quality of Examples

Copilot Examples: Comprehensive — covers virtually every feature page. Multiple variations (default, custom agent, harness, BYOK, GHEC, GHES).

Claude Examples: Present and growing (+4 this review), but concentrated in 6 files (faq.md, network.md, frontmatter-hash-specification.md, research-plan-assign-ops.md, cli.md, quick-start.mdx). Critical pages (threat-detection.md, imports.md, web-search.md, mcp-scripts.md) still lack any Claude example.

Recommended Actions

Priority 1: Critical Documentation Fixes

Fix Claude API key URL — Replace (platform.claude.com/redacted) with https://console.anthropic.com/settings/keys`. File: docs/src/content/docs/reference/auth.mdx:109.
Add Copilot disclosure to gh aw init — Note in cli.md:134 and custom-agent-for-aw.mdx that the generated agentic-workflows.agent.md is only consumed by Copilot/VSCode Agent Mode. Alternatively, add a --skip-agent flag.
Add Crush/OpenCode/Pi rows to auth table — File: docs/src/content/docs/reference/auth.mdx:17-22.

Priority 2: Major Improvements

Add Claude examples to high-traffic reference pages — imports.md, threat-detection.md, web-search.md, frontmatter.md at minimum. File: those reference pages.
Reframe "default engine" language — how-they-work.mdx:26 and engines.md:24 should follow the README's neutral framing rather than promoting Copilot as default. File: docs/src/content/docs/introduction/how-they-work.mdx:26, docs/src/content/docs/reference/engines.md:24.
Expand "Which engine should I choose?" — One sentence is not enough decision support. Suggest a 3-row decision table: pick Claude for max-turns/long reasoning sessions, pick Copilot for custom agents/autopilot, pick Codex/Gemini for budget reasons. File: docs/src/content/docs/reference/engines.md:26-28.

Priority 3: Nice-to-Have Enhancements

Worked engine: custom example — Or remove it from the supported list until docs catch up.
Claude-specific troubleshooting recipes — max-turns exhaustion, MCP gateway tool timeout (default 60 s for Claude vs 120 s for Codex per tools.md:155).
Engine-balanced architecture diagrams — architecture.mdx:228 config sample is Copilot; add a Claude variant next to it.

Positive Findings

✅ README.md:40 leads with engine-neutral framing ("pick whichever AI account you already have"). Strong setup.
✅ Quick Start add-wizard flow explicitly prompts engine choice (quick-start.mdx:70). User isn't forced into Copilot.
✅ engines.md:14-22 and engines.md:34-46 give clear, tabular per-engine feature parity — no hidden assumptions.
✅ Claude --bare mode is documented (engines.md:325-336) including the engine-specific mechanism.
✅ Authentication explicitly addresses CLAUDE_CODE_OAUTH_TOKEN (not supported) — saves Claude Code users from a common dead-end.
✅ Codex examples grew +4 this cycle (1 → 5), showing maintainers are diversifying.
✅ Claude examples grew +4 this cycle (7 → 11) — biggest single-cycle gain so far.
✅ gh aw new --engine claude works (cli.md:202-216) and Claude shows up in the generated template.
✅ gh aw secrets bootstrap --engine claude is supported (cli.md:245).

Conclusion

Can Claude Code Users Successfully Adopt gh-aw?

Answer: Yes, with friction.

Reasoning: A Claude-only developer can complete gh aw add-wizard, pick engine: claude, set ANTHROPIC_API_KEY (assuming they ignore the broken URL on auth.mdx and use Quick Start's correct one), and run their first workflow. The CLI does not block them. However, they will encounter unexpected Copilot-only artifacts in gh aw init, find empty Claude examples on multiple reference pages, hit a broken Anthropic URL on the auth page, and read "Copilot (default)" framing throughout the introduction docs. None of these prevent adoption; together they make the experience feel like Claude is a second-class citizen even though the engine itself is fully supported.

Overall Assessment Score: 6/10

Breakdown:

Clarity for non-Copilot users: 6/10
Claude engine documentation: 7/10 (up from prior trend due to example growth, held back by broken URL)
Alternative approaches provided: 6/10
Engine parity: 5/10 (Copilot still owns engine.agent, engine.harness, max-continuations, BYOK)

Trend vs Prior Run (2026-05-22)

Metric	Prior	Current	Δ
Overall score	6/10	6/10	unchanged
Claude doc examples	7	11	+4
Copilot doc examples	62	76	+14
Codex doc examples	1	5	+4
Custom engine examples	0	0	unchanged
Major obstacles	4	4	unchanged
Claude auth URL inconsistency	yes	yes	unresolved
Default engine = Copilot	yes	yes	unresolved
Dispatcher agent Copilot-only warning	none	none	unresolved

Next Steps

Land the auth.mdx URL fix — single-line change, immediate quality win.
Disclose the Copilot-only nature of gh aw init's dispatcher artifact — multi-line doc change.
Sprinkle Claude examples through the 4 highest-traffic reference pages (imports.md, threat-detection.md, web-search.md, frontmatter.md).
Reframe "Copilot (default)" language in 2 introduction pages.

These four changes would lift the overall score to 7/10 and substantially close the perceived gap for Claude Code users without requiring any product changes.

Appendix: Files Reviewed

Complete List of Documentation Files Analyzed

README.md
docs/src/content/docs/setup/quick-start.mdx
docs/src/content/docs/introduction/how-they-work.mdx
docs/src/content/docs/introduction/architecture.mdx
docs/src/content/docs/reference/tools.md
docs/src/content/docs/setup/cli.md
docs/src/content/docs/reference/engines.md
docs/src/content/docs/reference/auth.mdx
docs/src/content/docs/reference/custom-agent-for-aw.mdx
Spot checks: docs/src/content/docs/reference/{serena,audit,imports,threat-detection,web-search,sandbox,templating,mcp-scripts,qmd,frontmatter,inline-sub-agents,copilot-custom-agents}.md

References:

§26332951547

Generated by 📝 Claude Code User Documentation Review · ● 14.7M · ◷

expires on May 24, 2026, 12:48 PM UTC

2026-05-24T12:52:24Z

github-actions[bot]
Bot May 24, 2026
Author

This discussion has been marked as outdated by Claude Code User Documentation Review.

A newer discussion is available at Discussion #34426.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-05-23 #34207

Uh oh!

{{title}}

Uh oh!

Can a Claude Code user understand and get started with gh-aw?

What features don't work without Copilot?

Where does the documentation assume Copilot usage?

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-05-23 #34207

Uh oh!

github-actions[bot] Bot May 23, 2026

Executive Summary

Persona Context

Can a Claude Code user understand and get started with gh-aw?

What features don't work without Copilot?

Where does the documentation assume Copilot usage?

Severity-Categorized Findings

🚫 Critical Blockers (Score: 10/10)

⚠️ Major Obstacles (Score: 6/10)

💡 Minor Confusion Points (Score: 6/10)

Engine Comparison Analysis

Available Engines

Documentation Quality by Engine

Tool Availability Analysis

Authentication Requirements

Current Documentation Coverage

Missing for Claude Users

Secret Names

Example Workflow Analysis

Workflow Count by Engine (literal engine: (id) matches)

Quality of Examples

Recommended Actions

Priority 1: Critical Documentation Fixes

Priority 2: Major Improvements

Priority 3: Nice-to-Have Enhancements

Positive Findings

Conclusion

Can Claude Code Users Successfully Adopt gh-aw?

Overall Assessment Score: 6/10

Trend vs Prior Run (2026-05-22)

Next Steps

Appendix: Files Reviewed

Replies: 1 comment

Uh oh!

github-actions[bot] Bot May 24, 2026 Author

github-actions[bot]
Bot May 23, 2026

Workflow Count by Engine (literal `engine: (id)` matches)

github-actions[bot]
Bot May 24, 2026
Author