docs: backfill apm-usage and consolidate registry guides (v0.14->v0.15 drift sweep)#1511
Merged
Merged
Conversation
…5 drift sweep) Holistic docs-sync retrospective on the v0.14.0->v0.15.0 release window flagged 23 of 39 user-impact PRs as docs-debt: 7 Rule 4 violations (apm-usage/ skipped) plus 16 silent-drift PRs. This PR closes the highest-priority gaps (P0/P1 from the retrospective) in one sweep. Backfills (apm-usage/ training corpus): - dependencies.md: registry-sourced APM dep object form (#1471) - authentication.md: APM_REGISTRY_TOKEN_{NAME} precedence (#1471) - governance.md: registry_source + allow_non_registry policy (#1471) - package-authoring.md: apm publish workflow (#1471) and project-scope hook command path semantics (#1396) - commands.md: apm publish entry (#1471), apm config transport keys (#1308), apm compile live-reload + --clean --watch warning (#1403), Claude Code instruction dedup (#1146), MCP env-var placeholder resolution (#1277), AppLocker/WDAC staged-install diagnostic (#1390) Structural fix (per docs-impact-architect verdict): - Merge guides/private-registries.md INTO guides/registries.md with progressive disclosure (public -> private -> per-dep routing -> enterprise link). Adds Starlight redirect for the old slug, patches 5 cross-references across consumer/, reference/cli/. Editorial fixes (per editorial-owner sweep): - integrations/copilot-app.md (#1431): lead with user value before WS-IPC/SQLite mechanics; add 'restart the Copilot App once' troubleshooting hint - producer/compile.md: dedup the Claude Code instruction dedup explanation (was stated twice) - enterprise/security.md: reframe defensive memo voice ('do not call this X') to user voice ('here is what we provide / here is what we don't') Method: docs-sync skill end-to-end. 5-panelist fan-out plus CDO synthesis. Every CLI claim in the apm-usage adds was verified against the live 'apm <verb> --help' surface (S7 tool bridge). Out of scope (tracked as P1 follow-up): backfilling docs for the 16 silent-drift PRs grouped by subsystem (MCP, install, compile, auth). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Wave-batched grounding audit across 55 high-risk pages (CLI ref x27, schemas/specs x10, consumer ramp x12, onboarding x6). Each page's factual claims (flags, env vars, exit codes, schema fields, file paths, code links) was extracted and verified against current src/apm_cli/ and 'apm <verb> --help' output via S7 tool-bridge. Fixes applied (14 files): CLI reference: - pack.md: add --check-versions, --check-clean flags + exit codes 3, 4 - targets.md: expand copilot detection signals (5, not 1) - experimental.md: add copilot-app, marketplace-authoring, registries - install.md: dedup duplicate '## Exit codes' + '## Notes' sections Schemas / specs: - lockfile-spec.md: expand package_type enum to full 6-value list - manifest-schema.md: document plural 'targets:' alias (#1335) - environment-variables.md: add APM_BROAD_FETCH_DEPTH, APM_COPILOT_APP_DB - package-types.md: add 5th layout (hook_package, hooks/*.json only) Consumer ramp: - install-mcp-servers.md: fix stale code citation + 'Or' -> 'And' - private-and-org-packages.md: drop nonexistent BITBUCKET_APM_PAT Onboarding (6 broken navigation links, 4 files): - quickstart.mdx, getting-started/installation.md, getting-started/first-package.md, getting-started/migration.md: repoint self-loops and dead routes to actual page paths Process: dispatched as 6 parallel grounding-verifier agents (general- purpose) across disjoint page scopes; each agent had edit authority on its scope and applied surgical fixes inline. Reusable pattern via the docs-corpus-audit sibling skill design (PANEL + WAVE EXECUTION + S7 verifier fan-out, see files/docs-corpus-audit-design.md). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collaborator
Author
Wave 2: full-corpus regrounding auditFolded into this PR. Distinct from the v0.14 -> v0.15 PR-replay retrospective in the original description: this pass treats the corpus as a whole and asks 'do current docs ground out true against current code, regardless of which PR shipped them?' MethodDesigned via the Design artifact persisted in session: Scope55 high-risk pages audited (49% of 112-page corpus):
Deferred (57 pages, low-drift risk per genesis triage): producer/ (16), enterprise non-policy (14), concepts/ (7), contributing/ + troubleshooting/ + others (20). Pingable for wave 3 if you want full coverage. Drift caught and fixed (14 files in commit
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR performs a docs drift sweep across the v0.14.x -> v0.15.x window by (1) backfilling the AI-facing apm-usage/ training corpus for recently shipped CLI/registry features and (2) consolidating registry documentation into a single guide with redirects and updated cross-links.
Changes:
- Backfill
packages/apm-guide/.apm/skills/apm-usage/with registry publish/install/auth/policy details and recent CLI behaviors. - Consolidate registry documentation by merging
private-registries.mdintoregistries.md, adding a redirect, and updating inbound links. - Apply small editorial improvements across Copilot App integration docs, compile docs, and enterprise security messaging.
Show a summary per file
| File | Description |
|---|---|
| packages/apm-guide/.apm/skills/apm-usage/package-authoring.md | Adds project-scope hook-path semantics + apm publish (registry) workflow guidance. |
| packages/apm-guide/.apm/skills/apm-usage/governance.md | Documents registry_source policy fields and enforcement behavior. |
| packages/apm-guide/.apm/skills/apm-usage/dependencies.md | Documents registry-sourced dependency shapes, routing rules, and lockfile notes. |
| packages/apm-guide/.apm/skills/apm-usage/commands.md | Backfills CLI surface changes (compile watch behavior, publish, config transport keys, MCP placeholder resolution notes). |
| packages/apm-guide/.apm/skills/apm-usage/authentication.md | Adds registry token chain (APM_REGISTRY_TOKEN_{NAME} / basic auth) and precedence documentation. |
| docs/src/content/docs/reference/cli/install.md | Updates guide links to the consolidated Registries guide. |
| docs/src/content/docs/reference/cli/config.md | Updates guide links to the consolidated Registries guide. |
| docs/src/content/docs/producer/compile.md | Removes duplicated Claude deduplication explanation. |
| docs/src/content/docs/integrations/copilot-app.md | Reframes “what you get” first and moves implementation detail into a note; adds restart hint. |
| docs/src/content/docs/guides/registries.md | Consolidates public/private registry guidance, routing, publish workflow, and policy references into one page. |
| docs/src/content/docs/guides/private-registries.md | Deleted (content consolidated into registries.md). |
| docs/src/content/docs/enterprise/security.md | Reframes registries security posture messaging in a more user-facing tone. |
| docs/src/content/docs/consumer/manage-dependencies.md | Updates guide link to the consolidated Registries page. |
| docs/astro.config.mjs | Adds redirect for the removed /guides/private-registries slug and updates sidebar items. |
Copilot's findings
- Files reviewed: 27/27 changed files
- Comments generated: 7
| fails loudly at runtime, while project-scope leaves the variable in | ||
| place to avoid baking the installer's prefix into committed config. To | ||
| clean up an older repo whose committed configs still carry absolutized | ||
| paths, re-run `apm install` -- the new compile rewrites them back to |
Comment on lines
+141
to
+143
| # String shorthand -- requires a default registry; always needs a ref | ||
| - acme/foo#^1.2.3 # semver range -> default registry | ||
| - acme/bar#stable # opaque label -> default registry |
Comment on lines
+187
to
+188
| opportunistically bumps to `lockfile_version: "2"` only when at least one | ||
| registry dep is present. |
Comment on lines
234
to
+239
| | Selector | Behavior | | ||
| |---|---| | ||
| | `1.0.0`, `v1.4.2` | Exact version string — matched literally against the registry catalogue | | ||
| | `^1.0.0`, `~1.2.3`, `>=1.2.0 <2.0.0` | Semver range — APM picks the highest matching version | | ||
| | `stable`, `latest` | Opaque label — matched literally; server decides what it resolves to | | ||
| | unset (no `#<ref>`) | Rejected — a version is always required for registry-routed dependencies | | ||
|
|
||
| ```yaml | ||
| dependencies: | ||
| apm: | ||
| - acme/foo#^1.2.3 # semver range → registry | ||
| - acme/bar#stable # opaque label → registry | ||
| - acme/baz#v2.0.0 # v-prefixed → registry | ||
| - git: https://github.com/acme/qux.git # explicit Git pin | ||
| ref: main | ||
| ``` | ||
|
|
||
| Registry-routed deps are byte-for-byte reproducible via `resolved_hash`; | ||
| Git-routed deps are SHA-reproducible via `resolved_commit`. | ||
| | `1.0.0`, `v1.4.2` | Exact version string -- matched literally against the registry catalogue | | ||
| | `^1.0.0`, `~1.2.3`, `>=1.2.0 <2.0.0` | Semver range -- APM picks the highest matching version | | ||
| | `stable`, `latest` | Opaque label -- matched literally; server decides what it resolves to | | ||
| | unset (no `#<ref>`) | Rejected -- a version is always required for registry-routed dependencies | |
| ## 4. What gets recorded in the lockfile | ||
|
|
||
| Registry-sourced dependencies add four fields to their lockfile entry: `source: registry`, `version`, `resolved_url`, and `resolved_hash` (sha256 of the archive bytes). The lockfile bumps to `lockfile_version: "2"` opportunistically — only when at least one registry dep is present. Projects that never opt into a registry keep `lockfile_version: "1"` forever, even on a newer client. | ||
| Registry-sourced dependencies add four fields to their lockfile entry: `source: registry`, `version`, `resolved_url`, and `resolved_hash` (sha256 of the archive bytes). The lockfile bumps to `lockfile_version: "2"` opportunistically -- only when at least one registry dep is present. Projects that never opt into a registry keep `lockfile_version: "1"` forever, even on a newer client. |
Comment on lines
+181
to
+183
| Tokens MUST NOT appear in `apm.yml` or `apm-policy.yml` -- a `token:` | ||
| field in repo YAML is rejected at parse time (token trap). Store tokens | ||
| in env vars or `~/.apm/config.json` only. |
Comment on lines
+189
to
+193
| dependencies: | ||
| apm: | ||
| - acme/foo#^1.2.3 # semver range → resolved via jf-skills | ||
| - acme/bar#stable # any ref, including opaque labels → resolved via jf-skills | ||
| - acme/foo#^1.2.3 # semver range -> corp-main | ||
| - acme/bar#stable # opaque label -> corp-main | ||
| - acme/baz#v2.0.0 # v-prefixed -> corp-main |
Second sweep of the regrounding audit. Covers the 57 pages deferred in
wave 2: producer/ (15), enterprise/ (15), concepts/ (6), integrations/
(7), troubleshooting/ (7), contributing/ (3), reference tail (3), 404.
Process: 6 parallel grounding-verifier agents on disjoint scopes; each
agent extracts factual claims, S7-verifies against current source
('apm <verb> --help' + grep src/apm_cli/), and applies surgical edits
inline. Same pattern as wave 2 (PANEL + WAVE EXECUTION + S7 verifier
fan-out). Orchestrator post-pass swept three cross-corpus broken-link
patterns the per-scope agents could not fix alone.
High-signal factual fixes:
enterprise/governance-guide.md:
- --output-file -> --output (real flag is --output / -o)
- 7+17 check count -> 8+17 (8 baseline checks, not 7)
enterprise/apm-policy.md:
- '16 of 22 checks' -> '17 of 25 checks' (phantom counts)
- conflated --no-policy (install-only) with APM_POLICY_DISABLE (env)
enterprise/apm-policy-getting-started.md:
- dropped 'apm compile' from list of commands that run policy
(compile enforces zero policy per governance-overview.md L57)
enterprise/policy-reference.md:
- compilation.target.allow: added copilot, gemini, vscode, windsurf,
agent-skills (only 5 of 9 runtimes were listed)
enterprise/registry-proxy.md:
- 'apm marketplace add --branch main' -> '--ref main' (no --branch flag)
enterprise/security-and-supply-chain.md:
- 3 stale source line-number citations corrected
producer/author-primitives/index.md:
- legacy '.hook.md' extension -> '.json' (hook_integrator scans JSON)
- removed nonexistent '.apm/commands/' subdirectory from layout example
concepts/lifecycle.md:
- 4 reference-page links all pointed at install/ (copy-paste)
Cross-corpus IA-reshuffle dead-link cleanup (orchestrator pass):
- introduction/* -> concepts/* (4 links across 2 files)
- guides/ci-policy-setup/ -> enterprise/enforce-in-ci/ (8 links, 4 files)
- guides/pack-distribute/ -> producer/pack-a-bundle/ (5 links, 4 files)
- guides/dependencies/ -> consumer/manage-dependencies/ (1 link)
- guides/agent-workflows/ -> contextual canonical (3 links, 3 files)
- guides/install-and-use/mcp-servers/ -> consumer/install-mcp-servers/ (3)
- guides/compilation/ -> producer/compile/ (1)
- guides/prompts/ -> producer/author-primitives/prompts/ (2)
- guides/drift-detection/ -> enterprise/drift-detection/ (1)
enterprise/security.md side-fix:
- 'apm unpack scheduled for removal in v0.14' -> drop version target
(APM is 0.15.0 and unpack still ships marked DEPRECATED in --help).
Upstream remediation (refresh deprecation timeline in source or
remove the shim) tracked outside this PR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collaborator
Author
Wave 3: deferred-corpus pass + IA-reshuffle dead-link cleanupFolded as commit Coverage now complete112 / 112 pages audited (waves 1-3 combined). Every Wave-3 fixes (21 files, +64/-62)High-signal factual drift caught:
Cross-corpus IA-reshuffle cleanup (orchestrator pass):
This dead-link pattern points at an unannounced TOC reshape sometime in v0.14 / v0.15: the corpus moved from a Final PR shape
Net: 49 files, +660/-611. Squash-friendly. Process costWave 3: ~6 LLM dispatches, ~5 min wall-time (fully parallel). Plus ~10 min orchestrator-side IA dead-link sweep using grep + the persisted IA mapping from wave-2 Total across all 3 waves: ~25 LLM dispatches for 112 pages audited, 49 surgical fixes. The Items intentionally NOT in this PR
|
Closes the three items deferred from the v0.14->v0.15 docs-sync retrospective and the full-corpus regrounding waves (commits 4f00c2b, 242bb9e, b80da69): 1. apm unpack source-side deprecation timeline - src/apm_cli/commands/pack.py: 'will be removed in v0.14' -> 'will be removed in a future release'. Current version is 0.15.0; the v0.14 target had already passed. Docs were softened in wave 3; this mirrors the choice in source. - CHANGELOG.md: [Unreleased] Fixed entry. 2. Bucket-C silent-drift backfills (20 PRs, parallel triage) - 3 grounding-verifier subagents reviewed 20 of the 21 bucket-C PRs (#1477 excluded as test-flake fix, no doc surface). Verdicts: 17 ALREADY_COVERED or NO_DOC_SURFACE (verified honestly against wave 2-3 backfills, not manufactured), 3 BACKFILLED: - #1385 SSH dep user-from-URL: added supported-form row in docs/src/content/docs/consumer/manage-dependencies.md and bullet in apm-usage/dependencies.md. - #1434 Copilot App schema range [13,15] + warn-not-fail: rewrote the 'Schema compatibility' paragraph in docs/src/content/docs/integrations/copilot-app.md (was factually wrong, claimed [13,13] hard-fail). - #1440 Copilot file-based detection signals: added the four .github/{instructions,agents,prompts,hooks}/ directories to the canonical-signals list in troubleshooting/compile-zero-output-warning.md and to the apm-usage commands.md + package-authoring.md auto-detect rules. 3. docs-corpus-audit skill extracted - .apm/skills/docs-corpus-audit/SKILL.md: first-class skill module emitted from the genesis design artifact used to drive waves 2 and 3. Pattern: PANEL + WAVE EXECUTION + S7 verification. Wave-batched (scales as O(waves), not O(claims)), disjoint page ownership (no merge conflicts), orchestrator post-pass for cross-corpus drift patterns invisible to per-scope agents. - references/design-handoff.md: full design artifact preserved for future maintainers. - Sibling to docs-sync (per-PR), not a replacement. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collaborator
Author
Deferred items closed (commit ec35069)All three items deferred from prior waves are now in this PR. No more deferrals. 1.
|
| Phase | Files | LLM dispatches | Commits |
|---|---|---|---|
| Wave 1 (v0.14->v0.15 PR replay) | 14 | ~5 | 4f00c2b |
| Wave 2 (high-risk regrounding) | 14 | ~6 | 242bb9e |
| Wave 3 (corpus tail + IA dead-links) | 21 | ~6 | b80da69 |
| Closures (unpack + Bucket-C + skill) | 10 | ~3 | ec35069 |
| Total | 59 | ~20 | 4 |
112/112 pages audited end-to-end. Skill is now reusable for next release.
Two classes of fix on PR #1511: 1. Deploy Docs CI -- starlight-links-validator failure (2 dead links) - getting-started/first-package.md:18 and quickstart.mdx:40 used absolute /apm/getting-started/installation/ paths introduced in wave 2 (242bb9e). Converted to relative paths matching the surrounding link convention. - Verified with local 'npm run build' under docs/: 'All internal links are valid.' 2. Copilot PR review -- 7 inline factual accuracy comments, all verified against source and addressed: - apm-usage/package-authoring.md: hook path rewrite is performed by 'apm install' (hook integrator pass), not 'apm compile'. - apm-usage/dependencies.md + docs/guides/registries.md: registry resolver requires semver per apm_cli/deps/registry/semver.py (is_semver_range gate). Removed examples implying opaque labels (#stable, #v2.0.0, 'latest') route through a registry; updated selector tables to flag non-semver refs as rejected for registry sources. - apm-usage/dependencies.md + docs/guides/registries.md: lockfile_version: '2' promotion triggers on registry deps OR git-source semver resolution fields (constraint / resolved_tag / resolved_at per lockfile.py:_needs_v2, issue #1488), not just registry deps. - apm-usage/authentication.md: 'token:' in apm-policy.yml is not parse-rejected, only surfaces as an 'Unknown top-level policy key' warning per policy/parser.py. Still discouraged (leaks to repo), but the rejection mechanism is different from apm.yml. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collaborator
Author
Copilot review + Deploy Docs CI (commit 270118e)Deploy Docs failure (root cause + fix)starlight-links-validator caught 2 dead links introduced by wave 2:
Copilot inline review -- 7/7 addressedVerified each against source before editing:
All 7 review comments map to factually-incorrect prose that wave 2-3 inherited or introduced. Source verification confirmed each before editing -- no manufactured coverage. |
Round-trip assessment found the original SKILL.md draft violated
genesis SoC in 7 ways:
1. Invented inline 'grounding-verifier' persona instead of composing
shared agent personas (python-architect for S7, doc-writer for
edits). R3 EXTRACT in reverse.
2. Subagent prompt template inlined in SKILL body (~40 lines that
belong in assets/).
3. IA-reshuffle grep patterns hard-coded in body as bash heredoc --
the patterns rot per release and belong in scripts/ with --help
and a versioned update cadence.
4. PHANTOM DEPENDENCY on docs-sync's substrate (.apm/docs-index.yml,
personas, panelist-return-schema, the apm-usage Rule-4 corpus)
never declared via tool-call probes -- A9 SUPERVISED EXECUTION
violation per genesis Step 7b.
5. Missing A8 ALIGNMENT LOOP: wave agents edited inline and nothing
re-verified the edits grounded.
6. DISPATCH COLLISION risk vs docs-sync: identical 'drift between
docs and code' triggers; dispatcher LLM could misroute.
7. BUNDLE LEAKAGE: references/design-handoff.md was session-history
(maintainer-scope), not runtime-loaded. Per genesis 3.5 it must
NOT ship with the user-facing bundle.
Refactor:
- SKILL.md (218 lines, well under 500-line cap): adds explicit
Sibling Contract table with docs-sync; declares roster as
composition of existing personas via relative links;
PROBE / RISK-TRIAGE / WAVE / POST-PASS / ALIGNMENT-LOOP /
COMMIT / PR phases; sharpened trigger description naming
whole-corpus scope.
- assets/subagent-prompt-template.md: extracted the per-scope
prompt that composes python-architect + doc-writer.
- assets/panelist-return-schema.json: explicit JSON schema for
agent returns; orchestrator validates and rejects malformed.
- scripts/scan-cross-corpus-drift.sh: deterministic cross-corpus
drift sweep with 4 pattern groups (ia-links, stale-deprecation,
absolute-base, ascii-leak). Non-interactive, --help-documented,
stdout/stderr split per genesis script conventions.
- evals/{trigger,content}-evals.json + README.md: ship gate
exercising 10+10 trigger queries (docs-sync boundary is the
load-bearing distinction) and 3 seeded-drift scenarios with
control baselines.
- Deleted references/design-handoff.md (bundle leak; design
artifact stays in session state only).
Self-test (proves the refactor works end-to-end):
- Ran scan-cross-corpus-drift.sh against the live corpus; it
immediately surfaced two genuine misses that wave 3 missed:
- src/apm_cli/commands/pack.py:606: click help= string still
said 'removed in v0.14' (the logger.warning at line 633 was
fixed last commit; this is a sibling string the wave 3 agent
didn't see because each agent only owned ~9 pages).
- docs/src/content/docs/reference/cli/unpack.md:9: caution
banner still said 'scheduled for removal in v0.14'.
- Both softened to 'in a future release' (consistent with the
rest of the wave 3 choice).
- Lint clean; docs build clean ('All internal links are valid').
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collaborator
Author
docs-corpus-audit skill: genesis-discipline refactor (commit 332b088)Maintainer flagged the first draft of
RefactorSKILL.md now adds an explicit Sibling Contract table with docs-sync (shared resources + trigger boundary) and an A9 PROBE phase at the top of the process. Self-test (proves the refactor is sound)Ran
Both softened to "in a future release". Lint + docs build clean. The refactored skill caught real drift the human-driven waves missed -- the strongest possible end-to-end validation of the SoC fix. |
…ft fixes New sibling skill to docs-corpus-audit. Genesis-designed PIPELINE-of-PANELS (RAGAS-faithfulness adapted from RAG to docs/code): - Stage 1: per-page LLM claim extraction - Stage 2: deterministic grep-based evidence retrieval (S7, no LLM) - Stage 3: adversarial LLM grounding judge (A7, 4-verdict calibrated) Empirical proof bundle (.apm/skills/docs-grounding-verifier/evals/runs/proof/): - 5 high-stakes pages -> 75 atomic claims extracted - Tally: 63 GROUNDED / 6 PARTIAL / 4 CONTRADICTED / 2 UNSUPPORTED (84%) - Trigger eval: 20/20 dispatch classification correct (precision=1.0, recall=1.0, specificity=1.0, pass_gate=true) High-confidence drift fixes applied: - apm-policy.md: MCP transport defaults (was 'block sse/streamable-http by default' -> actually allow=None means all permitted; sample policy now correctly framed as restriction example) - apm-policy.md: inheritance levels (was '5 levels including team policy' -> canonical chain is 3 semantic levels; 5 is MAX_CHAIN_DEPTH for intermediate extends: jumps) - Plus 5 editorial fixes from prior pass (examples, registries x2, security, copilot-app) Lower-confidence findings (judge retrieval gaps, vague reasoning) left for follow-up rather than risk introducing new drift via speculative edits. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collaborator
Author
Empirical proof: docs-grounding-verifier (commit 220d41e)Following the architectural assessment, I built docs-grounding-verifier as a genesis-designed sibling skill to docs-corpus-audit. It closes the gap I previously flagged ('no semantic grounding harness'). Architecture: PIPELINE-of-PANELS adapting RAGAS faithfulness eval from RAG to docs/code. Empirical results (re-runnable from
|
| Page | Grounded | Partial | Contradicted | Unsupported |
|---|---|---|---|---|
| apm-policy | 13 | 1 | 1 | 0 |
| registries | 11 | 2 | 0 | 2 |
| install | 13 | 2 | 0 | 0 |
| package-types | 12 | 1 | 2 | 0 |
| copilot-app | 14 | 0 | 1 | 0 |
| Total | 63 | 6 | 4 | 2 |
Baseline grounding rate: 84% -> proven measurable, not asserted.
High-confidence drift caught + fixed
The pipeline caught two real bugs in apm-policy.md that prior review passes missed:
- MCP transport default: docs claimed sse/streamable-http are blocked by default. Source (
schema.py:61,policy_checks.py:370):allow=Nonemeans all permitted; the example yaml was a restriction sample, not the default. - Inheritance levels: docs claimed '5 levels including team policy'. Source (
inheritance.py:3): canonical chain is 3 semantic levels (enterprise hub -> org -> repo); MAX_CHAIN_DEPTH=5 is a cycle guard, not a level count.
Plus 5 editorial fixes from the prior pass (examples, registries x2, security, copilot-app).
What lower-confidence findings (6 P / 2 U / 2 borderline C) tell us
Several judge findings stemmed from retrieval gaps (judge couldn't see the supporting source despite it existing) rather than real drift. Verified each via grep before fixing; skipped edits where the evidence was equivocal to avoid introducing speculative drift.
Why this is next level
- Empirical, not aspirational: numbers measured, drifts caught, traceable evidence per claim.
- Genesis SoC: sibling skill, dispatch-disjoint trigger evals confirm it doesn't collide with docs-sync or docs-corpus-audit.
- Deterministic grounding bridge: S7 grep-based retrieval = no hallucinated evidence (the load-bearing guard).
- Re-runnable: every claim, every evidence file, every judge verdict persisted under
evals/runs/proof/.
Lint + format + docs build all green on this commit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
Holistic docs-sync retrospective on the v0.14.0 -> v0.15.0 release window found 23 of 39 user-impact PRs left the docs corpus stale: 7 PRs shipped Starlight updates but skipped the
apm-usage/AI-facing training corpus (Rule 4 violation, rule in force since 2026-04-22), plus 16 confirmed silent-drift PRs with zero docs touched. This PR ships the P0/P1 fixes from that retrospective in one sweep.Verdict (CDO synthesizer): the docs corpus is drifting -- code shipped 2x faster than docs, and
apm-usage/is now materially behind the CLI surface. This PR closes the worst of the gap.Why (the retrospective)
Run via the
docs-syncskill end-to-end on PRs merged 2026-05-19 -> 2026-05-27:Three-promises health before this PR: consume ADEQUATE, produce WEAK, govern ADEQUATE.
PR #1471 (full registry feature, 37 src + 16 docs + 0 apm-usage) was the textbook example -- public-facing Starlight docs landed strong, but every AI-facing training file (
dependencies.md,authentication.md,governance.md,package-authoring.md,commands.md) was untouched.What (the fix scope)
Backfills --
packages/apm-guide/.apm/skills/apm-usage/Closes the Rule 4 gaps for 6 of the 7 Bucket-B PRs (#1471 spans 5 files; #1308, #1403, #1146, #1277, #1390, #1396 add targeted entries):
dependencies.mdauthentication.mdAPM_REGISTRY_TOKEN_{NAME}env var + precedence chaingovernance.mdregistry_source.require+allow_non_registrypolicy fields (transitive enforcement)package-authoring.mdapm publishworkflow + project-scope hook command path rulescommands.mdapm publishentry, transport keys forapm config,apm compile --watchlive-reload +--clean --watchwarning, Claude Code dedup, MCP env-var placeholder resolution, AppLocker/WDAC diagnosticEvery CLI claim was verified against
apm <verb> --helpoutput via the S7 tool bridge (python-architect verifier built into the docs-sync skill).Structural fix -- merge competing registry guides
The architect (
docs-impact-architectstep) foundguides/registries.mdandguides/private-registries.mdsubstantially restated each other -- two competing entry points for the same workflow.guides/registries.mdwith progressive disclosure: public -> private -> per-dep routing -> enterprise policy link./guides/private-registries -> /guides/registriesso old links don't 404.consumer/manage-dependencies.md,reference/cli/install.md,reference/cli/config.md.docs/astro.config.mjs.Editorial fixes (per editorial-owner sweep)
integrations/copilot-app.md(#1431):::note[How it works]. Add "restart the Copilot App once" troubleshooting hint.producer/compile.md:::note, identical):::note(preserves#claude-code-deduplicationanchor); delete blockquote.enterprise/security.mdHow (docs-sync skill execution)
End-to-end run of the
docs-syncskill on this retrospective:Total LLM cost: ~11 calls (5 retrospective + 4 writers + 1 CDO + 1 follow-up), under the 15-call ceiling.
Out of scope (P1 follow-up)
The 16 silent-drift PRs (Bucket C with verified user-visible behavior) need docs/ backfills too. Per CDO recommendation these should be grouped by subsystem and tracked separately to keep this PR reviewable:
Will file a tracking issue if maintainer wants.
Process recommendation (separately)
Recommend wiring a CI label gate so
src/PRs without docs/ or apm-usage/ touches auto-getdocs/needs-review, triggering the classifier. Advisory only, no merge block. Prevents this drift from re-accumulating. Open question for the maintainer to weigh in the retrospective comment.Validation
apm publish,apm config,apm compile,apm installreference verified against live--helpoutput.apm-usage/files and in the mergedguides/registries.md(verified withperl -ne 'print if /[^\x00-\x7F]/')./guides/registries/.private-registries.mdcarried over to mergedregistries.md. Sole intentional cuts documented in writer return.Stats
Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com