docs(ai-agents): design doc for context-aware next-step guidance

[antriksh30]

Fixes #7975 initial draft design doc context-aware Next: guidance after every successful azd ai agent command, plus a new azd ai agent doctor diagnostic.

Scope: entirely inside cli/azd/extensions/azure.ai.agents/. References core files for context but proposes no edits outside the extension.

Doc covers:

• Background & today's gaps (per-command table)
• Architecture: new nextstep package + thin command callers
• State model with layered sources (flags > runtime probes > azure.yaml > azd env)
• Per-command decision trees (init / run / invoke / show / deploy / doctor)
• Output format & TTY/pipe/JSON discipline
• Hook points per command (incl. why deploy uses the azdext.Artifact SDK field)
• OpenAPI invoke-payload extraction order
• Doctor checks 16 (MVP) + 711 deferred
• Testing strategy, risks, phased rollout

[Copilot]

Pull request overview

Adds a design document proposing context-aware “Next:” guidance for successful azd ai agent commands and a new azd ai agent doctor diagnostic within the azure.ai.agents extension, including a proposed nextstep package, state-resolution model, and per-command decision trees.

Changes:

• Introduces an end-to-end architecture for resolving and printing state-aware next-step suggestions (including deploy-time artifact note embedding).
• Defines the proposed state model, command-specific resolution policy, output/TTY discipline, and OpenAPI-derived invoke payload extraction strategy.
• Outlines an MVP doctor command with a phased checklist plan, plus testing strategy, risks, and alternatives.

[wbreza]

🔍 Design Doc Review

Thanks for the thorough design, @antriksh30! Strong technical depth with excellent extension-boundary discipline. Below are findings from a multi-perspective review (architecture, PM, testing, documentation).

---

P0 — Must Fix Before Merge (5)

1. RBAC principal ID + subscription path leakage
📄 �zd-ai-agent-doctor-remote-checks.md — Check 10

Check 10's fix command templates --assignee and --scope into stdout. In CI, these end up in persistent build logs — a credential-adjacent leak. The design needs a security-aware output contract: redact in non-interactive mode, or route sensitive fix commands to --output json only.

2. Check 7 auth mechanism contradicts between documents
📄 Both docs — Check 7

�zd-ai-agent-nextsteps.md says �z account show; �zd-ai-agent-doctor-remote-checks.md says �zidentity.NewAzureDeveloperCLICredential. These are different mechanisms with different failure modes. Suggest aligning on AzureDeveloperCLICredential since this is an �zd extension.

3. Parallel fan-out contradicts dependency matrix
📄 �zd-ai-agent-doctor-remote-checks.md — Risks table vs Dependency matrix

Risks table claims checks 8+10 can run concurrently, but the dependency matrix shows check 10 depends on check 8. Both can't be true. Either remove 8 as a dependency of 10 (RBAC calls ARM, not Foundry) or correct the risks table.

4. Security considerations section missing entirely
📄 �zd-ai-agent-doctor-remote-checks.md

A feature handling Azure credentials, UPN display, and CI output needs an explicit security/threat section covering: what sensitive data appears in output, behavior in CI/non-interactive contexts, and telemetry exclusions for PII fields.

5. �gent.yaml assumption contradicts issue #7975
📄 �zd-ai-agent-nextsteps.md

Issue #7975 states "assumes unified �zure.yaml" (agent.yaml gone), but the design references �gent.yaml throughout. Add an "Assumptions" section clarifying which config schema the design targets and what happens when unification lands.

---

P1 — Should Fix (17)

Expand P1 findings

#	File	Finding
6	nextsteps	State model missing IsAuthenticated field — resolver can't distinguish auth states; implementer will hit this mid-build
7	remote-checks	Check 11 dependency on check 10 too strict — Reader-role users get agent status skipped unnecessarily; consider removing check 10 from 11's deps
8	remote-checks	Check 8 conflates auth failure with reachability — 403 could be wrong-tenant, not missing RBAC; update fail message to mention both possibilities
9	nextsteps	Stray trailing triple-backtick at end of file corrupts markdown rendering
10	nextsteps	Duplicate "Multi-line note pre-indentation" heading — copy-paste error, remove duplicate
11	nextsteps	Broken #alternatives-considered anchor — links to a section that doesn't exist; add the section or remove the link
12	nextsteps	Backward compatibility not addressed — new Next: stdout output could break scripts; add a "Backward Compatibility" section
13	remote-checks	No --output json for doctor --remote — CI consumers need machine-readable doctor results
14	both	No doctor exit codes specified — CI can't gate on doctor (0=pass, 1=fail, 2=skip)
15	both	No test strategy for TTY/pipe/JSON output routing — core correctness contract needs test coverage
16	both	No test for non-interactive skip path — isTerminal detection failure in CI = hanging process
17	remote-checks	Runner skip-cascade testing missing — dependency matrix logic is the most complex part
18	both	Doctor file placement inconsistency — nextsteps says doctor.go, remote-checks says doctor/ subpackage
19	nextsteps	No user personas — design can't distinguish new developer from CI platform engineer
20	nextsteps	No success metrics/KPIs — no way to measure feature effectiveness
21	both	No telemetry/observability plan — check results should be instrumented for iteration
22	remote-checks	Foundry API version not specified — "pin api-version" without naming which one

---

P2 — Nice to Have (9)

Expand P2 findings

• Missing newline at EOF (remote-checks)
• Orphaned period at start of line in deploy decision tree
• Relative links to extension source paths need verification
• Trailing whitespace on multiple lines
• Check 1 "extension running" is tautological — consider renaming to "gRPC channel health"
• Adversarial/inconsistent State inputs not in test strategy
• Multi-agent cap of 5 is unjustified — consider data-backed threshold or making configurable
• OpenAPI $ref resolution not mentioned in extraction logic
• Check 9 model version mismatch deferred — could be low-cost Warn since data is already fetched

---

Strengths

• Extension-boundary discipline is excellent — no modifications outside the extension
• Decision trees are tabular and testable
• Artifact-note path for deploy is pragmatic and well-justified
• Phased implementation with independent shippability is the right call
• Risk tables are realistic, not boilerplate
• Cross-references between companion docs are symmetric and correct

[jongio]

See my detailed review below (review #4232410333) for the full analysis. This first review was a test artifact that couldn't be deleted.

[jongio]

@wbreza already covered this thoroughly - adding a few things that review didn't flag.

Artifact integration mechanism is underspecified (the most important gap)

The doc repeatedly says "attach Next: to the artifact's note field" and "populate the artifact's note field" - but azdext.Artifact has no Note field. The proto defines: Kind, Location, LocationKind, and Metadata (map). Today's deploy artifacts are ARTIFACT_KIND_ENDPOINT with Metadata["label"] rendered as - label: url.

For the Next: block to work through this channel, the design needs to specify:

1. Which ArtifactKind carries freeform text? A new enum value like ARTIFACT_KIND_NOTE? Or reusing an existing one?
2. How does core's MessageUxItem renderer know to display multi-line text instead of a - label: url line?
3. If using Metadata["note"] - does core's rendering code already handle that key specially, or does it need changes (which contradicts the "no edits outside the extension" scope)?

This is the one integration point between the extension and core that isn't fully self-contained - and the design doc's scope claim ("no files outside the extension are modified") depends on getting this right.

State model type inconsistency

HasUnresolvedInfraVars is bool but HasUnresolvedManualVars is []string. The init resolver needs manual var names to emit azd env set <KEY> <value>. But if infra vars are unresolved, knowing which Bicep outputs are missing would let the guidance be more specific than just "run azd provision".

Deploy payload derivation gap

The run decision tree has OpenAPI extraction logic for deriving the invoke payload. The deploy tree just says '' literally. Should the post-deploy Next: block attempt OpenAPI discovery against the remote endpoint, or reference the README? The run case and deploy case have asymmetric payload handling without explaining why.

[antriksh30]

Thanks @wbreza, ran a multi-model critique on the fix plan first then applied. Status of your list (commit 4358e68):

P0 (5/5): all addressed — RBAC redaction (Check 10) ✅ · auth alignment to AzureDeveloperCLICredential ✅ · removed Check 8 from Check 10's deps + reconciled risks ✅ · new Security & PII / Telemetry section ✅ · new Assumptions section (agent.yaml is current; azure.yaml placeholders for unified stanza per #7975) ✅

P1 (14/17 + 3 explicitly deferred): IsAuthenticated → tri-state AuthState w/ WithAuthProbe opt-in ✅ · removed Check 10 from Check 11's deps ✅ · 403 broadened to "wrong tenant or insufficient RBAC" ✅ · stray backtick / duplicate heading verified clean ✅ · broken #alternatives-considered anchor removed ✅ · Backward Compatibility section ✅ · Exit codes (0/1/2) + JSON envelope ✅ · TTY/pipe/JSON test rows ✅ · skip-cascade + non-interactive skip tests ✅ · doctor/ subpackage now consistent across both docs ✅ · api-version pinned to DefaultAgentAPIVersion constant (agent_context.go:18) ✅. Deferred (in new "Deferred follow-ups" section): personas, KPIs, full telemetry schema — telemetry sketch is in the new Telemetry subsection.

P2 (4/9): Check 1 → "gRPC channel health" ✅ · multi-agent cap-of-5 replaced ✅ · OpenAPI $ref explicit out-of-scope ✅ · adversarial state inputs implicit in expanded testing ✅. Cosmetic items (EOF newline, trailing whitespace, orphan period, link-verify, Check 9 model-warn) skipped — happy to do a follow-up cleanup pass if useful.

Plus jongio's three: anchored Status block on Metadata["note"] + ArtifactKindEndpoint (no core renderer change), renamed to MissingInfraVars []string, deploy resolver explicitly uses cached spec only (no live probe — agent may not be reachable yet, which is what Check 11 observes).

[github-actions]

📋 Prioritization Note

Thanks for the contribution! The linked issue isn't in the current milestone yet.
Review may take a bit longer — reach out to @rajeshkamal5050 or @kristenwomack if you'd like to discuss prioritization.

[jongio]

Addresses my previous feedback. The new commit clarifies all three gaps I flagged:

1. Artifact mechanism - now explicitly specifies Metadata["note"] on ArtifactKindEndpoint, references pkg/project/artifact.go ToString (~line 128). Verified this path exists and works as described.
2. State model asymmetry - renamed to MissingInfraVars []string and MissingManualVars []string (both []string now).
3. Deploy payload - added "No live OpenAPI probe at deploy time" paragraph with the fallback chain (cached spec -> README -> generic literal).

Also good additions: AuthState enum with AuthUnknown default (avoids login noise), WithAuthProbe opt-in, exit code table, JSON envelope schema, redaction rules, security section, backward compatibility section, and the $ref out-of-scope note.

One minor observation: the isTerminal helper is referenced as living in internal/cmd/helpers.go - worth confirming the exact function name in a code comment so implementers don't have to grep for it.

[therealjohn]

@rajeshkamal5050 Is this a good candidate and time for us to do azure.ai.projects extension and put this as a shared framework for next steps suggestions? Eventually this looks like something that could be provided by azd core, but for now, we can at least share it across azd ai * extensions.

[trangevi]

Discussed in the large chat, I think for now let's leave it as part of the agent extension focused on the agent commands, and we can expand later.

[jongio]

All three items from my earlier review are addressed. Latest commit (flag model flip to --local-only, check 12 for agent identity roles, terminology section) introduces no new issues. Approving.

[trangevi]

Looks good enough to proceed in my opinion. I'm sure other stuff might come up during implementation, but don't think we need to block on getting a perfect spec

[trangevi]

Discussed in the large chat, I think for now let's leave it as part of the agent extension focused on the agent commands, and we can expand later.

[trangevi]

We will not have a unified azure.yaml by //build, so for now this should proceed with the azure.yaml and agent.yaml (definition).

[trangevi]

How will "stale" be determined?

[trangevi]

I don't really like the idea of this showing up as an error, I feel like that would potentially confuse anyone looking at the output to determine what happened in case of a failure/might confuse an agent thinking that these are failures. If anything, wouldn't we want this to be part of the parseable output, so it helps guide what is next?

[trangevi]

why not put everything at the end of the runE if possible? I understand some commands like run and deploy are unique, but most I think it would make sense to put in the same place for consistency?