Skip to content

feat(compile): add target: job and target: stage for ADO template output#519

Open
jamesadevine wants to merge 2 commits into
mainfrom
feat/stage-job-template-targets
Open

feat(compile): add target: job and target: stage for ADO template output#519
jamesadevine wants to merge 2 commits into
mainfrom
feat/stage-job-template-targets

Conversation

@jamesadevine
Copy link
Copy Markdown
Collaborator

Summary

Add two new compile targets (target: job and target: stage) that produce reusable ADO YAML templates for embedding agentic stages into existing pipelines.

target: job — generates a job-level template (jobs: at root) for inclusion in a flat pipeline or inside a user-defined stage:

# Flat pipeline
jobs:
  - template: agents/review.lock.yml

# Or inside a stage
stages:
  - stage: AgenticReview
    dependsOn: Build
    jobs:
      - template: agents/review.lock.yml

target: stage — generates a stage-level template (stages: wrapping jobs) for direct inclusion in multi-stage pipelines, with native ADO dependsOn and condition at the call site:

stages:
  - template: agents/review.lock.yml
    dependsOn: Build
    condition: succeeded()

Design decisions

  • Pool is baked from front matter (not a template parameter)
  • dependsOn/condition are set natively at the ADO call site (not template params)
  • Job names are prefixed with PascalCase agent name for uniqueness (e.g., DailyReview_Agent, DailyReview_Detection, DailyReview_Execution)
  • Triggers (on:) are ignored with a warning in template targets
  • Template parameters only include clearMemory and user-defined params (if any)

New files

  • src/compile/job.rsJobCompiler implementing the Compiler trait
  • src/compile/stage.rsStageCompiler implementing the Compiler trait
  • src/data/job-base.yml — job-level template derived from base.yml
  • src/data/stage-base.yml — stage-level template wrapping jobs in a stage block
  • tests/fixtures/job-agent.md and stage-agent.md — test fixtures

Test plan

  • cargo build — clean compilation
  • cargo test --bin ado-aw — 1350 tests pass, 0 failures
  • cargo clippy --all-targets --all-features — no new warnings
  • Both fixtures compile correctly (cargo run -- compile tests/fixtures/job-agent.md and stage-agent.md)
  • Verified output YAML has correctly prefixed job names and dependency chains
  • Added both targets to REQUIRED_TARGETS in tests/bash_lint_tests.rs

Add two new compile targets that produce reusable ADO YAML templates
for embedding agentic stages into existing pipelines:

- target: job — generates a job-level template (jobs: at root) that can
  be included in a flat pipeline or inside a user-defined stage
- target: stage — generates a stage-level template (stages: wrapping
  jobs) for direct inclusion in multi-stage pipelines

Key design decisions:
- Pool is baked in from front matter (not a template parameter)
- dependsOn and condition are set natively at the ADO call site
- Job names are prefixed with PascalCase agent name for uniqueness
  (e.g., DailyReview_Agent, DailyReview_Detection, DailyReview_Execution)
- Triggers (on:) are ignored with a warning in template targets
- Template parameters only include clearMemory and user-defined params

New files:
- src/compile/job.rs — JobCompiler implementing the Compiler trait
- src/compile/stage.rs — StageCompiler implementing the Compiler trait
- src/data/job-base.yml — job-level template derived from base.yml
- src/data/stage-base.yml — stage-level template wrapping jobs in stage

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Good feature addition overall — clean architecture, proper trait implementation, and solid test coverage. Three issues worth addressing before merge.

Findings

🐛 Bugs / Logic Issues

  • src/compile/job.rs:147 and src/compile/stage.rs:124repos: entries silently dropped from header docs

    The condition correctly checks both front_matter.repos and front_matter.repositories, but the documentation loop only iterates front_matter.repositories (legacy field). When a user uses the modern repos: field, the header emits the scaffolding comment block with no actual repo entries:

    // Condition fires on new-style repos:
    if !front_matter.repos.is_empty() || !front_matter.repositories.is_empty() {
        ...
        for repo in &front_matter.repositories { // ← never iterates repos entries

    The repos field uses ReposItem (string shorthand or object). After lower_repos() is called, the lowered form is in front_matter.repositories, but at header-generation time that lowering may not have run yet on the original FrontMatter. Worth either lowering first or generating the docs from the repos field directly.

  • src/compile/job.rs:136 and src/compile/stage.rs:113.replace(".md", ".lock.yml") corrupts directory names

    source_path.replace(".md", ".lock.yml")

    This replaces all occurrences of .md in the path string. A path like markdown-docs/my-agent.md becomes lock.yml-docs/my-agent.lock.yml. Should use Path-aware extension replacement:

    input_path.with_extension("lock.yml").to_string_lossy().replace('\\', "/")

⚠️ Suggestions

  • src/compile/common.rs:1009generate_stage_prefix preserves Unicode alphanumerics

    The function uses Rust's Unicode-aware char::is_alphanumeric(), so agent names with non-ASCII letters (e.g. "über-agent""ÜberAgent") pass through and produce ADO job names that fail ADO's [A-Za-z0-9_] identifier requirement at pipeline queue time. Consider filtering to ASCII only:

    .split(|c: char| !c.is_ascii_alphanumeric())

    And in the leading-digit guard, starts_with(|c: char| c.is_ascii_digit()) is already correct — just the split needs the is_ascii_ variant.

✅ What Looks Good

  • Clean factoring of shared compilation logic via skip_header: bool in CompileConfig — minimal diff on existing compilers.
  • The extra_replacements mechanism correctly threads {{ stage_prefix }} and {{ template_parameters }} through compile_shared without needing new trait methods.
  • Job/stage dependency chain in both templates correctly gates _Execution on both _Agent and _Detection outputs.
  • Bash lint tests properly extended to cover both new targets in REQUIRED_TARGETS.
  • The generate_stage_prefix fallback to "Agent" and leading-digit prefix guard are sensible ADO safety rails.

Generated by Rust PR Reviewer for issue #519 · ● 921.3K ·

- Fix path corruption: use Path::with_extension instead of
  string replace to derive .lock.yml paths in header comments
  (avoids corrupting directory names containing '.md')
- Fix repos header docs: only check front_matter.repositories
  (populated by resolve_repos() before compile) instead of
  also checking front_matter.repos (raw input, may be empty)
- Fix Unicode in stage prefix: use is_ascii_alphanumeric() to
  split on non-ASCII characters, ensuring ADO-valid identifiers
  (ADO requires [A-Za-z0-9_] for job/stage names)
- Add test for Unicode stripping in generate_stage_prefix

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Solid implementation with good marker coverage and test infrastructure — two things worth addressing before merge.

Findings

⚠️ Suggestions

  • docs/template-markers.md not updated (src/compile/common.rs, src/data/job-base.yml, src/data/stage-base.yml): Two new template markers are introduced — {{ stage_prefix }} and {{ template_parameters }} — that are exclusive to the new template targets. Per project convention in AGENTS.md: "New template markers are documented in docs/template-markers.md". Both markers need entries describing their source and semantics.

  • Significant code duplication (src/compile/job.rs, src/compile/stage.rs): The compile() implementations are essentially identical — same extensions collection, same MCPG config generation, same CompileConfig construction, and identical extra_replacements list. Only the template file and the header generation function differ. A shared private helper (e.g., compile_template_target(template: &str, header_fn, ...)) would halve the maintenance surface and reduce the risk of the two compilers diverging silently. This is particularly notable because stage.rs already has the same generate_stage_header name pattern but independently reimplements everything.

  • info!("Using {} compiler", ...) removed without replacement (src/compile/mod.rs): The removed log applied to all targets. The two new compilers add their own info!() in compile(), but StandaloneCompiler and OneESCompiler now log nothing about which compiler path was selected. Easy fix: either keep the mod.rs log or add matching info! calls to the existing compilers.

  • Silent non-ASCII truncation in generate_stage_prefix (src/compile/common.rs:1007): "über-agent" silently produces "BerAgent", dropping the initial character. This is documented in the function-level doc comment and tested, but a user compiling an agent with a non-ASCII name gets no warning. A warn!() when the resulting prefix differs from what a naive strip would suggest would help — though this is low-priority given the ADO identifier constraint is real.

✅ What Looks Good

  • Template marker coverage is complete: every {{ marker }} in both job-base.yml and stage-base.yml is accounted for — either in extra_replacements or handled by compile_shared's standard replacement table. No leftover markers to worry about.
  • serde deserialization is correct: #[serde(rename_all = "lowercase")] on CompileTarget means target: job and target: stage deserialize correctly without explicit #[serde(rename)] attributes.
  • generate_stage_prefix output is safe: the split-on-non-alphanumeric logic guarantees the output only ever contains [A-Za-z0-9_], making it injection-safe for both YAML job names and ADO condition expressions.
  • Test coverage is thorough: seven unit tests for generate_stage_prefix covering edge cases (empty, leading digit, unicode), fixture files for both targets, and both targets added to REQUIRED_TARGETS in bash_lint_tests.rs.
  • Error handling follows project conventions: anyhow::Result throughout, .context() on fallible calls, no silent unwrap() on user-facing paths.

Generated by Rust PR Reviewer for issue #519 · ● 1.5M ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant