Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ Every compiled pipeline runs as three sequential jobs:
│ │ ├── common.rs # Shared helpers across targets
│ │ ├── standalone.rs # Standalone pipeline compiler
│ │ ├── onees.rs # 1ES Pipeline Template compiler
│ │ ├── job.rs # Job-level ADO template compiler (target: job)
│ │ ├── stage.rs # Stage-level ADO template compiler (target: stage)
│ │ ├── gitattributes.rs # .gitattributes management for compiled pipelines
│ │ ├── filter_ir.rs # Filter expression IR: Fact/Predicate types, lowering, validation, codegen
│ │ ├── pr_filters.rs # PR trigger filter generation (native ADO + gate steps)
Expand Down Expand Up @@ -122,6 +124,8 @@ Every compiled pipeline runs as three sequential jobs:
│ ├── data/
│ │ ├── base.yml # Base pipeline template for standalone
│ │ ├── 1es-base.yml # Base pipeline template for 1ES target
│ │ ├── job-base.yml # Job-level ADO template for target: job
│ │ ├── stage-base.yml # Stage-level ADO template for target: stage
│ │ ├── ecosystem_domains.json # Network allowlists per ecosystem
│ │ ├── init-agent.md # Dispatcher agent template for `init` command
│ │ └── threat-analysis.md # Threat detection analysis prompt template
Expand Down
1 change: 1 addition & 0 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Global flags (apply to all subcommands): `--verbose, -v` (enable info-level logg
- `--output, -o <path>` - Optional output path for the generated YAML (only valid when a path is provided). If the path is an existing directory, the compiled YAML is written inside that directory using the default filename derived from the markdown source (e.g. `foo.md` → `<dir>/foo.lock.yml`).
- `--skip-integrity` - *(debug builds only)* Omit the "Verify pipeline integrity" step from the generated pipeline. Useful during local development when the compiled output won't match a released compiler version. This flag is not available in release builds.
- `--debug-pipeline` - *(debug builds only)* Include MCPG debug diagnostics in the generated pipeline: `DEBUG=*` environment variable for verbose MCPG logging, stderr streaming to log files, and a "Verify MCP backends" step that probes each backend with MCP initialize + tools/list before the agent runs. This flag is not available in release builds.
- For `target: job` and `target: stage`, the output is an ADO YAML template (not a complete pipeline). Job names are prefixed with the agent name for uniqueness. Triggers configured via `on:` are ignored with a warning.
- `check <pipeline>` - Verify that a compiled pipeline matches its source markdown
- `<pipeline>` - Path to the pipeline YAML file to verify
- The source markdown path is auto-detected from the `@ado-aw` header in the pipeline file
Expand Down
2 changes: 1 addition & 1 deletion docs/front-matter.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The compiler expects markdown files with YAML front matter similar to gh-aw:
---
name: "name for this agent"
description: "One line description for this agent"
target: standalone # Optional: "standalone" (default) or "1es". See docs/targets.md.
target: standalone # Optional: "standalone" (default), "1es", "job", or "stage". See docs/targets.md.
engine: copilot # Engine identifier. Defaults to copilot. Currently only 'copilot' (GitHub Copilot CLI) is supported.
# engine: # Alternative object format (with additional options)
# id: copilot
Expand Down
76 changes: 76 additions & 0 deletions docs/targets.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,79 @@ target: 1es
```

When using `target: 1es`, the pipeline will extend `1es/1ES.Unofficial.PipelineTemplate.yml@1ESPipelinesTemplates`.

### `job`

Generates a **job-level ADO YAML template** with `jobs:` at root. This is a
reusable template that can be included in an existing pipeline — it does not
generate a complete pipeline.

The output contains the same 3-job chain (Agent → Detection → Execution) as
`standalone`, with:
- Job names prefixed with the agent name for uniqueness (e.g., `DailyReview_Agent`)
- No triggers, pipeline name, or resource declarations (the parent pipeline owns those)
- Pool baked in from the front matter `pool:` field

Example front matter:
```yaml
target: job
```

#### Usage in a flat pipeline

```yaml
jobs:
- job: Build
steps: ...
- template: agents/review.lock.yml
```

#### Usage inside a user-defined stage

```yaml
stages:
- stage: Build
jobs: ...
- stage: AgenticReview
dependsOn: Build
jobs:
- template: agents/review.lock.yml
```

#### Notes

- Triggers (`on:`) are ignored with a warning (the parent pipeline controls triggers)
- If the agent declares additional repositories via `repos:`, add them to the
parent pipeline's `resources:` block (documented in the generated file header)

### `stage`

Generates a **stage-level ADO YAML template** with `stages:` at root. This
wraps the 3-job chain inside a stage block for direct inclusion in multi-stage
pipelines.

Example front matter:
```yaml
target: stage
```

#### Usage

```yaml
stages:
- stage: Build
jobs: ...
- template: agents/review.lock.yml
dependsOn: Build
condition: succeeded()
```

ADO natively supports `dependsOn` and `condition` at the template call site —
no template parameters are needed for stage ordering.

#### Notes

- Same 3-job chain, job-name prefixing, and pool handling as `target: job`
- Triggers (`on:`) are ignored with a warning
- If the agent declares additional repositories via `repos:`, add them to the
parent pipeline's `resources:` block
68 changes: 65 additions & 3 deletions src/compile/common.rs
Original file line number Diff line number Diff line change
Expand Up @@ -999,6 +999,60 @@ pub fn sanitize_filename(name: &str) -> String {
/// Default pool name
pub const DEFAULT_POOL: &str = "AZS-1ES-L-MMS-ubuntu-22.04";

/// Derive a valid ADO identifier from the agent name for use as a job-name
/// prefix and stage name. Converts to PascalCase, stripping non-alphanumeric
/// characters.
///
/// Examples:
/// - `"Daily Code Review"` → `"DailyCodeReview"`
/// - `"my-agent-123"` → `"MyAgent123"`
/// - `""` → `"Agent"` (fallback)
/// - `"123start"` → `"_123start"` (prefix underscore for leading digit)
/// - `"über-agent"` → `"BerAgent"` (non-ASCII stripped; ADO requires `[A-Za-z0-9_]`)
pub fn generate_stage_prefix(name: &str) -> String {
let pascal: String = name
.split(|c: char| !c.is_ascii_alphanumeric())
.filter(|s| !s.is_empty())
.map(|word| {
let mut chars = word.chars();
match chars.next() {
None => String::new(),
Some(first) => {
let upper = first.to_uppercase().to_string();
upper + chars.as_str()
}
}
})
.collect();

if pascal.is_empty() {
"Agent".to_string()
} else if pascal.starts_with(|c: char| c.is_ascii_digit()) {
format!("_{}", pascal)
} else {
pascal
}
}

/// Generate the template-level `parameters:` YAML block for job/stage
/// template targets.
///
/// Includes clearMemory (if cache-memory enabled) and user-defined
/// parameters from front matter. Returns empty string if no parameters
/// are needed.
pub fn generate_template_parameters(front_matter: &FrontMatter) -> Result<String> {
let has_memory = front_matter
.tools
.as_ref()
.and_then(|t| t.cache_memory.as_ref())
.is_some_and(|cm| cm.is_enabled());
let params = build_parameters(&front_matter.parameters, has_memory);
if params.is_empty() {
return Ok(String::new());
}
generate_parameters(&params)
}

/// Version of the AWF (Agentic Workflow Firewall) binary to download from GitHub Releases.
/// Update this when upgrading to a new AWF release.
/// See: https://github.com/github/gh-aw-firewall/releases
Expand Down Expand Up @@ -2423,6 +2477,10 @@ pub struct CompileConfig {
/// to append `GITHUB_PATH: $(GITHUB_PATH)` to the engine env block without
/// re-collecting path prepends from extensions.
pub has_awf_paths: bool,
/// When true, `compile_shared` omits the standard `# @ado-aw` header.
/// Template-producing compilers (Job, Stage) set this to prepend their
/// own custom header with usage instructions.
pub skip_header: bool,
}

/// Shared compilation flow used by both standalone and 1ES compilers.
Expand Down Expand Up @@ -2704,9 +2762,13 @@ pub async fn compile_shared(
replace_with_indent(&yaml, placeholder, replacement)
});

// 15. Prepend header
let header = generate_header_comment(input_path);
Ok(format!("{}{}", header, pipeline_yaml))
// 15. Prepend header (unless the caller will prepend its own)
if config.skip_header {
Ok(pipeline_yaml)
} else {
let header = generate_header_comment(input_path);
Ok(format!("{}{}", header, pipeline_yaml))
}
}

#[cfg(test)]
Expand Down
Loading
Loading