Conversation
Add a new 'workflow-automation' pillar (repo-health group) with five new checkers and six scored criteria covering process automation signals that reduce engineering friction per PR: New checkers (checkers.ts + monolithic readiness.ts): - hasIssueTemplates: .github/ISSUE_TEMPLATE/ dir or flat ISSUE_TEMPLATE.md - hasCommitConvention: commitlint config files or @commitlint/* devDeps - hasReleaseAutomation: .releaserc/.changeset config or workflow scan - hasAutoLabeler: .github/labeler.yml or actions/labeler workflow reference - hasBranchRulesets: .github/rulesets/*.json or branch-protection.json New criteria (workflow-automation pillar): - issue-templates (L2, high impact, low effort) - pr-template (L2, high impact, low effort) — promoted from extras - commit-convention (L3, high impact, low effort) - pr-labeling (L3, medium impact, low effort) - release-automation (L4, high impact, medium effort) New criterion (security-governance pillar): - branch-protection (L3, high impact, medium effort) Supporting changes: - types.ts: add 'workflow-automation' to ReadinessPillar union and PILLAR_GROUPS - scoring.ts: add 'Workflow Automation' display name - extras.ts: remove pr-template (now a scored criterion) - policy/adapter.ts: add 'workflow-automation' to PILLAR_NAMES Tests: update baseline/adapter tests for new pillar count and criterion IDs; add 30 new integration tests covering all new criteria pass/fail scenarios
…nd adjust references to 10 pillars
There was a problem hiding this comment.
Pull request overview
Adds a new Workflow Automation readiness pillar to AgentRC’s readiness model, expanding repo-health scoring signals around process automation and governance.
Changes:
- Introduces the
workflow-automationpillar and new scored criteria (issue templates, PR templates, commit convention, PR labeling, release automation), plus a newbranch-protectioncriterion under security/governance. - Implements new repository checkers to detect the above signals (files/configs + lightweight workflow scanning).
- Updates tests and documentation to reflect 10 pillars and the new criteria/extras set.
Reviewed changes
Copilot reviewed 21 out of 21 changed files in this pull request and generated 17 comments.
Show a summary per file
| File | Description |
|---|---|
| vscode-extension/resources/walkthrough/readiness.md | Updates walkthrough text to 10 pillars and lists the new pillar. |
| vscode-extension/README.md | Updates extension README to 10 pillars. |
| src/services/tests/readiness.test.ts | Adds integration tests for new criteria and ensures new pillar is present. |
| src/services/tests/readiness-baseline.test.ts | Updates baselines for pillar IDs, criteria IDs, and extras IDs. |
| src/services/tests/policy-adapter.test.ts | Updates expected pillar count to 10. |
| packages/core/src/services/readiness/types.ts | Adds workflow-automation to the modular readiness pillar union/groups. |
| packages/core/src/services/readiness/scoring.ts | Adds “Workflow Automation” display name mapping. |
| packages/core/src/services/readiness/extras.ts | Removes pr-template from extras (now a scored criterion). |
| packages/core/src/services/readiness/criteria.ts | Adds the new scored criteria (and branch-protection) to the modular criteria builder. |
| packages/core/src/services/readiness/checkers.ts | Adds new repo checkers for workflow automation + branch protection detection. |
| packages/core/src/services/readiness.ts | Mirrors the new pillar/criteria/checkers in the monolithic exported readiness entrypoint. |
| packages/core/src/services/policy/adapter.ts | Adds pillar display name mapping for workflow-automation. |
| docs/policies.md | Updates extras list from 4 → 3 (removes pr-template). |
| docs/maturity-models.md | Adds canonical maturity model documentation (new file). |
| docs/maturity-in-practice.md | Adds companion adoption guidance doc (new file). |
| docs/dev/product.md | Updates product brief to 10 pillars. |
| docs/concepts.md | Updates concepts doc to include workflow automation and 10 pillars. |
| docs/commands.md | Updates readiness command docs to 10 pillars + links maturity models. |
| README.md | Updates root README to 10 pillars and links new docs. |
| CHANGELOG.md | Adds changelog entry for the new pillar and criteria. |
| .github/prompts/generate-improvements.prompt.md | Updates prompt context to reference 10 pillars. |
| const templatePath = path.join(context.repoPath, ".github", "PULL_REQUEST_TEMPLATE.md"); | ||
| const content = await fs.readFile(templatePath, "utf8"); | ||
| const hasLinkedIssue = /fixes\s+#|closes\s+#|resolves\s+#/iu.test(content); | ||
| return { | ||
| status: "pass", | ||
| reason: hasLinkedIssue | ||
| ? undefined | ||
| : 'PR template found but lacks a linked-issue reference ("Fixes #", "Closes #"). Add one to enable automatic issue closing on merge.' | ||
| }; |
There was a problem hiding this comment.
/fixes\s+#|closes\s+#|resolves\s+#/ can match substrings inside other words (e.g. "preFIXES #"), causing false positives and suppressing the advisory reason. Consider adding word boundaries (e.g. \bfixes\b) and optionally allowing common punctuation like Fixes: #.
There was a problem hiding this comment.
@copilot open a new pull request to apply changes based on this feedback
docs/concepts.md
Outdated
| @@ -1,4 +1,4 @@ | |||
| # Concepts | |||
| # Concepts | |||
There was a problem hiding this comment.
Line 1 appears to include a UTF-8 BOM (invisible character before # Concepts). Recommend removing it to avoid encoding-related issues.
| # Concepts | |
| # Concepts |
| @@ -1,4 +1,4 @@ | |||
| --- | |||
| --- | |||
There was a problem hiding this comment.
The opening frontmatter delimiter appears to have a UTF-8 BOM prefix (invisible character before ---). Some frontmatter parsers treat this as a different token and will fail to detect the header. Remove the BOM so the delimiter is exactly ---.
| --- | |
| --- |
docs/maturity-models.md
Outdated
| @@ -0,0 +1,295 @@ | |||
| # AgentRC: The Agentic SDLC Report Card | |||
There was a problem hiding this comment.
Line 1 appears to include a UTF-8 BOM (invisible character before the # heading). Suggest removing it to avoid encoding-related issues.
| # AgentRC: The Agentic SDLC Report Card | |
| # AgentRC: The Agentic SDLC Report Card |
| const templatePath = path.join(context.repoPath, ".github", "PULL_REQUEST_TEMPLATE.md"); | ||
| const content = await fs.readFile(templatePath, "utf8"); | ||
| const hasLinkedIssue = /fixes\s+#|closes\s+#|resolves\s+#/iu.test(content); | ||
| return { | ||
| status: "pass", | ||
| reason: hasLinkedIssue | ||
| ? undefined | ||
| : 'PR template found but lacks a linked-issue reference ("Fixes #", "Closes #"). Add one to enable automatic issue closing on merge.' | ||
| }; |
There was a problem hiding this comment.
The linked-issue detection regex can match substrings inside other words, which may incorrectly treat a template as containing "Fixes #" and omit the advisory reason. Adding word boundaries (and optionally supporting Fixes: #) would make this check more accurate.
docs/dev/product.md
Outdated
| # AgentRC — Product Brief | ||
|
|
There was a problem hiding this comment.
Line 1 appears to include a UTF-8 BOM (invisible character before the # heading). Removing the BOM will prevent encoding/tooling quirks and keeps diffs clean.
README.md
Outdated
| @@ -1,4 +1,4 @@ | |||
| # AgentRC | |||
| # AgentRC | |||
There was a problem hiding this comment.
Line 1 appears to include a UTF-8 BOM (invisible character before # AgentRC). Consider removing the BOM to avoid tooling/rendering quirks and reduce diff noise.
| # AgentRC | |
| # AgentRC |
docs/maturity-in-practice.md
Outdated
| @@ -0,0 +1,131 @@ | |||
| # SDLC Readiness Assessment: Toward Level-5 Agentic Delivery | |||
There was a problem hiding this comment.
Line 1 appears to include a UTF-8 BOM (invisible character before the # heading). Recommend removing it to keep the file encoding consistent.
| # SDLC Readiness Assessment: Toward Level-5 Agentic Delivery | |
| # SDLC Readiness Assessment: Toward Level-5 Agentic Delivery |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 21 out of 21 changed files in this pull request and generated 6 comments.
Comments suppressed due to low confidence (1)
packages/core/src/services/readiness.ts:1642
- This helper makes
branch-protection.md/branch_protection.mdcount as “branch protection configured”, which can yield false positives (it’s just documentation). Consider restricting this check to actual config files (.github/rulesets/*.json,.github/branch-protection.json) so the readiness signal reflects enforcement rather than docs.
const rootFiles = await safeReadDir(repoPath);
if (
rootFiles.some(
(f) =>
f.toLowerCase() === "branch_protection.md" || f.toLowerCase() === "branch-protection.md"
)
)
return true;
return fileExists(path.join(repoPath, ".github", "branch-protection.json"));
| try { | ||
| const templatePath = path.join(context.repoPath, ".github", "PULL_REQUEST_TEMPLATE.md"); | ||
| const content = await fs.readFile(templatePath, "utf8"); | ||
| const hasLinkedIssue = /fixes\s+#|closes\s+#|resolves\s+#/iu.test(content); |
There was a problem hiding this comment.
Same issue as in the core criteria: /fixes\s+#|closes\s+#|resolves\s+#/ can match inside other words and produce false positives. Use word boundaries / stricter parsing for linked-issue keywords so the advisory reason isn’t skipped accidentally.
This issue also appears on line 1634 of the same file.
| const hasLinkedIssue = /fixes\s+#|closes\s+#|resolves\s+#/iu.test(content); | |
| const hasLinkedIssue = /\b(?:fixes|closes|resolves)\s+#/iu.test(content); |
| reason: found | ||
| ? undefined | ||
| : "No branch ruleset or protection config found (.github/rulesets/*.json or .github/branch-protection.json). Add branch protection rules to prevent unreviewed merges.", | ||
| evidence: [".github/rulesets/*.json", ".github/branch-protection.json"] | ||
| }; |
There was a problem hiding this comment.
The failure reason mentions only .github/rulesets/*.json, but the evidence list also includes .github/branch-protection.json. Update the message to include both locations (or align evidence/message) so users know all supported config options.
docs/maturity-models.md
Outdated
| @@ -0,0 +1,295 @@ | |||
| # AgentRC: The Agentic SDLC Report Card | |||
There was a problem hiding this comment.
This new doc file starts with a UTF-8 BOM (U+FEFF) before the heading. Please remove the BOM and save as UTF-8 without BOM to avoid subtle Markdown rendering/anchor issues in different viewers.
| # AgentRC: The Agentic SDLC Report Card | |
| # AgentRC: The Agentic SDLC Report Card |
docs/dev/product.md
Outdated
| @@ -1,4 +1,4 @@ | |||
| # AgentRC — Product Brief | |||
| # AgentRC — Product Brief | |||
There was a problem hiding this comment.
This file now starts with a UTF-8 BOM (U+FEFF) before the heading. Please remove the BOM and save as UTF-8 without BOM to avoid subtle Markdown rendering/anchor issues.
| # AgentRC — Product Brief | |
| # AgentRC — Product Brief |
vscode-extension/README.md
Outdated
| @@ -1,4 +1,4 @@ | |||
| # AgentRC — AI Repository Setup | |||
| # AgentRC — AI Repository Setup | |||
There was a problem hiding this comment.
This file now starts with a UTF-8 BOM (U+FEFF) before the heading. Please remove the BOM and save as UTF-8 without BOM; BOMs can cause odd rendering in VS Code walkthrough/markdown contexts.
| # AgentRC — AI Repository Setup | |
| # AgentRC — AI Repository Setup |
| try { | ||
| const templatePath = path.join(context.repoPath, ".github", "PULL_REQUEST_TEMPLATE.md"); | ||
| const content = await fs.readFile(templatePath, "utf8"); | ||
| const hasLinkedIssue = /fixes\s+#|closes\s+#|resolves\s+#/iu.test(content); |
There was a problem hiding this comment.
The linked-issue detection regex can match substrings inside other words (e.g. "preFIXES #"), which will suppress the advisory reason incorrectly. Tighten the pattern with word boundaries (and optionally allow punctuation like "Fixes: #") so only real keywords trigger a match.
| const hasLinkedIssue = /fixes\s+#|closes\s+#|resolves\s+#/iu.test(content); | |
| const hasLinkedIssue = /\b(?:fixes|closes|resolves)\b\s*:?\s*#/iu.test(content); |
|
@copilot open a new pull request to apply changes based on the comments in this thread |
|
@copilot open a new pull request to apply changes based on the comments in this thread |
…ity across multiple files
…MSBart2/agentrc into feat/workflow-automation-pillar
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (2)
packages/core/src/services/readiness/criteria.ts:552
pr-labelingcan pass via.github/labeler.yaml/.jsonor a workflow referencingactions/labeler, but theevidencelist only includes.github/labeler.yml. Alignevidence(and/or the failure reason) with whathasAutoLabeler()actually checks so users know all valid configurations.
const found = await hasAutoLabeler(context.repoPath);
return {
status: found ? "pass" : "fail",
reason: found
? undefined
: "No PR auto-labeler configured (.github/labeler.yml or actions/labeler workflow). Auto-labeling routes PRs to the right reviewers without manual triage.",
evidence: [".github/labeler.yml"]
};
packages/core/src/services/readiness.ts:951
pr-labelingcan pass via.github/labeler.yaml/.jsonor a workflow that referencesactions/labeler, but theevidencefield only lists.github/labeler.yml. Updateevidence(and/or the failure reason) so the report accurately reflects all accepted configuration paths.
const found = await hasAutoLabeler(context.repoPath);
return {
status: found ? "pass" : "fail",
reason: found
? undefined
: "No PR auto-labeler configured (.github/labeler.yml or actions/labeler workflow). Auto-labeling routes PRs to the right reviewers without manual triage.",
evidence: [".github/labeler.yml"]
};
| async function hasIssueTemplates(repoPath: string): Promise<boolean> { | ||
| const single = await fileExists(path.join(repoPath, ".github", "ISSUE_TEMPLATE.md")); | ||
| if (single) return true; | ||
| const dir = path.join(repoPath, ".github", "ISSUE_TEMPLATE"); | ||
| try { | ||
| const entries = await fs.readdir(dir); | ||
| return entries.some((e) => /\.(md|yml|yaml)$/iu.test(e)); | ||
| } catch { | ||
| return false; | ||
| } | ||
| } | ||
|
|
||
| async function hasCommitConvention(repoPath: string): Promise<boolean> { | ||
| const configs = [ | ||
| ".commitlintrc", | ||
| ".commitlintrc.json", | ||
| ".commitlintrc.yml", | ||
| ".commitlintrc.yaml", | ||
| ".commitlintrc.js", | ||
| ".commitlintrc.cjs", | ||
| ".commitlintrc.mjs", | ||
| ".commitlintrc.ts", | ||
| "commitlint.config.js", | ||
| "commitlint.config.cjs", | ||
| "commitlint.config.mjs", | ||
| "commitlint.config.ts", | ||
| ".cz-config.js", | ||
| ".czrc" | ||
| ]; | ||
| for (const config of configs) { | ||
| if (await fileExists(path.join(repoPath, config))) return true; | ||
| } | ||
| const pkg = await readJson(path.join(repoPath, "package.json")); | ||
| if (pkg) { | ||
| const allDeps = { | ||
| ...((pkg.dependencies as Record<string, unknown>) ?? {}), | ||
| ...((pkg.devDependencies as Record<string, unknown>) ?? {}) | ||
| }; | ||
| if ( | ||
| Object.keys(allDeps).some( | ||
| (dep) => dep.startsWith("@commitlint/") || dep === "commitlint" || dep === "commitizen" | ||
| ) | ||
| ) { | ||
| return true; | ||
| } | ||
| } | ||
| return false; | ||
| } | ||
|
|
||
| async function hasReleaseAutomation( | ||
| repoPath: string | ||
| ): Promise<{ found: boolean; evidence: string[] }> { | ||
| const evidence: string[] = []; | ||
| const configs = [ | ||
| "release.config.js", | ||
| "release.config.cjs", | ||
| "release.config.mjs", | ||
| "release.config.ts", | ||
| ".releaserc", | ||
| ".releaserc.json", | ||
| ".releaserc.yml", | ||
| ".releaserc.yaml", | ||
| ".releaserc.js", | ||
| ".releaserc.cjs", | ||
| ".changeset/config.json", | ||
| "release-please-config.json" | ||
| ]; | ||
| for (const config of configs) { | ||
| if (await fileExists(path.join(repoPath, config))) { | ||
| evidence.push(config); | ||
| } | ||
| } | ||
| const workflowsDir = path.join(repoPath, ".github", "workflows"); | ||
| try { | ||
| const workflows = await fs.readdir(workflowsDir); | ||
| for (const workflow of workflows) { | ||
| if (!/\.ya?ml$/iu.test(workflow)) continue; | ||
| try { | ||
| const content = await fs.readFile(path.join(workflowsDir, workflow), "utf8"); | ||
| if ( | ||
| content.includes("semantic-release") || | ||
| content.includes("changesets/action") || | ||
| content.includes("googleapis/release-please-action") | ||
| ) { | ||
| evidence.push(`.github/workflows/${workflow}`); | ||
| } | ||
| } catch { | ||
| // skip unreadable workflow | ||
| } | ||
| } | ||
| } catch { | ||
| // no workflows directory | ||
| } | ||
| return { found: evidence.length > 0, evidence }; | ||
| } | ||
|
|
||
| async function hasAutoLabeler(repoPath: string): Promise<boolean> { | ||
| for (const file of [".github/labeler.yml", ".github/labeler.yaml", ".github/labeler.json"]) { | ||
| if (await fileExists(path.join(repoPath, file))) return true; | ||
| } | ||
| const workflowsDir = path.join(repoPath, ".github", "workflows"); | ||
| try { | ||
| const workflows = await fs.readdir(workflowsDir); | ||
| for (const workflow of workflows) { | ||
| if (!/\.ya?ml$/iu.test(workflow)) continue; | ||
| try { | ||
| const content = await fs.readFile(path.join(workflowsDir, workflow), "utf8"); | ||
| if (content.includes("actions/labeler")) return true; | ||
| } catch { | ||
| // skip unreadable workflow | ||
| } | ||
| } | ||
| } catch { | ||
| // no workflows directory | ||
| } | ||
| return false; | ||
| } | ||
|
|
||
| async function hasBranchRulesets(repoPath: string): Promise<boolean> { | ||
| const rulesetsDir = path.join(repoPath, ".github", "rulesets"); | ||
| if (await fileExists(rulesetsDir)) { | ||
| try { | ||
| const entries = await fs.readdir(rulesetsDir); | ||
| if (entries.some((e) => e.endsWith(".json"))) return true; | ||
| } catch { | ||
| // skip unreadable directory | ||
| } | ||
| } | ||
| return fileExists(path.join(repoPath, ".github", "branch-protection.json")); | ||
| } |
There was a problem hiding this comment.
These new helper functions (hasIssueTemplates, hasCommitConvention, hasReleaseAutomation, hasAutoLabeler, hasBranchRulesets) duplicate logic that also exists in packages/core/src/services/readiness/checkers.ts. Keeping two copies increases the chance they drift; consider extracting them into a single shared module and reusing it from both the monolithic and modular readiness implementations.
| "Missing PR template (.github/PULL_REQUEST_TEMPLATE.md). A template with linked-issue and testing sections standardises agent-generated PR descriptions.", | ||
| evidence: [".github/PULL_REQUEST_TEMPLATE.md"] |
There was a problem hiding this comment.
The PR template criterion’s failure message/evidence only points to .github/PULL_REQUEST_TEMPLATE.md, but hasPullRequestTemplate() also accepts templates in .github/PULL_REQUEST_TEMPLATE/. Update the reason/evidence (and optionally the content-quality check) so the report reflects both supported locations.
This issue also appears on line 545 of the same file.
| "Missing PR template (.github/PULL_REQUEST_TEMPLATE.md). A template with linked-issue and testing sections standardises agent-generated PR descriptions.", | |
| evidence: [".github/PULL_REQUEST_TEMPLATE.md"] | |
| "Missing PR template (.github/PULL_REQUEST_TEMPLATE.md or .github/PULL_REQUEST_TEMPLATE/). A template with linked-issue and testing sections standardises agent-generated PR descriptions.", | |
| evidence: [".github/PULL_REQUEST_TEMPLATE.md", ".github/PULL_REQUEST_TEMPLATE/"] |
| "Missing PR template (.github/PULL_REQUEST_TEMPLATE.md). A template with linked-issue and testing sections standardises agent-generated PR descriptions.", | ||
| evidence: [".github/PULL_REQUEST_TEMPLATE.md"] |
There was a problem hiding this comment.
The PR template criterion’s failure message/evidence only points to .github/PULL_REQUEST_TEMPLATE.md, but hasPullRequestTemplate() also considers templates in .github/PULL_REQUEST_TEMPLATE/. This mismatch can mislead users into thinking they must use the single file even when a directory template is supported; update the reason/evidence (and optionally the content-quality check) to account for both locations.
This issue also appears on line 944 of the same file.
| "Missing PR template (.github/PULL_REQUEST_TEMPLATE.md). A template with linked-issue and testing sections standardises agent-generated PR descriptions.", | |
| evidence: [".github/PULL_REQUEST_TEMPLATE.md"] | |
| "Missing PR template (.github/PULL_REQUEST_TEMPLATE.md or .github/PULL_REQUEST_TEMPLATE/). A template with linked-issue and testing sections standardises agent-generated PR descriptions.", | |
| evidence: [".github/PULL_REQUEST_TEMPLATE.md", ".github/PULL_REQUEST_TEMPLATE/"] |
feat: add workflow-automation pillar with 6 new scored criteria
Add a new 'workflow-automation' pillar (repo-health group) with five new
checkers and six scored criteria covering process automation signals that
reduce engineering friction per PR:
New checkers (checkers.ts + monolithic readiness.ts):
New criteria (workflow-automation pillar):
New criterion (security-governance pillar):
Supporting changes:
Tests: update baseline/adapter tests for new pillar count and criterion IDs;
add 30 new integration tests covering all new criteria pass/fail scenarios
Various doc updates