fix(skills): address NVSkills validator findings on nemoclaw-user-* skills#4456
fix(skills): address NVSkills validator findings on nemoclaw-user-* skills#4456jyaunches wants to merge 1 commit into
Conversation
Apply the validator's fix recommendations from PR #4438 NVSkills CI runs (GitLab pipeline 52953267, job 328478588). Targets the static QUALITY/SCHEMA findings and the discoverability/efficiency dimension regressions called out on both nemoclaw-user-agent-skills and nemoclaw-user-get-started. Frontmatter (both skills): - Add metadata.author and metadata.tags so SKILL_SPEC recommended fields are populated. Clears 3x [MEDIUM] QUALITY findings per skill. Description (nemoclaw-user-agent-skills): - Rewrite to disambiguate from nemoclaw-skills-guide. Validator finding #1 on the 15:40 run identified codex conflating the two skills, dropping discoverability to 0.50 with skill_efficiency=0.0 on case 003. New description scopes this skill explicitly to install/load workflows and redirects discovery queries to nemoclaw-skills-guide. SKILL.md structure (both skills): - Add ## Instructions section. Clears [MEDIUM] SCHEMA finding 'Missing recommended section' and gives the LLM judge a stable anchor for instruction-quality scoring. - Add ## Examples section. Clears the matching [MEDIUM] SCHEMA finding. - Both Instructions sections explicitly cap tool invocations (one Read of one reference, no chaining), addressing validator fix item #4 'Cap or sequence tool invocations so skill_efficiency stops collapsing to 0.0 on multiple trials.' Out of scope for this commit: - Mirror to skills/<name>/ for NVSkills CI signing (separate commit). - evals.json adjustments.
📝 WalkthroughWalkthroughTwo NemoClaw agent skill prompts updated: the agent-skills skill now enforces a fixed interaction sequence with reference loading and tool budgets, while the get-started skill adds platform-aware onboarding with similar constraints and expanded metadata across both files. ChangesNemoClaw Agent Skills Documentation
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
E2E Advisor RecommendationRequired E2E: Dispatch hint: Auto-dispatched E2E: Full advisor summaryE2E Recommendation AdvisorBase: Required E2E
Optional E2E
New E2E recommendations
Dispatch hint
|
E2E Scenario Advisor RecommendationRequired scenario E2E: None Full scenario advisor summaryE2E Scenario AdvisorBase: Required scenario E2E
Optional scenario E2E
Relevant changed files
|
PR Review AdvisorFindings: 0 needs attention, 4 worth checking, 0 nice ideas Review findings🛠️ Needs attention
🔎 Worth checking
🌱 Nice ideas
This is an automated advisory review. A human maintainer must make the final merge decision. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
.agents/skills/nemoclaw-user-get-started/SKILL.md (1)
1-234:⚠️ Potential issue | 🔴 Critical | 🏗️ Heavy liftCritical: Direct edit to autogenerated file violates established workflow.
This file matches the pattern
.agents/skills/nemoclaw-user-*/*.md, which per the coding guidelines "are autogenerated and must never be edited directly." Retrieved learnings confirm these files are auto-generated fromdocs/viadocs-to-skills.py, and that changes should be made to the sourcedocs/content and then regenerated.This PR edits the file directly without any corresponding changes to
docs/source files. When the pre-commit hook next runsscripts/docs-to-skills.py, these manual edits will be overwritten and lost.Required fix: Move these changes to the corresponding source file under
docs/(likely a quickstart guide indocs/), then regenerate the.agents/skills/files using thedocs-to-skills.pyscript.As per coding guidelines: "User skills under
.agents/skills/nemoclaw-user-*/*.mdare autogenerated and must never be edited directly". Based on learnings: In NVIDIA/NemoClaw,.agents/skills/nemoclaw-user-*/SKILL.mdfiles are autogenerated outputs from the corresponding content underdocs/viadocs-to-skills.py. Reviewers should not hand-edit these files; any required changes should be made in the sourcedocs/content and then regenerated.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.agents/skills/nemoclaw-user-get-started/SKILL.md around lines 1 - 234, This change edits an autogenerated skill file (.agents/skills/nemoclaw-user-get-started/SKILL.md) which violates the workflow; revert the direct edits to that SKILL.md, make the intended content changes in the authoritative docs source under docs/ (locate the corresponding quickstart markdown in docs/), then run the generator scripts/docs-to-skills.py to regenerate the .agents/skills output (verify the updated SKILL.md appears), and commit the docs/ change and the regenerated artifacts together; reference the generator script name docs-to-skills.py and the SKILL.md pattern .agents/skills/nemoclaw-user-* to locate affected files..agents/skills/nemoclaw-user-agent-skills/SKILL.md (1)
1-45:⚠️ Potential issue | 🔴 Critical | 🏗️ Heavy liftCritical: Direct edit to autogenerated file violates established workflow.
This file matches the pattern
.agents/skills/nemoclaw-user-*/*.md, which per the coding guidelines "are autogenerated and must never be edited directly." Retrieved learnings confirm these files are auto-generated fromdocs/viadocs-to-skills.py, and that changes should be made to the sourcedocs/content and then regenerated.This PR edits the file directly without any corresponding changes to
docs/source files. When the pre-commit hook next runsscripts/docs-to-skills.py, these manual edits will be overwritten and lost.Required fix: Move these changes to the corresponding source file under
docs/(likelydocs/resources/agent-skills.mdxor a related file), then regenerate the.agents/skills/files using thedocs-to-skills.pyscript.As per coding guidelines: "User skills under
.agents/skills/nemoclaw-user-*/*.mdare autogenerated and must never be edited directly". Based on learnings: In NVIDIA/NemoClaw,.agents/skills/nemoclaw-user-*/SKILL.mdfiles are autogenerated outputs from the corresponding content underdocs/viadocs-to-skills.py. Reviewers should not hand-edit these files; any required changes should be made in the sourcedocs/content and then regenerated.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.agents/skills/nemoclaw-user-agent-skills/SKILL.md around lines 1 - 45, This SKILL.md under .agents/skills/nemoclaw-user-* was edited directly but those files are autogenerated; revert your changes in .agents/skills/nemoclaw-user-*/SKILL.md, make the intended edits in the canonical docs source (e.g., docs/resources/agent-skills.mdx or the appropriate file in docs/), then regenerate the autogenerated skill files by running scripts/docs-to-skills.py (or the repo's docs-to-skills workflow) so the updated content is propagated; ensure you reference the SKILL.md filename and docs-to-skills.py when committing so CI/pre-commit won't overwrite your work.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In @.agents/skills/nemoclaw-user-agent-skills/SKILL.md:
- Around line 1-45: This SKILL.md under .agents/skills/nemoclaw-user-* was
edited directly but those files are autogenerated; revert your changes in
.agents/skills/nemoclaw-user-*/SKILL.md, make the intended edits in the
canonical docs source (e.g., docs/resources/agent-skills.mdx or the appropriate
file in docs/), then regenerate the autogenerated skill files by running
scripts/docs-to-skills.py (or the repo's docs-to-skills workflow) so the updated
content is propagated; ensure you reference the SKILL.md filename and
docs-to-skills.py when committing so CI/pre-commit won't overwrite your work.
In @.agents/skills/nemoclaw-user-get-started/SKILL.md:
- Around line 1-234: This change edits an autogenerated skill file
(.agents/skills/nemoclaw-user-get-started/SKILL.md) which violates the workflow;
revert the direct edits to that SKILL.md, make the intended content changes in
the authoritative docs source under docs/ (locate the corresponding quickstart
markdown in docs/), then run the generator scripts/docs-to-skills.py to
regenerate the .agents/skills output (verify the updated SKILL.md appears), and
commit the docs/ change and the regenerated artifacts together; reference the
generator script name docs-to-skills.py and the SKILL.md pattern
.agents/skills/nemoclaw-user-* to locate affected files.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 3bc6eeb4-7436-45aa-9094-5eb0491483d0
📒 Files selected for processing (2)
.agents/skills/nemoclaw-user-agent-skills/SKILL.md.agents/skills/nemoclaw-user-get-started/SKILL.md
Selective E2E Results — ✅ All requested jobs passedRun: 26595363745
|
wscurran
added
enhancement: skill
3 participants
Summary
Applies the NVSkills validator's verbatim fix recommendations from the most recent failed validation runs in the context of #4438 (GitLab pipeline 52953267, job 328478588).
This PR edits source skills only in
.agents/skills/<name>/SKILL.md. A follow-up commit will mirror the changes intoskills/<name>/per the publishing flow introduced in #4448 so NVSkills CI signs the published copy.Targeted at
main(not stacked) so it can land independently of #4438.What this addresses
The validator flagged 5 [MEDIUM] static lints and 5 HIGH agent-eval regressions across the two user skills. This PR targets the structural items the validator can verify deterministically:
metadata.authormissing (MEDIUM × both skills)Miyoung Choi <miyoungc@nvidia.com>metadata.tagsmissing (MEDIUM × both)## Instructions(MEDIUM × both)## Examples(MEDIUM × both)nemoclaw-user-agent-skills— "codex conflates with nemoclaw-skills-guide"description:to scope to install/load and redirect discovery queries tonemoclaw-skills-guide## Instructionssections now cap to one Read of one reference, no chainingOut of scope (follow-up commit)
skills/nemoclaw-user-agent-skills/SKILL.mdandskills/nemoclaw-user-get-started/SKILL.mdso NVSkills CI sees them when signing.evals/evals.jsonrubric tuning.Test plan
After mirror commit lands, comment
/nvskills-cion the relevant published-skills PR and confirm:nemoclaw-user-agent-skillsrises from 0.50 toward the 0.70 threshold.skill_efficiencyno longer collapses to 0.0 on case 003 trials.Summary by CodeRabbit