fix(skills): address NVSkills validator findings on user skills (stacked on #4438)#4455
Closed
jyaunches wants to merge 1 commit into
Closed
Conversation
Apply the validator's fix recommendations from PR #4438 NVSkills CI runs (GitLab pipeline 52953267, job 328478588). Targets the static QUALITY/SCHEMA findings and the discoverability/efficiency dimension regressions called out on both nemoclaw-user-agent-skills and nemoclaw-user-get-started. Frontmatter (both skills): - Add metadata.author and metadata.tags so SKILL_SPEC recommended fields are populated. Clears 3x [MEDIUM] QUALITY findings per skill. Description (nemoclaw-user-agent-skills): - Rewrite to disambiguate from nemoclaw-skills-guide. Validator finding #1 on the 15:40 run identified codex conflating the two skills, dropping discoverability to 0.50 with skill_efficiency=0.0 on case 003. New description scopes this skill explicitly to install/load workflows and redirects discovery queries to nemoclaw-skills-guide. SKILL.md structure (both skills): - Add ## Instructions section. Clears [MEDIUM] SCHEMA finding 'Missing recommended section' and gives the LLM judge a stable anchor for instruction-quality scoring. - Add ## Examples section. Clears the matching [MEDIUM] SCHEMA finding. - Both Instructions sections explicitly cap tool invocations (one Read of one reference, no chaining), addressing validator fix item #4 'Cap or sequence tool invocations so skill_efficiency stops collapsing to 0.0 on multiple trials.' Out of scope for this commit: - Mirror to skills/<name>/ for NVSkills CI signing (separate commit). - evals.json adjustments.
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Contributor
E2E Advisor RecommendationRequired E2E: None Full advisor summaryE2E Recommendation AdvisorBase: Required E2E
Optional E2E
New E2E recommendations
|
Contributor
E2E Scenario Advisor RecommendationRequired scenario E2E: None Full scenario advisor summaryE2E Scenario AdvisorBase: Required scenario E2E
Optional scenario E2E
Relevant changed files
|
Contributor
Author
|
Recreating off main instead of stacked on #4438. |
Contributor
PR Review AdvisorFindings: 3 needs attention, 1 worth checking, 0 nice ideas Review findings🛠️ Needs attention
🔎 Worth checking
🌱 Nice ideas
This is an automated advisory review. A human maintainer must make the final merge decision. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on top of #4438. Applies the NVSkills validator's verbatim fix recommendations from the most recent failed validation runs on
export-nemoclaw-user-agent-skills(GitLab pipeline 52953267, job 328478588).This PR edits source skills only (
.agents/skills/<name>/SKILL.md). A follow-up commit will mirror the changes intoskills/<name>/per the publishing flow introduced in #4448.What this addresses
Per the failure report, the validator flagged 5 [MEDIUM] static lints and 5 HIGH agent-eval regressions across the two user skills. This PR targets the structural items the validator can verify deterministically:
metadata.authormissing (MEDIUM × both)Miyoung Choi <miyoungc@nvidia.com>metadata.tagsmissing (MEDIUM × both)## Instructions(MEDIUM × both)## Examples(MEDIUM × both)nemoclaw-user-agent-skills— "codex conflates with nemoclaw-skills-guide"description:to scope to install/load and redirect discovery queries to nemoclaw-skills-guide## Instructionssections now cap to one Read of one reference, no chainingOut of scope (follow-up commit)
skills/nemoclaw-user-agent-skills/SKILL.mdandskills/nemoclaw-user-get-started/SKILL.mdso NVSkills CI sees them when signing.evals/evals.jsonrubric tuning.Test plan
After mirror commit lands, comment
/nvskills-cion #4438 and confirm:nemoclaw-user-agent-skillsrises from 0.50 toward the 0.70 threshold.skill_efficiencyno longer collapses to 0.0 on case 003 trials.