feat: Auto research skill#2419
Conversation
fafbf11 to
ca90191
Compare
|
/claude review |
terrykong
left a comment
There was a problem hiding this comment.
Review: PR #2419 — feat: Auto research skill
Nice work — this skill provides a well-structured framework for running iterative RL experiments with git as the experiment journal. The exploration-ideas guide and git-workflow reference are thorough and practical. The safety guardrails in git-workflow.md (no stash/reset/overwrite without consent) are particularly good.
A few suggestions to align with existing repo conventions and improve consistency:
Directory naming mismatch
All other skill directories use hyphens (build-and-dependency, config-conventions, launch-nemo-rl, etc.), but this one uses an underscore (auto_research). The frontmatter name field is auto-research (with hyphen), creating an inconsistency. Consider renaming the directory to auto-research/ to match the convention.
Nemo-gym coverage
The PR description mentions guiding agents on research with "Nemo-RL and Nemo-gym", but the SKILL.md workflow (step 3) only references NeMo-RL paths (examples/run_grpo.py, nemo_rl/models/, etc.). The Nemo-gym entrypoints (examples/nemo_gym/) are not mentioned. Consider either adding Nemo-gym paths to the workflow or adjusting the PR description to match the actual scope.
See inline comments for additional suggestions.
Generated by Claude Code
Signed-off-by: Vinh Nguyen <vinhn@nvidia.com>
29bb26f to
7aee365
Compare
|
Thanks, @terrykong, for the prompt review. Fixed the reported issues and tightened all 3 skills. Add best practices and gotchas observed with Codex, but could happen with other agents. |
Signed-off-by: Vinh Nguyen <vinhn@nvidia.com>
Signed-off-by: Vinh Nguyen <vinhn@nvidia.com>
Signed-off-by: Vinh Nguyen <vinhn@nvidia.com>
Signed-off-by: Vinh Nguyen <vinhn@nvidia.com>
Signed-off-by: Vinh Nguyen <vinhn@nvidia.com>
Signed-off-by: Vinh Nguyen <vinhn@nvidia.com>
Signed-off-by: Vinh Nguyen <vinhn@nvidia.com>
Signed-off-by: Vinh Nguyen <vinhn@nvidia.com>
Signed-off-by: Vinh Nguyen <vinhn@nvidia.com>
Signed-off-by: Vinh Nguyen <vinhn@nvidia.com>
Signed-off-by: Vinh Nguyen <vinhn@nvidia.com>
Signed-off-by: Vinh Nguyen <vinhn@nvidia.com>
Signed-off-by: Vinh Nguyen <vinhn@nvidia.com>
Signed-off-by: Vinh Nguyen <vinhn@nvidia.com>
Signed-off-by: Vinh Nguyen <vinhn@nvidia.com>
|
/ok to test a8b8513 |
|
/ok to test 54c71ec |
|
/ok to test 9880d5e |
|
/ok to test 39de685 |
What does this PR do ?
This PR adds an auto research skill that guides agents on how to do a prolonged research session with Nemo-RL and Nemo-gym. It sets some operating guidelines on how to form and test hypotheses, how to organize git branches, how to monitor and report progress, and how to explicitly check for the stopping conditions of the campaign.
Issues
N/A
Usage
You can prompt Codex, such as:
For this skill to be effective, Codex should have sufficient knowledge of the local operating environment (e.g. Slurm or local machine). A prerequisite to using the auto research skill is therefore, for the agent to be able to automatically run a baseline workload on the given environment.