fix: allow wandb config value changes on resume#2137
fix: allow wandb config value changes on resume#2137gkaplun-nvidia wants to merge 1 commit intoNVIDIA-NeMo:mainfrom
Conversation
When resuming a wandb run, `config.update()` raises an error if a key already exists with a different value. This is common when hyperparameters are re-logged on resume. Adding `allow_val_change=True` prevents the crash while still updating the config. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughUpdated hyperparameter logging in both Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment Tip CodeRabbit can use Trivy to scan for security misconfigurations and secrets in Infrastructure as Code files.Add a .trivyignore file to your project to customize which findings Trivy reports. |
Summary
Add
allow_val_change=Truetowandb.config.update()calls in bothWandbLoggerandMLflowWandbLogger.Problem
When resuming a wandb run,
config.update()raises an error if a hyperparameter key already exists with a different value. This is common when hyperparameters are re-logged on resume (e.g., if the config is passed again duringsetup()).Fix
Pass
allow_val_change=Trueso that config values can be overwritten without raising. This matches the expected behavior for resumed runs where the same config is re-applied.Test plan
Summary by CodeRabbit