Mosaic memory profiling tutorial #3752
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Introduces a beginner tutorial demonstrating how to use Mosaic for GPU memory analysis in PyTorch. The tutorial covers: - Analyzing memory savings from activation checkpointing - Debugging unexpected memory usage from abandoned code - Integrating Mosaic into training pipelines for CI/CD Includes graceful handling for environments without GPU access.
Add HAS_MOSAIC_CLI check to skip Mosaic CLI subprocess calls when the mosaic package is not installed. This prevents FileNotFoundError in CI environments that have CUDA but don't have Mosaic installed.
Remove check=True from subprocess.run calls to prevent exceptions when Mosaic CLI commands fail. Instead, check return codes and print informative messages. This allows the tutorial to run in environments where Mosaic is partially installed or configured differently.
Set __main__.__file__ to a valid file path if not present. Transformers library reads this file to inspect source code, so we provide the tutorial file path or fall back to the transformers module path if __file__ is not available.
Wrap buggy model instantiation in try/except to handle ValueError from newer transformers versions that don't support experts implementation on GPT2Model. Falls back gracefully when the demo cannot run.
Co-authored-by: Svetlana Karslioglu <svekars@meta.com>
- Add tutorial to "What's new in PyTorch tutorials" section - Add customcarditem in Profiling section of index.rst - Add customcarditem and toctree entry in ecosystem.rst
…buggy model - Add GPT-2 memory profiling images (with/without activation checkpointing) - Add Google Colab download instructions for generated files - Fix subprocess.run to capture and print Mosaic CLI output - Split Mosaic analysis into separate code blocks for readability - Refactor GPT2WithDebugOverhead to use wrapper pattern instead of subclassing, fixing transformers version compatibility issues - Remove try/except workaround that was bypassing the tutorial's purpose - Update section formatting (bold headers instead of RST underlines)
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/3752
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
Contributor
Author
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tutorial Content Changes