Skip to content

Add checkpoint deletion options to configuration and Orbax checkpoint manager#3488

Open
awonak wants to merge 2 commits intoAI-Hypercomputer:mainfrom
awonak:awonak-orbax-options
Open

Add checkpoint deletion options to configuration and Orbax checkpoint manager#3488
awonak wants to merge 2 commits intoAI-Hypercomputer:mainfrom
awonak:awonak-orbax-options

Conversation

@awonak
Copy link

@awonak awonak commented Mar 23, 2026

Description

GCS does not support fast deletions. Checkpoint saves would get stuck on force deletion of a large checkpoint directory.

This change introduces the Orbax flag to "rename" old checkpoint saves instead of deleting them directly.

https://orbax.readthedocs.io/en/latest/api_reference/checkpoint.checkpoint_manager.html#checkpointmanageroptions

FIXES: b/485730473

Tests

TBD

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run end-to-end tests tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant