Skip to content

Conversation

@harsh-sutariya
Copy link
Contributor

  • Implement AdaBelief optimizer following the NeurIPS 2020 paper
  • Add unit tests following MLX standard patterns
  • Update optimizer documentation to include AdaBelief

Fixes #2479

Proposed changes

  • Added AdaBelief class to python/mlx/optimizers/optimizers.py
  • Implemented with bias correction enabled by default (matching paper recommendations)
  • Supports decoupled weight decay
  • Added unit tests in python/tests/test_optimizers.py
  • Updated documentation in docs/src/python/optimizers/common_optimizers.rst

Checklist

  • I have read the CONTRIBUTING document
  • I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the necessary documentation (if needed)

- Implement AdaBelief optimizer following the NeurIPS 2020 paper
- Add unit tests following MLX standard patterns
- Update optimizer documentation to include AdaBelief

Fixes ml-explore#2479
@zcbenz
Copy link
Collaborator

zcbenz commented Jan 18, 2026

As discussed in #1733 we aren't planning to add new optimizers unless there is a really compelling reason, please contributing this to https://github.com/stockeh/mlx-optimizers instead.

@zcbenz zcbenz closed this Jan 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Add AdaBelief Optimizer

2 participants