Skip to content

added center_scale_norm#5

Merged
nkern merged 2 commits into
mainfrom
center_scale_norm
May 1, 2026
Merged

added center_scale_norm#5
nkern merged 2 commits into
mainfrom
center_scale_norm

Conversation

@nkern
Copy link
Copy Markdown
Owner

@nkern nkern commented May 1, 2026

Added center_scale_norm that centers and then normalizes by absolute deviation. Need to keep data centered to train diffusion models with a mean-zero noise scheduler

@nkern nkern requested a review from JiamingPan May 1, 2026 02:11
Copy link
Copy Markdown
Collaborator

@JiamingPan JiamingPan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two small questions:

  1. The function center_scale_norm still says median/absolute deviation, but the code uses mean/max-abs. Should we update the docstring?
  2. center_scale_norm currently mutates x in-place with x -= avg and x /= x.abs().max(). Since load_data can use tensors backed by NumPy memmaps, should this be out-of-place, e.g. x = x - x.mean(); x = x / x.abs().max(), to avoid the non-writable-array warning?

@nkern
Copy link
Copy Markdown
Owner Author

nkern commented May 1, 2026

great points, will fix this, thanks!

@nkern
Copy link
Copy Markdown
Owner Author

nkern commented May 1, 2026

@JiamingPan lmk if you have any other concerns! if not I will merge it

@nkern nkern merged commit b884ae5 into main May 1, 2026
2 checks passed
@nkern nkern deleted the center_scale_norm branch May 1, 2026 02:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants