[Bug]: Activations are only offloaded if layer offload fraction > 0 #1136
Description
What happened?
There is this toggle:
It offloads activations, not (only) the model itself. At higher batch sizes and resolutions, this is significant:
| Chroma vram | baseline | offloading |
|---|---|---|
| 2x1024 | 11776 | |
| 4x1024 | 13266 | 12234 |
| 8x1024 | OOM | 13938 |
But with CPU_OFFLOADED and this toggle on, nothing happens. Only if "Layer offload fraction" is > 0, activations appear to be offloaded, even if I set it to 1e-5.
I don't think this was intended.
What did you expect would happen?
see above
Relevant log output
Generate and upload debug_report.log
No response