Fix: handle remaining meta tensors in from_single_file before dispatch #13117
+98
−0
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #12009
When loading models from single-file checkpoints (e.g., GGUF format), some parameters or buffers may not be present in the checkpoint and remain on the
metadevice afterload_model_dict_into_meta. This causesdispatch_modelto fail withCannot copy out of meta tensorerrors.Root Cause
In
from_single_file(), models are created underinit_empty_weights(), which initializes all tensors on themetadevice. After loading checkpoint weights viaload_model_dict_into_meta, any tensors not present in the checkpoint remain onmeta. This includes:freqs_cos,freqs_sininWanTransformer3DModel) — these are computed deterministically during__init__()and intentionally excluded from checkpointsFix
Three layers of handling before
dispatch_model():init_empty_weights()context, allowing deterministic buffers to be properly recomputedTesting
Validated with:
WanTransformer3DModel.from_single_file()with GGUF checkpoints (Q2_K, Q4_K_M, Q8_0)Specific models tested:
QuantStack/Wan2.2-T2V-A14B-GGUF(1095 params, 2 non-persistent buffers recomputed)QuantStack/Wan2.2-I2V-A14B-GGUF(safety net correctly catches 150+ missing I2V-specific params from incomplete key mapping)Before this fix
After this fix