Feature(backend): Add user toggle to run encoder models on CPU #8777
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds the ability to configure standalone text encoder models to run on CPU exclusively, thereby freeing up VRAM that might otherwise compete with the denoiser and other large models. Users can set a text encoder to run in CPU from the Model Manager, by clicking on a new toggle in the details area shown below:
All the text encoders are supported, including CLIPEmbed, T5Encoder, Qwen3Encoder, CLIPVision, SigLIP, and LlavaOnevision. However, Invoke only offers the option of changing the text encoder for some of the more recent main models, chiefly Flux.1, Flux.2 and Z-Image.
In most cases it does not make sense to run the text encoder on CPU, as execution speed suffers greatly (up to 5x slower for Qwen3 encoders). However, for users who have very low VRAM (e.g. 8 GB), this may allow them to run encoder models that would otherwise be inaccessible.
Related Issues / Discussions
Brief discussion on Discord regarding Comfy's use of a similar strategy: https://discord.com/channels/1020123559063990373/1020123559831539744/1462795385469796591
QA Instructions
CPU mode on
CPU mode off
Repeat this with other text encoders and main models.
Merge Plan
Simple merge.
Checklist
What's Newcopy (if doing a release after this PR)