Add SeedVR2 support#110
Open
pollockjj wants to merge 7 commits into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
This pull request adds first-class SeedVR2 support across ComfyUI’s model loading/config detection, VAE encode/decode (including tiled + OOM fallback dispatch), latent format handling, and attention/RoPE-related regression coverage.
Changes:
- Introduces a new
SeedVR2supported model + model detection rules and amodel_base.SeedVR2wrapper. - Extends
comfy.sd.VAEto detect/load SeedVR2 VAEs, route tiled decode/encode correctly for SeedVR2 latent layouts, and add SeedVR2-specific memory estimation + latent post-processing. - Adds extensive unit/regression tests (and minor workflow/node schema updates) to lock in SeedVR2 contracts.
Reviewed changes
Copilot reviewed 56 out of 58 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests-unit/comfy_test/test_var_attention_pytorch_seedvr2_guard.py | Adds regression tests for the SeedVR2-specific guard behavior in PyTorch varlen attention. |
| tests-unit/comfy_test/test_vae_encode_tiled_seedvr2_method.py | Tests VAE.encode_tiled_seedvr2 existence, signature, and delegation semantics. |
| tests-unit/comfy_test/test_vae_encode_tiled_fallback_dispatcher_seedvr2.py | Tests OOM fallback routing for SeedVR2 vs generic 3D encode tiling. |
| tests-unit/comfy_test/test_vae_encode_tiled_explicit_dispatcher_seedvr2.py | Tests explicit encode_tiled dispatcher routing for SeedVR2 inputs. |
| tests-unit/comfy_test/test_vae_decode_tiled_dispatcher_seedvr2_4d.py | Tests decode OOM fallback routing for 4D collapsed SeedVR2 latents. |
| tests-unit/comfy_test/test_seedvr2_windows_static_verify.py | Static token verification for SeedVR2 node/VAE boundary contracts. |
| tests-unit/comfy_test/test_seedvr2_vae_graph_boundaries.py | Boundary tests ensuring node↔VAE interfaces preserve SeedVR2 latent contracts. |
| tests-unit/comfy_test/test_seedvr2_saved_latent_decode_boundary.py | Ensures saved/loaded SeedVR2 latents decode without re-running preprocessing. |
| tests-unit/comfy_test/test_seedvr2_resize_and_pad_pre_encode_state.py | Tests SeedVR2 resize/padding preprocessing node behavior. |
| tests-unit/comfy_test/test_seedvr2_refactor_nodes.py | Regression tests for node-side metadata/shape behaviors after refactor. |
| tests-unit/comfy_test/test_seedvr2_non_goal_static_audit.py | Optional git-diff audit test for “non-goal” files during local development. |
| tests-unit/comfy_test/test_seedvr2_hidden_state_static_audit.py | AST/static audit to prevent hidden state usage in SeedVR2 decode paths. |
| tests-unit/comfy_test/test_seedvr2_dtype.py | Tests dtype and conditioning/attention/RoPE-related SeedVR2 invariants. |
| tests-unit/comfy_test/test_seedvr_var_attention_backends.py | Tests attention backend registry, flags, fallbacks, and split backend semantics. |
| tests-unit/comfy_test/test_seedvr_vae_tiled_temporal_slicing.py | Tests temporal slicing behavior in tiled SeedVR VAE encode/decode. |
| tests-unit/comfy_test/test_seedvr_vae_tiled_encode_runt_slice_override.py | Tests encode slicing runt handling and min-size restoration on failure. |
| tests-unit/comfy_test/test_seedvr_vae_tiled_decode_latent_min_size_override.py | Tests decode slicing disablement and min-size restoration on failure. |
| tests-unit/comfy_test/test_seedvr_vae_tiled_decode_5d.py | Verifies JSON contract for an external “real probe” payload (schema-level). |
| tests-unit/comfy_test/test_seedvr_vae_tiled_args_no_mutate.py | Static regression ensuring tiled_args object state is not used. |
| tests-unit/comfy_test/test_seedvr_vae_loader_metadata.py | Tests SeedVR2 VAE loader metadata application in comfy.sd.VAE.__init__. |
| tests-unit/comfy_test/test_seedvr_vae_decode_unpadded_t.py | Tests frame padding/cropping behavior in SeedVR2 post-processing. |
| tests-unit/comfy_test/test_seedvr_vae_decode_guards.py | Tests SeedVR2 VAE wrapper decode input validation and tiling argument validation. |
| tests-unit/comfy_test/test_seedvr_vae_decode_batch_axes.py | Tests batch/time axis preservation across SeedVR2 wrapper decode paths. |
| tests-unit/comfy_test/test_seedvr_vae_attention_fence.py | Ensures SeedVR2 VAE self-attention uses VAE-local attention path. |
| tests-unit/comfy_test/test_seedvr_vae_5d_tiled_decode.py | Tests SeedVR2 decode tiling behavior, argument mapping, and layout preservation. |
| tests-unit/comfy_test/test_seedvr_rope_rewrite.py | Regression suite for SeedVR2 RoPE rewrite onto apply_rope1. |
| tests-unit/comfy_test/test_seedvr_rope_delegation.py | Ensures apply_rotary_emb delegates to apply_rope1 and matches oracle. |
| tests-unit/comfy_test/test_seedvr_latent_format.py | Tests SeedVR2 latent format behavior for empty latents and collapsed channels. |
| tests-unit/comfy_test/test_seedvr_groupnorm_limit.py | Regression tests for GroupNorm memory limit gating in SeedVR VAE. |
| tests-unit/comfy_test/test_seedvr_forward_no_device_cast.py | Ensures SeedVR forward paths don’t call get_torch_device at runtime. |
| tests-unit/comfy_test/test_seedvr_clear_vae_memory_soft_empty_cache.py | Ensures clear_vae_memory uses soft_empty_cache not torch.cuda.empty_cache. |
| tests-unit/comfy_test/test_seedvr_7b_final_block_text_path.py | Tests SeedVR2 7B/3B block routing invariants and rope buffer contracts. |
| tests-unit/comfy_test/test_diffusers_metadata_guard.py | Regression tests for diffusers metadata guard in comfy.sd.VAE.__init__. |
| tests-unit/comfy_test/seedvr_vae_wrapper_forward_test.py | Tests wrapper forward return contract and absence of .sample access. |
| tests-unit/comfy_test/seedvr_vae_forward_test.py | Tests SeedVR VAE forward contracts (no diffusers-style .latent_dist/.sample). |
| tests-unit/comfy_test/seedvr_model_test.py | Conditioning split hardening tests and AST pinning for SeedVR2 model forward path. |
| tests-unit/comfy_test/model_detection_test.py | Adds model detection coverage for SeedVR2 7B/3B checkpoints. |
| tests-unit/comfy_extras_test/test_seedvr2_node_boundaries.py | Tests SeedVR2 resize nodes remain preprocess-only and avoid encode/decode paths. |
| tests-unit/comfy_extras_test/test_seedvr_node_signature.py | Tests schema input IDs match execute() parameter order for SeedVR2 resize nodes. |
| nodes.py | Updates VAE tiled encode/decode node schemas and enables nodes_seedvr.py extra nodes. |
| comfy/supported_models.py | Adds SeedVR2 supported model configuration and dtype behavior. |
| comfy/supported_models_base.py | Extends set_inference_dtype signature to optionally accept device. |
| comfy/sd.py | Implements SeedVR2 VAE detection/loading, tiled dispatchers, memory estimation, and dtype helper. |
| comfy/sample.py | Preserves collapsed empty latent channel multiples for formats like SeedVR2. |
| comfy/model_detection.py | Adds SeedVR2 UNet config detection rules (7B/3B variants). |
| comfy/model_base.py | Adds SeedVR2 BaseModel wrapper and extra conditioning plumbing. |
| comfy/ldm/modules/diffusionmodules/model.py | Extends timestep embedding helper to support flip/shift parameters used by SeedVR2. |
| comfy/latent_formats.py | Adds SeedVR2 latent format and a preserve_empty_channel_multiples flag. |
| .gitignore | Ignores .pyisolate_venvs/. |
| .github/workflows/test-unit.yml | Runs unit tests on develop branch and removes continue-on-error. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+106
to
+111
| def test_missing_api_raises_seedvr2_runtime_error(monkeypatch): | ||
| monkeypatch.delattr(torch.nested, "nested_tensor_from_jagged", raising=False) | ||
| q, k, v, heads, cu_q, cu_k, _, _ = _inputs() | ||
|
|
||
| with pytest.raises(RuntimeError, match=r"SeedVR2.*nested_tensor_from_jagged"): | ||
| var_attention_pytorch(q, k, v, heads, cu_q, cu_k) |
Comment on lines
+126
to
+140
| def test_present_api_returns_expected_shape(): | ||
| q, k, v, heads, cu_q, cu_k, total_tokens, embed_dim = _inputs() | ||
|
|
||
| torch_fx_logger = logging.getLogger("torch.fx._symbolic_trace") | ||
| old_torch_fx_level = torch_fx_logger.level | ||
| torch_fx_logger.setLevel(logging.ERROR) | ||
| try: | ||
| with warnings.catch_warnings(): | ||
| warnings.filterwarnings( | ||
| "ignore", | ||
| message="The PyTorch API of nested tensors is in prototype stage.*", | ||
| category=UserWarning, | ||
| ) | ||
| out = var_attention_pytorch(q, k, v, heads, cu_q, cu_k) | ||
| finally: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds native SeedVR2 model, VAE, workflow nodes, sampler support, and regression coverage.