[PyTorch] Add `pad_between_seqs` support for non-CP and CP (A2A and P2P) with FA3 + THD (varlen)#2596

Open

sudhakarsingh27 wants to merge 7 commits intoNVIDIA:mainfrom

sudhakarsingh27:flash_attn_pad_bw_seqs

Commits on Apr 24, 2026

[PyTorch] Add pad_between_seqs support for FlashAttention 3 with CP
sudhakarsingh27
committed
[PyTorch] Add pad_between_seqs tests for CP and non-CP FlashAttention
sudhakarsingh27
committed
[QA] Add CP deterministic tests to L3 and support TE_PATH in FA test
sudhakarsingh27
committed
[PyTorch] Fix FA3 deterministic gate to match upstream backward constraint
sudhakarsingh27
committed
[PyTorch] Disable FlashAttention 4 for pad_between_seqs with THD
sudhakarsingh27
committed
Merge branch 'main' of github.com:NVIDIA/TransformerEngine into flash_attn_pad_bw_seqs
sudhakarsingh27
committed

Commits on Apr 25, 2026

[QA] Fix cutlass-dsl utils shadow in FA versions test
sudhakarsingh27
committed