Draft:Extended Tensor Parallelism#2960

Draft

jiemingz wants to merge 43 commits intoNVIDIA:mainfrom

jiemingz:cuda_graph_final

Commits on Mar 5, 2026

ETP
jiemingz
committed
cleanup
jiemingz
committed

Commits on Mar 10, 2026

fix post_hook not being called in certain cases
jiemingz
committed

Commits on Mar 19, 2026

Cudagraph Support
jiemingz
committed

Commits on Mar 30, 2026

Commits on Mar 31, 2026

Commits on Apr 3, 2026

Commits on Apr 7, 2026

import fix
fanshiqing
committed

Commits on Apr 9, 2026

move ag init to first pass

jiemingz
authored and
fanshiqing
committed
ETP+CG: 2-chain(dense+expert, no cross link prefetching) + shared ag/rs streams
fanshiqing
committed

Commits on Apr 10, 2026

fix the case when ETP_Config.weight_prefetch is False.
fanshiqing
committed

Commits on Apr 13, 2026

ETP+emb/output layers: remove these two layers from the prefetch chain.
fanshiqing
committed

Commits on Apr 15, 2026

Commits on Apr 16, 2026

Commits on Apr 17, 2026

Revert "ETP+CG: re-enable bwd ETP RS overlapping across Graphs."
fanshiqing
committed

Commits on Apr 19, 2026

ETP: fix iter-2 NaN + unbounded wgrad pool growth; partition streams
fanshiqing
committed

Commits on Apr 21, 2026

ETP+nvfp4: coalescing amax reduction for fprop groupedgemm
fanshiqing
committed

Commits on Apr 22, 2026

Commits on Apr 23, 2026

ETP+NVFP4: fused multi-tensor amax kernel
fanshiqing
committed

Commits on Apr 24, 2026

Commits on Apr 25, 2026

Commits on Apr 27, 2026