Search results

[None][refactor] AutoDeploy: Use PyTorch-native format for NVFP4 weight_scale in IR ops, defer kernel-specific swizzling to fusion passes

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.

Model optimization<NV>Model-specific performance optimizations and tuning<NV>Model-specific performance optimizations and tuning

Status: Open.

Task

#11988 In NVIDIA/TensorRT-LLM;

· lucaslie opened on Mar 6, 2026

<code>config_file_lock()</code> fails with <code>OSError: [Errno 37] No locks available</code> on NFS-backed HF cache

Infra<NV>automated tests, build checks, github actions, system stability & efficiency.<NV>automated tests, build checks, github actions, system stability & efficiency.

Status: Open.

#11958 In NVIDIA/TensorRT-LLM;

· sara4dev opened on Mar 5, 2026

[Feature]: Support Dense Multi-Modal Qwen3.5 (e.g Qwen3.5-9b) unified HF checkpoints (ModelOpt NVFP4/FP8) in trtllm-serve PyTorch/AutoDeploy backends

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#11947 In NVIDIA/TensorRT-LLM;

· xyehya opened on Mar 5, 2026

[Bug]: Fix TPS/User glitches in BTK

bugSomething isn't workingSomething isn't working

Status: Open.

#11936 In NVIDIA/TensorRT-LLM;

· MrGeva opened on Mar 5, 2026

[Bug] TRTLLMGenFusedMoE + CUTLASS MoE both fail on SM120 (RTX PRO 6000) with FP4 Qwen3-Next MoE in v1.3.0rc4

Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.

Model optimization<NV>Model-specific performance optimizations and tuning<NV>Model-specific performance optimizations and tuning

Status: Open.

#11932 In NVIDIA/TensorRT-LLM;

· bhaktatejas922 opened on Mar 5, 2026

[Feature] [ AutoDeploy]: Sharding with multiple graphs

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#11928 In NVIDIA/TensorRT-LLM;

· govind-ramnarayan opened on Mar 5, 2026

[Feature]: AutoDeploy : Automatically apply weight preloading

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#11925 In NVIDIA/TensorRT-LLM;

· taylor-yb-lee opened on Mar 5, 2026

[AutoDeploy][Feature]: Qwen 3.5 MTP

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#11918 In NVIDIA/TensorRT-LLM;

· govind-ramnarayan opened on Mar 4, 2026

[Usage]: MPI Abort Error when runing executorExampleDisaggregated.cpp

Disaggregated serving<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.

questionFurther information is requestedFurther information is requested

Status: Open.

#11915 In NVIDIA/TensorRT-LLM;

· Fan-Yunfan opened on Mar 4, 2026

[Bug]: KvEvent Metrics, usedNumBlocks, can have negative block sizes in disagg/prefill mode

bugSomething isn't workingSomething isn't working

Disaggregated serving<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.

Status: Open.

#11879 In NVIDIA/TensorRT-LLM;

· michaelfeil opened on Mar 4, 2026

[Feature]: Add support for reusing kv cache across different LoRAs

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

KV-Cache Managementkv-cache management for efficient LLM inferencekv-cache management for efficient LLM inference

Lora/P-tuningParameter-Efficient Fine-Tuning (PEFT) like LoRA/P-tuning in TRTLLM: adapter use & perf.Parameter-Efficient Fine-Tuning (PEFT) like LoRA/P-tuning in TRTLLM: adapter use & perf.

Status: Open.

#11874 In NVIDIA/TensorRT-LLM;

· ShuaiShao93 opened on Mar 3, 2026

[Usage]: apply NVFP4 quantization to all Diffusers models (visual_gen)

questionFurther information is requestedFurther information is requested

VisualGen

Status: Open.

#11856 In NVIDIA/TensorRT-LLM;

· christopher5106 opened on Mar 3, 2026