**This RC version will be the last one supporting the TensorRT backend, in the next version the TensorRT backend will be removed!** - Known Issues - DeepSeek V3/V3.2 can crash with an illegal memory access or hang during warm up. - Autotuning for Qwen3-family models can crash with "Assertion failed: Failed to initialize cutlass TMA WS grouped gemm." - API - Add API to configure TeaCache coefficients (#13170) - BREAKING CHANGE: Make request `chat_template` opt-in (#14646) - Feature - Add DeepSeek V4 preparation (#15378, #15379, #15381, #15394, #15402, #15222) - Add MXFP8 weight format plus CUTLASS W8A8 Linear and MoE (#14962) - Add Marlin NVFP4 backend for MoE and Linear on Hopper (#13476) - Add CUDA graph wrapper for multimodal encoders (#14829) - Support cross-attention with FlashInfer TRT-LLM Gen kernels on Blackwell (#15429) - Support post-norm and per-aux `fc_norm` for Eagle3 draft models (Eagle 3.1) (#14988) - Add EPLB support for Qwen3.5 (#15543) - Optimize CuteDSL NVFP4 MoE grouped/SwiGLU GEMM accumulation pipeline (#15258) - Add CuTe DSL GVR-TopK load-balance optimization (#15304) - Enable split-KV heuristic for low-occupancy cross-attention in LTX-2 FA4 (#15399) - Fuse MLP up-GEMM + bias + GELU(tanh) + NVFP4-quant into the CuteDSL epilogue for LTX2 and WAN (#15299) - Add async mp4 encode and configurable noise latent via env vars in VisualGen (#15229) - Fix - Harden disagg cache transceiver teardown (#15422) - Fix encoder-decoder beam search corruption via per-slot `fragmentPointerDevice` (#15461) - Fix overallocation of draft KV cache (#15017) - Disable NCCL window buffers on GB10 (#15559) - Fix wrong NCCL fallback in nemotron-h (#15294) - Fix CuteDSL NVFP4 EPLB weight layout (#15538) - Enable CuTe DSL BF16 kernels for SM100 PP (#14993) - Fix Gemma4 multimodal vision TP and xgrammar startup crashes (#15566) - Add necessary methods for guided decoding in Kimi K2.5 (#15180) - Re-enable Ulysses for LTX-2 v2a cross-attention (#15303) - Fix passing scaled timestep to `time_embedder` in Cosmos3 (#15545) - Clarify and align trtllm-bench runtime logging (#15254) - Documentation - Add deploy guide for Minimax M3 (#15587) - Add Qwen Image visual generation examples (#15235) - Benchmark - Add Qwen-Image-Bench evaluator (#14837) - Add modularized perf tests for attention and MoE (discrete/continuous) (#15541) - Add Qwen3.5-397B-A17B-NVFP4 B200 aggregated perf-sanity tests (#15650) - Add DeepSeek R1 0528 FP4 performance test to `llm_perf_core.yml` (#15453) - Test & Infra - Move more test cases to post-merge (#15568) - Stabilize perf-sanity tests (#15440) - Avoid type checking failures due to pip dependency resolution (#15517) - Gate GPT-OSS TRT-LLM Gen MoE tests to SM100/SM103 (#15128) - Add GPT-OSS disagg test for transceiver v2 (#15301) - Fix Cosmos3 tests after VisualGen config split (#15170) - Fix visual gen test leaked issue (#15236) - Fix Qwen3-Next bf16 4gpu test (#15206) - Clean up Nemotron test cases (#15586) - Fix and unwaive step3p7 test cases (#15583) - Add test coverage for MiniMax model with multi-node M2.5 checkpoints eval (#15361) - Add GLM NVFP4 stress test (#15437) - Remove unreferenced accuracy tests and orphaned entries (#15593) - Update `.gitattributes` (#15606) ## What's Changed * [None][fix] AutoDeploy: Fixed wrong dist_backend AUTO detection when using trtllm-llmapi-launch by @MrGeva in https://github.com/NVIDIA/TensorRT-LLM/pull/15423 * [None][test] Waive 2 failed cases for main in QA CI by @tensorrt-cicd in https://github.com/NVIDIA/TensorRT-LLM/pull/15341 * [TRTLLMINF-81][feat] Avoid failed runners on infra retry by @dpitman-nvda in https://github.com/NVIDIA/TensorRT-LLM/pull/15237 * [https://nvbugs/6179661][fix] Harden disagg cache transceiver teardown by @chienchunhung in https://github.com/NVIDIA/TensorRT-LLM/pull/15422 * [https://nvbugs/6273846][test] gate GPT-OSS TRTLLM Gen MoE tests to SM100/SM103 by @dongfengy in https://github.com/NVIDIA/TensorRT-LLM/pull/15128 * [None][fix] avoid type checking failures due to pip dependency resolution by @ixlmar in https://github.com/NVIDIA/TensorRT-LLM/pull/15517 * [None][feat] VisualGen: async mp4 encode + fixed noise latent via env vars by @wu6u3tw in https://github.com/NVIDIA/TensorRT-LLM/pull/15229 * [https://nvbugs/6337235][test] Fix MX/GMS model loader fixtures by @chienchunhung in https://github.com/NVIDIA/TensorRT-LLM/pull/15471 * [None][test] Un-waive K2.5 Thinking FP4 disagg-NIXL e2e/gen_only tests by @chenfeiz0326 in https://github.com/NVIDIA/TensorRT-LLM/pull/15443 * [None][test] Waive 3 failed cases for main in QA CI by @tensorrt-cicd in https://github.com/NVIDIA/TensorRT-LLM/pull/15509 * [None][test] Waive 11 failed cases for main in QA CI by @tensorrt-cicd in https://github.com/NVIDIA/TensorRT-LLM/pull/15506 * [None][test] Waive 4 failed cases for main in QA CI by @tensorrt-cicd in https://github.com/NVIDIA/TensorRT-LLM/pull/15505 * [TRTLLM-13550][feat] WideEP FT: add MPI signal handler replacement (1d.0) by @chienchunhung in https://github.com/NVIDIA/TensorRT-LLM/pull/14160 * [None][test] Remove 60 closed-bug waive entries for main by @tensorrt-cicd in https://github.com/NVIDIA/TensorRT-LLM/pull/15511 * [#3237][fix] Support negative numbers in MajorityVote digit validation by @nikJ13 in https://github.com/NVIDIA/TensorRT-LLM/pull/12294 * [None][test] Waive 10 failed cases for main in post-merge by @tensorrt-cicd in https://github.com/NVIDIA/TensorRT-LLM/pull/15535 * [None][test] Waive 9 failed cases for main in QA CI by @tensorrt-cicd in https://github.com/NVIDIA/TensorRT-LLM/pull/15504 * [None][test] Waive 1 failed cases for main in QA CI by @tensorrt-cicd in https://github.com/NVIDIA/TensorRT-LLM/pull/15499 * [None][test] Waive 4 failed cases for main in QA CI by @tensorrt-cicd in https://github.com/NVIDIA/TensorRT-LLM/pull/15510 * [None][fix] AutoDeploy: handle torch dist all_gather in multi_stream MLA transform by @MrGeva in https://github.com/NVIDIA/TensorRT-LLM/pull/15456 * [None][feat] Add Gemma-4 NVFP4 quantized models to AutoDeploy registry by @marinayanov in https://github.com/NVIDIA/TensorRT-LLM/pull/15382 * [None][fix] Fix encoder-decoder beam search corruption via per-slot fragmentPointerDevice by @achartier in https://github.com/NVIDIA/TensorRT-LLM/pull/15461 * [https://nvbugs/6306936][test] Re-enable AutoDeploy disagg tests by @govind-ramnarayan in https://github.com/NVIDIA/TensorRT-LLM/pull/15325 * [None][infra] split single-node perf sanity GB200 by @tburt-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/15548 * [None][chore] Bump version to 1.3.0rc20 by @yuanjingx87 in https://github.com/NVIDIA/TensorRT-LLM/pull/15551 * [#10710][fix] clarify and align trtllm-bench runtime logging by @marinayanov in https://github.com/NVIDIA/TensorRT-LLM/pull/15254 * [https://nvbugs/6290345][fix] Fix allreduce benchmark input setup by @nv-lschneider in https://github.com/NVIDIA/TensorRT-LLM/pull/15427 * [None][feat] DSv4 prep: IndexerTopK and TopK primitives by @lfr-0531 in https://github.com/NVIDIA/TensorRT-LLM/pull/15381 * [None][perf] Cutedsl NVF4 MOE: grouped/swiglu GEMM: Fix acc pipeline release arrive threads + FC2 meta stage code clean by @liyuhannnnn in https://github.com/NVIDIA/TensorRT-LLM/pull/15258 * [https://nvbugs/6271740][test] Update llm_perf_core.yml to include new performance test for DeepSeek R1 0528 FP4 model by @yufeiwu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/15453 * [None][fix] Stabilize perf-sanity tests by @chenfeiz0326 in https://github.com/NVIDIA/TensorRT-LLM/pull/15440 * [None][test] fix Cosmos3 tests after VisualGen config split by @bobboli in https://github.com/NVIDIA/TensorRT-LLM/pull/15170 * [None][feat] DSv4 prep: compressor and mHC primitives by @lfr-0531 in https://github.com/NVIDIA/TensorRT-LLM/pull/15379 * [None][infra] Waive 3 failed cases for main in post-merge 2802 by @ZhanruiSunCh in https://github.com/NVIDIA/TensorRT-LLM/pull/15571 * [https://nvbugs/6264844][fix] Fix wrong NCCL fallback in nemotron-h by @Wanli-Jiang in https://github.com/NVIDIA/TensorRT-LLM/pull/15294 * [None][test] Waive 6 failed cases for main in QA CI by @tensorrt-cicd in https://github.com/NVIDIA/TensorRT-LLM/pull/15570 * [https://nvbugs/6344108][fix] skip TestNemotron3Super120B on pre-blackwell by @bo-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/15539 * [None][fix] Fix passing scaled timestep to time_embedder in Cosmos3 by @bastefaniak in https://github.com/NVIDIA/TensorRT-LLM/pull/15545 * [None][chore] Remove nv-internal-release guardword comments in mega_moe_nvfp4 by @xxi-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/15575 * [None][ci] move more test cases to post merge by @QiJune in https://github.com/NVIDIA/TensorRT-LLM/pull/15568 * [https://nvbugs/6185146][fix] Use `mat_a.new_empty([m, n_out//2])` / `input_scale.new_empty([sf_size])` in the by @tensorrt-cicd in https://github.com/NVIDIA/TensorRT-LLM/pull/14710 * [TRTLLM-35882][feat] cute dsl gvr-topk load-balance optimization by @limin2021 in https://github.com/NVIDIA/TensorRT-LLM/pull/15304 * [None][test] Waive 2 failed cases for main in QA CI by @tensorrt-cicd in https://github.com/NVIDIA/TensorRT-LLM/pull/15579 * [None][test] waive hang issues by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/15576 * [None][test] waive hang issues by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/15581 * [#14874][feat] AutoDeploy : Perf optimization for gpt-oss-120b for low conc by @taylor-yb-lee in https://github.com/NVIDIA/TensorRT-LLM/pull/15531 * [TRTLLM-12982][perf] reuse multi-item scoring position_ids and params by @ixlmar in https://github.com/NVIDIA/TensorRT-LLM/pull/15413 * [TRTLLM-13599][test] Refine Qwen3.5 test cases by @nv-guomingz in https://github.com/NVIDIA/TensorRT-LLM/pull/15544 * [TRTLLMINF-111][infra] Reuse image sqsh file by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/15147 * [None][feat] DSv4 prep: MoE routing and backend support by @lfr-0531 in https://github.com/NVIDIA/TensorRT-LLM/pull/15402 * [None][feat] DSv4 prep: runtime cache foundations by @lfr-0531 in https://github.com/NVIDIA/TensorRT-LLM/pull/15378 * [https://nvbugs/6156233][fix] Lower GSM8K reference for the three GPT-OSS/20B-MXFP4 entries with… by @tensorrt-cicd in https://github.com/NVIDIA/TensorRT-LLM/pull/15393 * [None][chore] Small cleanups to MultimodalModelMixin by @2ez4bz in https://github.com/NVIDIA/TensorRT-LLM/pull/15322 * [TRTLLM-13123][feat] CUDA graph wrapper for multimodal encoders by @2ez4bz in https://github.com/NVIDIA/TensorRT-LLM/pull/14829 * [TRTLLM-12622][feat] Add native post-processing hook to trtllm-serve by @xwang233 in https://github.com/NVIDIA/TensorRT-LLM/pull/15239 * [None][feat] Add Qwen Image visual generation examples by @yibinl-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/15235 * [TRTLLM-13490][feat] Support cross-attention with FlashInfer TRTLLM-Gen kernels on Blackwell by @cascade812 in https://github.com/NVIDIA/TensorRT-LLM/pull/15429 * [None][fix] LTX-2: re-enable Ulysses for v2a cross-attention by @luyiyun1021 in https://github.com/NVIDIA/TensorRT-LLM/pull/15303 * [TRTLLM-13246][feat] Wave 1: migrate aliases to setup_aliases and stage GMS RO load by @chienchunhung in https://github.com/NVIDIA/TensorRT-LLM/pull/15014 * [None][feat] Support post-norm and per-aux fc_norm for Eagle3 draft models by @Dogacel in https://github.com/NVIDIA/TensorRT-LLM/pull/14988 * [None][fix] fix FA4 install in devel docker by @o-stoner in https://github.com/NVIDIA/TensorRT-LLM/pull/14706 * [https://nvbugs/6276842][test] Loosen rtol/atol on encoder CUDA graph logits parity check by @tingyangk in https://github.com/NVIDIA/TensorRT-LLM/pull/15527 * [#15179][fix] Add necessary methods for guided decoding in Kimi K2.5 by @chungen04 in https://github.com/NVIDIA/TensorRT-LLM/pull/15180 * [None][test] Waive failed unittest on all devices (nvbugs/6335726) by @guqiqi in https://github.com/NVIDIA/TensorRT-LLM/pull/15585 * [None][infra] add blossom-ci authorized users by @niukuo in https://github.com/NVIDIA/TensorRT-LLM/pull/15549 * [https://nvbugs/6166097][fix] Fix CuteDSL NVFP4 EPLB weight layout by @nv-xtf in https://github.com/NVIDIA/TensorRT-LLM/pull/15538 * [None][test] GPT-OSS disagg test for transceiver v2 by @Shixiaowei02 in https://github.com/NVIDIA/TensorRT-LLM/pull/15301 * [None][feat] Add BaseResourceManager-based KV-cache compression manager framework by @Hudayday in https://github.com/NVIDIA/TensorRT-LLM/pull/15106 * [None][infra] use default split when CBTS test-db download fails by @crazydemo in https://github.com/NVIDIA/TensorRT-LLM/pull/15592 * [#12715][fix] Disable NCCL window buffers on GB10 by @nv-lschneider in https://github.com/NVIDIA/TensorRT-LLM/pull/15559 * [TRTLLM-11353][feat] API to configure TeaCache coefficients by @o-stoner in https://github.com/NVIDIA/TensorRT-LLM/pull/13170 * [TRTLLM-12242][feat] Add Marlin NVFP4 backend for MoE and Linear on Hopper by @xuantengh in https://github.com/NVIDIA/TensorRT-LLM/pull/13476 * [https://nvbugs/6094068][fix] Fix Qwen3-Next bf16 4gpu test by @JadoTu in https://github.com/NVIDIA/TensorRT-LLM/pull/15206 * [None][feat] Dis-agg transceiver mass integration from the DSV4 branch by @Shixiaowei02 in https://github.com/NVIDIA/TensorRT-LLM/pull/15222 * [https://nvbugs/6224637][fix] Enable CuTe DSL BF16 kernels for SM100 PP by @yuxianq in https://github.com/NVIDIA/TensorRT-LLM/pull/14993 * [https://nvbugs/6256531][test] Unwaive Llama guided decoding xgrammar by @sunnyqgg in https://github.com/NVIDIA/TensorRT-LLM/pull/15240 * [None][feat] DSv4: sparse cache manager adapter by @lfr-0531 in https://github.com/NVIDIA/TensorRT-LLM/pull/15394 * [TRTLLM-12982][chore] relocate `torch_multi_arange` by @ixlmar in https://github.com/NVIDIA/TensorRT-LLM/pull/15416 * [None][infra] Waive 15 failed cases for main in post-merge 2804 by @ZhanruiSunCh in https://github.com/NVIDIA/TensorRT-LLM/pull/15620 * [None][test] Waive hang issues by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/15609 * [TRTLLM-13371][perf] LTX-2 FA4: enable split-KV heuristic (num_splits=0) for low-occupancy cross-attn by @luyiyun1021 in https://github.com/NVIDIA/TensorRT-LLM/pull/15399 * [https://nvbugs/6346546][fix] fix mRoPE CUDA graph gate for text requests by @yechank-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/15589 * [https://nvbugs/6274932] [fix] Fix and unwaive step3p7 test cases by @kaiyux in https://github.com/NVIDIA/TensorRT-LLM/pull/15583 * [TRTLLM-13600][test] Clean up Qwen3 test cases by @nv-guomingz in https://github.com/NVIDIA/TensorRT-LLM/pull/15591 * [TRTLLM-13601][test] Clean up Nemotron test cases by @nv-guomingz in https://github.com/NVIDIA/TensorRT-LLM/pull/15586 * [None][infra] Fix node list query failing on tcsh login nodes by @yiqingy0 in https://github.com/NVIDIA/TensorRT-LLM/pull/15623 * Revert "[TRTLLM-12622][feat] Add native post-processing hook to trtllm-serve" by @tburt-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/15629 * [TRTLLM-13612][test] Remove unreferenced accuracy tests and orphaned … by @nv-guomingz in https://github.com/NVIDIA/TensorRT-LLM/pull/15593 * [https://nvbugs/6215688][fix] Fix visual gen test leaked issue by @yibinl-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/15236 * [https://nvbugs/6021427][fix] BREAKING CHANGE: Make request chat_template opt-in by @yibinl-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/14646 * [None][infra] AutoDeploy: Add trtllm runner for standalone llm-c by @bmarimuthu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/15630 * [https://nvbugs/6274614][fix] remove spec tokens env for stress test by @chuangz0 in https://github.com/NVIDIA/TensorRT-LLM/pull/15153 * [TRTLLM-13444][test] Add Qwen-Image text-to-image unit tests by @yingguo-trt in https://github.com/NVIDIA/TensorRT-LLM/pull/15580 * [TRTLLM-13247][feat] Wave 2: stage Linear and Attention transforms by @chienchunhung in https://github.com/NVIDIA/TensorRT-LLM/pull/15288 * [https://nvbugs/6368480][fix] Cache the SM count once in FmhaDispatcher's constructor and reuse the cached… by @chenfeiz0326 in https://github.com/NVIDIA/TensorRT-LLM/pull/15611 * [None][test] Add modularized perf tests (attention + MoE discrete/continuous) by @ruodil in https://github.com/NVIDIA/TensorRT-LLM/pull/15541 * [#15565][fix] AutoDeploy: Fix Super MTP IMA introduced by checkpointing replay by @galagam in https://github.com/NVIDIA/TensorRT-LLM/pull/15622 * [#15613][fix] Gemma4 multimodal: fix vision TP and xgrammar startup crashes by @Thachnh in https://github.com/NVIDIA/TensorRT-LLM/pull/15566 * [TRTLLM-12762][test] Add Test coverage for MiniMax Model with multi-node, M2.5 checkpoints eval by @jieli-matrix in https://github.com/NVIDIA/TensorRT-LLM/pull/15361 * [TRTLLM-13575][feat] Add EPLB support for Qwen3.5 by @nv-guomingz in https://github.com/NVIDIA/TensorRT-LLM/pull/15543 * [None][test] add GLM nvfp4 stress test by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/15437 * [TRTLLM-12982][chore] improve multi-item scoring request validation by @ixlmar in https://github.com/NVIDIA/TensorRT-LLM/pull/15627 * [None][test] Add Qwen3.5-397B-A17B-NVFP4 B200 aggregated perf-sanity tests by @chenfeiz0326 in https://github.com/NVIDIA/TensorRT-LLM/pull/15650 * [None][infra] take test durations into account to determine cbts splits num by @crazydemo in https://github.com/NVIDIA/TensorRT-LLM/pull/15614 * [None][doc] Add deploy guide for Minimax M3 by @WeiHaocheng in https://github.com/NVIDIA/TensorRT-LLM/pull/15587 * [None][chore] Update .gitattributes by @ziyixiong-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/15606 * [https://nvbugs/6239637][fix] Unwaive Qwen3.5 cases on A100 platform by @nv-guomingz in https://github.com/NVIDIA/TensorRT-LLM/pull/15481 * [TRTLLM-13712][feat] Add Qwen-Image-Bench evaluator by @yibinl-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/14837 * [https://nvbugs/6248783][test] Unwaive Qwen3 skip softmax test by @bobboli in https://github.com/NVIDIA/TensorRT-LLM/pull/15652 * [None][fix] User/tjohnsen/evict empty blocks first by @thorjohnsen in https://github.com/NVIDIA/TensorRT-LLM/pull/11685 * [https://nvbugs/6293536][fix] Stage KV block offsets through a fresh host buffer by @thorjohnsen in https://github.com/NVIDIA/TensorRT-LLM/pull/15546 * [TRTLLM-13370][perf] LTX2 + WAN: Fuse MLP up-GEMM + bias + GELU(tanh) + NVFP4-quant into CuteDSL epilogue by @luyiyun1021 in https://github.com/NVIDIA/TensorRT-LLM/pull/15299 * [https://nvbugs/6248837][chore] waive memory polluters by @tburt-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/15665 * [https://nvbugs/6269778][fix] Fix overallocation of draft KV cache by @mikeiovine in https://github.com/NVIDIA/TensorRT-LLM/pull/15017 * [None][feat] add MXFP8 weight format + CUTLASS W8A8 Linear and MoE by @WeiHaocheng in https://github.com/NVIDIA/TensorRT-LLM/pull/14962 * [https://nvbugs/6344612][test] relax GPT-OSS GPQA references due to high variance in random sampling by @dongfengy in https://github.com/NVIDIA/TensorRT-LLM/pull/15567 * [https://nvbugs/6062416][fix] Cache NCCL window allocation failures by size by @nv-lschneider in https://github.com/NVIDIA/TensorRT-LLM/pull/15596 ## New Contributors * @nikJ13 made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/12294 * @bastefaniak made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/15545 * @Dogacel made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/14988 * @guqiqi made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/15585 * @nv-xtf made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/15538 * @xuantengh made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/13476 * @Thachnh made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/15566 **Full Changelog**: https://github.com/NVIDIA/TensorRT-LLM/compare/v1.3.0rc19...v1.3.0rc20

## What's Changed * [None][feat] Support tensor parallelism for nemotron-h model by @Wanli-Jiang in https://github.com/NVIDIA/TensorRT-LLM/pull/11470 * [None][test] Add tests for all database configs. by @fsaady in https://github.com/NVIDIA/TensorRT-LLM/pull/11653 * [https://nvbugs/5911143][fix] add async worker to MTP/Eagle3 sampler,… by @dhansen-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/11573 * [TRTLLM-10886][feat] Support PARD(Parallel Draft Model) in one-model spec dec by @ziyixiong-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11438 * [None][fix] Fix disagg cancellation by @Tabrizian in https://github.com/NVIDIA/TensorRT-LLM/pull/11730 * [None][fix] Use prefer_pinned() in pard.py by @mikeiovine in https://github.com/NVIDIA/TensorRT-LLM/pull/11762 * [None][fix] Make KVCacheManagerV2 release mem immediately on shutdown by @lowsfer in https://github.com/NVIDIA/TensorRT-LLM/pull/11746 * [TRTLLM-11115][feat] enable autotuner for visual gen + Compilation Config by @NVShreyas in https://github.com/NVIDIA/TensorRT-LLM/pull/11660 * [None][chore] Minor fix in w4a8 mxfp4 mxfp8 test. by @Tracin in https://github.com/NVIDIA/TensorRT-LLM/pull/11745 * [None][infra] Move B200 test stage to AIHub by @yuanjingx87 in https://github.com/NVIDIA/TensorRT-LLM/pull/11692 * [None][infra] Waive failed cases for main on 02/27 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/11770 * [TRTLLM-11064][fix] Remove duplicated MoE Computation with Helix CP+DP by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11167 * [TRTLLM-10386][fix] torch.compile: register add+norm fallback pass in multi-GPU mode by @luyiyun1021 in https://github.com/NVIDIA/TensorRT-LLM/pull/11739 * [None][feat] Support heterogeneous tokens_per_block by @lowsfer in https://github.com/NVIDIA/TensorRT-LLM/pull/11751 * [None][chore] Remove closed bugs by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11527 * [None][test] local wheel installation support and add gb300 cases demo by @fredricz-20070104 in https://github.com/NVIDIA/TensorRT-LLM/pull/11742 * [None][feat] Refactor cache manager v2 to simplify new model support by @jiaganc in https://github.com/NVIDIA/TensorRT-LLM/pull/11749 * [https://nvbugs/5879614][fix] Waive test_guided_decoding_with_eagle3 xgrammar in disaggregated serving by @ziyixiong-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11773 * [https://nvbugs/5911788][test] Waive test_llm_partial_update_weights[Qwen3/Qwen3-8B] by @liji-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11785 * [None][feat] add globaltimer-based timing backend for autotuner profi… by @dhansen-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/11657 * [https://nvbugs/5926823][fix] Propagate logprobs from prefill to decode in disagg by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11727 * [TRTLLMINF-9][chore] Remove submodule pulls from TRT-LLM git checkouts by @dpitman-nvda in https://github.com/NVIDIA/TensorRT-LLM/pull/11693 * [https://nvbugs/5685010][fix] Delete test_eagle3_output_repetition_4gpus flaky assertions. by @zheyuf in https://github.com/NVIDIA/TensorRT-LLM/pull/11725 * [None][fix] enable separate draft KV cache pool for aggregated + KVBM… by @zyang-Modular in https://github.com/NVIDIA/TensorRT-LLM/pull/11689 * [TRTLLM-11058][feat] Support Helix CP with GQA by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11570 * [None][perf] Vectorize quantize_fp8_blockwise with CUDA kernel by @karljang in https://github.com/NVIDIA/TensorRT-LLM/pull/11724 * [https://nvbugs/5868616][fix] Fix warnings when building moe_kernels.cu by @yumin066 in https://github.com/NVIDIA/TensorRT-LLM/pull/11703 * [None][chore] Add CI trigger and test failure retrieval instructions to AGENTS.md by @lucaslie in https://github.com/NVIDIA/TensorRT-LLM/pull/11803 * [None][fix] Fix typo: avaiable_blocks -> available_blocks in scheduler by @kaiyux in https://github.com/NVIDIA/TensorRT-LLM/pull/11801 * [TRTLLM-11568][feat] Fix collective calls by @greg-kwasniewski1 in https://github.com/NVIDIA/TensorRT-LLM/pull/11632 * [None][perf] Use F.rms_norm for per-head QK normalization in visual gen by @karljang in https://github.com/NVIDIA/TensorRT-LLM/pull/11798 * [TRTLLM-11185][test] Add back WAN VBench test in CI by @chang-l in https://github.com/NVIDIA/TensorRT-LLM/pull/11804 * [TRTLLM-9782][feat] Support to skip KV cache memory estimation by @HuiGao-NV in https://github.com/NVIDIA/TensorRT-LLM/pull/11714 * [None][doc] Fix typos, grammar, and accuracy across documentation by @kaiyux in https://github.com/NVIDIA/TensorRT-LLM/pull/11766 * [None][fix] cleanup mem in rollout process by @hchings in https://github.com/NVIDIA/TensorRT-LLM/pull/11658 * [None][feat] Add --served-model-name option to serve command by @slin1237 in https://github.com/NVIDIA/TensorRT-LLM/pull/11711 * [None][chore] Update AGENTS.md by @lucaslie in https://github.com/NVIDIA/TensorRT-LLM/pull/11809 * [None][fix] AutoDeploy: Fix shape handling for singleton prefill by @galagam in https://github.com/NVIDIA/TensorRT-LLM/pull/11679 * [None][infra] Waive failed cases for main on 03/01 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/11811 * [None][feat] TRT-LLM Gen MoE finalize kernel optimization by @nekorobov in https://github.com/NVIDIA/TensorRT-LLM/pull/11501 * [None][test] Add E2E test for cancelled disagg gen request with overlap scheduler by @Tabrizian in https://github.com/NVIDIA/TensorRT-LLM/pull/11795 * [None][chore] pass nsight options to ray_executor and trigger profiling through collective_rpc by @davidmlw in https://github.com/NVIDIA/TensorRT-LLM/pull/11493 * [TRTLLM-10962][feat] Refactor video encoding to use ffmpeg CLI or pur… by @JunyiXu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11672 * [https://nvbugs/5823212][fix] Warmup maybe_compiled_cat in forward_context_with_chunked_prefill by @yuantailing in https://github.com/NVIDIA/TensorRT-LLM/pull/11743 * [None][feat] Extract embeding as .savetensors and support float8 quantized model by @nvyocox in https://github.com/NVIDIA/TensorRT-LLM/pull/11180 * [https://nvbugs/5885070][fix] fix deepeplowlatency with cutedsl moe backend by @leslie-fang25 in https://github.com/NVIDIA/TensorRT-LLM/pull/11769 * [None][fix] Fix FP8 per-tensor torch.compile graph break in dynamic quantization by @karljang in https://github.com/NVIDIA/TensorRT-LLM/pull/11759 * [TRTLLM-9687][feat] Improve are_stop_words performance by @stnie in https://github.com/NVIDIA/TensorRT-LLM/pull/11196 * [https://nvbugs/5883738][fix] fix bug for illegal memory access on Qwen3-235B-A22B-Thinking-2507-NVFP4 + Eagle3 by @sunnyqgg in https://github.com/NVIDIA/TensorRT-LLM/pull/11474 * [#10693][chore] AutoDeploy: Add L1 tests from coverage dashboard by @marinayanov in https://github.com/NVIDIA/TensorRT-LLM/pull/11530 * [https://nvbugs/5764627][fix] Fix generation logits with streaming and improve runtime of logits testcase. Also fixes https://nvbugs/5573238 by @stnie in https://github.com/NVIDIA/TensorRT-LLM/pull/10637 * [https://nvbugs/5934461][fix] Propagate logits from prefill to decode in disagg by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11767 * [#11726][feat] AutoDeploy: Fuse gemms of mixed children by @taylor-yb-lee in https://github.com/NVIDIA/TensorRT-LLM/pull/11793 * [None][fix] Fix overly aggressive capacity scheduler by @jthomson04 in https://github.com/NVIDIA/TensorRT-LLM/pull/11731 * [https://nvbugs/5689262][fix] use proper tokens when exclude_input_in_output is true by @lazykyama in https://github.com/NVIDIA/TensorRT-LLM/pull/9453 * [https://nvbugs/5863912][fix] Fix with move launch_dependent_grids after tmem free by @benzh-2025 in https://github.com/NVIDIA/TensorRT-LLM/pull/11812 * [https://nvbugs/5938603][fix] Fix E/PD disagg chunked prefill bug by @2ez4bz in https://github.com/NVIDIA/TensorRT-LLM/pull/11805 * [None][test] add deepseek RCCA perf test case by @ruodil in https://github.com/NVIDIA/TensorRT-LLM/pull/11736 * [None][fix] remove torch compile models arg by @NVShreyas in https://github.com/NVIDIA/TensorRT-LLM/pull/11836 * [None][test] add b200 multi nodes tests db by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11783 * [None][fix] Fix SM120 issue for rms_norm with nvfp4_quant_fusion by @Wanli-Jiang in https://github.com/NVIDIA/TensorRT-LLM/pull/11774 * [None][infra] Waive failed cases for main for post-merge 2564 by @ZhanruiSunCh in https://github.com/NVIDIA/TensorRT-LLM/pull/11848 * [https://nvbugs/5936502][fix] remove dead codes by @bo-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11813 * [None][chore] a GitHub Action to assign the PR to the author by @zhenhuaw-me in https://github.com/NVIDIA/TensorRT-LLM/pull/11673 * [None][infra] Fix a typo in waives.txt by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/11852 * [None][test] Fix wrong lora config by @yufeiwu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11818 * [None][test] fix flaky issues by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11814 * [None][fix] Fix OOM issue/dummy request allocation/chunked prefill/pp for KV Cache Manager V2 by @yizhang-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11710 * [None][test] update waive list by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11815 * [TRTLLM-9939][perf] Short-sequence MHA optimization for DSA MLA prefill by @kaiyux in https://github.com/NVIDIA/TensorRT-LLM/pull/11677 * [None][refactor] Revisit attention interface for AutoDeploy by @lucaslie in https://github.com/NVIDIA/TensorRT-LLM/pull/11796 * [None][feat] Add a flag in trtllm serve to support overriding kv cache dtype by @cjluo-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11487 * [TRTLLMINF-9][chore] Use checkoutFile in mergeWaiveList to avoid full clone by @dpitman-nvda in https://github.com/NVIDIA/TensorRT-LLM/pull/11794 * [None][chore] Refresh inferenceX configs in recipes by @venkywonka in https://github.com/NVIDIA/TensorRT-LLM/pull/11595 * [TRTLLM-11042][feat] Implement suffix automaton on device for spec and support one model by @cascade812 in https://github.com/NVIDIA/TensorRT-LLM/pull/11434 * [https://nvbugs/5941681][fix] Handle dict type for speculative_config by @ziyixiong-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11828 * [None][feat] Add Kimi-K2.5 text model support (NVFP4) by @lancelly in https://github.com/NVIDIA/TensorRT-LLM/pull/11777 * [None][chore] Bump version to 1.3.0rc7 by @yuanjingx87 in https://github.com/NVIDIA/TensorRT-LLM/pull/11864 * [https://nvbugs/5919026][fix] Fix AttributeError when DSA indexer accesses non-DSA kv_cache_manager by @ziyixiong-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11858 * [TRTLLM-11184][feat] Explicit video encode format support by @JunyiXu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11830 * [None][test] Enable DeepGemm + DeepEPLowLatency MoE test combination by @Tabrizian in https://github.com/NVIDIA/TensorRT-LLM/pull/11876 * [#10009][fix] Fix json_schema response_format to support OpenAI API w… by @JunyiXu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11497 * [https://nvbugs/5927620][fix] Override mMaxAttentionWindow with the actual largest window size by @ziyixiong-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11842 * [None][feat] Support mix quantization between shared experts and routed experts for dsv3 by @dmtri35 in https://github.com/NVIDIA/TensorRT-LLM/pull/11215 * [#11666][fix] Fix inmemory model dir detection by @capyun007 in https://github.com/NVIDIA/TensorRT-LLM/pull/11753 * [None][infra] Waive 3 failed cases for main in post-merge 2566 by @ZhanruiSunCh in https://github.com/NVIDIA/TensorRT-LLM/pull/11881 * [None][doc] Add sparse attention tech blog by @heyuhhh in https://github.com/NVIDIA/TensorRT-LLM/pull/11644 * [TRTLLM-9392][feat] Support MoE output to alltoall's workspace for all the quantization recipe of trtllm-gen. by @bobboli in https://github.com/NVIDIA/TensorRT-LLM/pull/11449 * [TRTLLM-10852][feat] Enhance logprobs functionality to always return prompt token logprobs in prompt logprobs by @stnie in https://github.com/NVIDIA/TensorRT-LLM/pull/11235 * [None][fix] Fix typos, grammar, and formatting in comments and docstrings by @kaiyux in https://github.com/NVIDIA/TensorRT-LLM/pull/11826 * [None][fix] Update check_is_moe into support mlp_layer_types after config.json update by @eagle705 in https://github.com/NVIDIA/TensorRT-LLM/pull/11477 * [https://nvbugs/5946303][fix] Fix incorrect GPU timing in time breakdown under overlap scheduler by @luyiyun1021 in https://github.com/NVIDIA/TensorRT-LLM/pull/11860 * [None][chore] Update autotuner by @jiahanc in https://github.com/NVIDIA/TensorRT-LLM/pull/11859 * [None][chore] Handle failure in auto-assign author workflow by @zhenhuaw-me in https://github.com/NVIDIA/TensorRT-LLM/pull/11906 * [https://nvbugs/5930934][fix] Fix OOM hang with NCCL_SYMMETRIC fallback during long-context inference by @peihu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11870 * [None][fix] Qwen3.5 fix positions ids input for text-only usage by @bmarimuthu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11877 * [None][fix] Refactor nanoV3+superV3 accuracy tests to load example config by @galagam in https://github.com/NVIDIA/TensorRT-LLM/pull/11458 * [None][chore] Deprecate eagle3 2-model by @mikeiovine in https://github.com/NVIDIA/TensorRT-LLM/pull/11761 * [#11819][fix] Disable preload for Llama4 scout by @taylor-yb-lee in https://github.com/NVIDIA/TensorRT-LLM/pull/11873 * [None][chore] Fix format issue in tensorrt_llm/serve/openai_server.py by @chienchunhung in https://github.com/NVIDIA/TensorRT-LLM/pull/11920 * [None][feat] Separate radix search tree implementation by @thorjohnsen in https://github.com/NVIDIA/TensorRT-LLM/pull/10862 * [None][feat] Add support for expert_number<=2048 and K<=32 by @ChristinaZ in https://github.com/NVIDIA/TensorRT-LLM/pull/11510 * [None][infra] Waive 1 failed cases for main in pre-merge 29212 by @ZhanruiSunCh in https://github.com/NVIDIA/TensorRT-LLM/pull/11929 * [None][fix] remove leak check for kimi by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11825 * [https://nvbugs/5907477][chore] unwaive test by @reasonsolo in https://github.com/NVIDIA/TensorRT-LLM/pull/11896 * [TRTLLM-10956][infra] Support build-only mode for GenPostMergeBuilds job by @mzweilz in https://github.com/NVIDIA/TensorRT-LLM/pull/11895 * [#11755][feat] AutoDeploy onboarding agent + Kimi K2.5 AD modeling code by @bmarimuthu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11780 * [None][fix] Prevent RuntimeError from dict mutation during iteration in EXAONE MoE weight mapper by @Bias92 in https://github.com/NVIDIA/TensorRT-LLM/pull/11862 * [TRTLLM-11101][feat] VisualGen benchmarking script by @zhenhuaw-me in https://github.com/NVIDIA/TensorRT-LLM/pull/11651 * [https://nvbugs/5820734][fix] Run extra general warmup to warm up memory pool by @liji-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10340 * [None][fix] Fix nemotron super MTP crash on SM90 by @sunnyqgg in https://github.com/NVIDIA/TensorRT-LLM/pull/11807 * [None][chore] Use cluster service discover in disagg CI tests by @ekou24 in https://github.com/NVIDIA/TensorRT-LLM/pull/11242 * [None][feat] External Drafter One Model by @IzzyPutterman in https://github.com/NVIDIA/TensorRT-LLM/pull/11758 * [None][chore] Update model list by @tcherckez-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/11827 * [#11578][fix] Use string stop/bad words in gRPC proto instead of pre-tokenized TokenSequence by @CatherineSue in https://github.com/NVIDIA/TensorRT-LLM/pull/11888 * [None][feat] Add support for bidirectional sliding window attention mask to fmha_v2 by @djns99 in https://github.com/NVIDIA/TensorRT-LLM/pull/11212 * [TRTLLM-11036][feat] Enable new moe test and clean the legacy moe test in the CI by @xxi-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11817 * [None][infra] Waive 4 failed cases for main in post-merge 2571 by @ZhanruiSunCh in https://github.com/NVIDIA/TensorRT-LLM/pull/11968 * [None][test] Fix deepseek-r1 OOM issue for H100 perf test by @yufeiwu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11948 * [None][fix] Remove incorrect Python import style rule from AGENTS.md by @yuxianq in https://github.com/NVIDIA/TensorRT-LLM/pull/11940 * [https://nvbugs/5896577][fix] fix bug of mistral large3 with eagle by @byshiue in https://github.com/NVIDIA/TensorRT-LLM/pull/11942 * [https://nvbugs/5819048][fix] unwaive test of qwen3-235b eagle3 by @byshiue in https://github.com/NVIDIA/TensorRT-LLM/pull/11969 * [None][feat] Avoid duplicated computation with ADP + Helix CP in GQA by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11891 * [https://nvbugs/5624818][fix] Add unittest for GPT-OSS non-paged_context_fmha by @pengbowang-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11415 * [#10245][feat] AutoDeploy: Support Finegrained FP8 quantization by @bmarimuthu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10897 * [TRTLLM-11284][infra] Move large models test to post-merge by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/11933 * [TRTLLM-11155][infra] Run multi-GPU tests even single-GPU tests are failed when use --disable-fail-fast by @yiqingy0 in https://github.com/NVIDIA/TensorRT-LLM/pull/11740 * [None][fix] Refine tests/unittest/_torch/flashinfer/test_trtllm_flashinfer_symbol_collision.py to reduce jit-compile time by @yihwang-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11890 * [#11422][feat] AutoDeploy: Piecewise cudagraph support Prototype by @nvchenghaoz in https://github.com/NVIDIA/TensorRT-LLM/pull/11515 * [TRTLLM-11189][fix] VisualGen isolated TeaCache Wan fix by @o-stoner in https://github.com/NVIDIA/TensorRT-LLM/pull/11964 * [https://nvbugs/5846166][fix] Update Perf Triage Scripts to Fix gen_only issue by @chenfeiz0326 in https://github.com/NVIDIA/TensorRT-LLM/pull/11802 * [TRTLLM-11057][feat] Add Helix CP support for DSV3.2 by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11507 * [#2912][feat] Support Cohere Command A model by @torotoki in https://github.com/NVIDIA/TensorRT-LLM/pull/11505 * [TRTLLM-11259][perf] Parallel VAE harness and implementation for WAN by @NVShreyas in https://github.com/NVIDIA/TensorRT-LLM/pull/11875 * [#11578][feat] support multimodal image input in gRPC server by @CatherineSue in https://github.com/NVIDIA/TensorRT-LLM/pull/11800 * [TRTLLM-11093][feat] add 5D A2A for fused ulysses by @NVShreyas in https://github.com/NVIDIA/TensorRT-LLM/pull/11787 * [TRTLLM-11189][fix] Fix TeaCache broken caching for FLUX.1 and FLUX.2 by @karljang in https://github.com/NVIDIA/TensorRT-LLM/pull/11868 * [None][refactor] Request management in ScheduledRequests by @Funatiq in https://github.com/NVIDIA/TensorRT-LLM/pull/11784 * [None][perf] Add Triton FP8 blockwise quant kernel and autotuner bucket-skip for visual gen by @chang-l in https://github.com/NVIDIA/TensorRT-LLM/pull/11854 * [TRTLLM-11290][feat] Enable trtllm-serve E2E tests by @JunyiXu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11985 * [None][feat] Optimize by fuse nvfp4_quant to layernorm_gated for mamba2_mixer by @Wanli-Jiang in https://github.com/NVIDIA/TensorRT-LLM/pull/11473 * [None][chore] Autodeploy: add models for sprint by @nvchenghaoz in https://github.com/NVIDIA/TensorRT-LLM/pull/11999 * [None][infra] Update CI allow list 20260305 by @yuanjingx87 in https://github.com/NVIDIA/TensorRT-LLM/pull/11965 * [None][chore] Mass integration of release/1.2 weekly - 6th by @dominicshanshan in https://github.com/NVIDIA/TensorRT-LLM/pull/11934 * [None][fix] Fix Collect Perf Sanity Result's import requests Error by @chenfeiz0326 in https://github.com/NVIDIA/TensorRT-LLM/pull/12002 * [TRTLLM-10956][infra] Skip updating gitlab status for GenPostMergeBuilds by @mzweilz in https://github.com/NVIDIA/TensorRT-LLM/pull/11954 * [None][feat] add ReLU2 NVFP4 fusion for AutoDeploy with tests by @tcherckez-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/11957 * [TRTLLM-11159][feat] Wire KVCacheBlock to UnifiedBlockTree, replacing mPrevBlock/mNextBlocks with lookup-node pointers. by @SimengLiu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11919 * [#11166][infra] AutoDeploy: improve test organization in CI and add overview doc by @lucaslie in https://github.com/NVIDIA/TensorRT-LLM/pull/11291 * [None][chore] Model update 260308 by @tcherckez-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/12011 * [None][infra] Update AutoDeploy CODEOWNERS coverage by @lucaslie in https://github.com/NVIDIA/TensorRT-LLM/pull/12013 * [https://nvbugs/5732958][bug] Fix TestLlama4MinLatency::test_llama_allclose_to_hf failure by @nvpohanh in https://github.com/NVIDIA/TensorRT-LLM/pull/10191 * [None][chore] Unwaive some skip for trtllm moe backend by @leslie-fang25 in https://github.com/NVIDIA/TensorRT-LLM/pull/11975 * [TRTLLM-11134][feat] export VisualGen API and update doc by @zhenhuaw-me in https://github.com/NVIDIA/TensorRT-LLM/pull/11911 * [https://nvbugs/5823783][test] add qa test case for trust-remote-code on multinode failure by @crazydemo in https://github.com/NVIDIA/TensorRT-LLM/pull/11905 * [None][feat] Use max_gpu_total_bytes to control v2's capacity by @jiaganc in https://github.com/NVIDIA/TensorRT-LLM/pull/11907 * [TRTLLM-11342][fix] Fix FLUX.1 TeaCache polynomial coefficients and default t… by @karljang in https://github.com/NVIDIA/TensorRT-LLM/pull/12007 * [None][fix] Use try/except fallback for Pydantic ValidatorIterator in chat message parsing by @Wanli-Jiang in https://github.com/NVIDIA/TensorRT-LLM/pull/11903 * [None][infra] Unwaive 2 cases on rtx-pro-6000d by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/12003 * [TRTLLM-11276][chore] Expose use_python_scheduler in SchedulerConfig and add UTs/ITs for python scheduler by @lancelly in https://github.com/NVIDIA/TensorRT-LLM/pull/11884 * [None][infra] Waive 7 failed cases for main in post-merge 2576 by @ZhanruiSunCh in https://github.com/NVIDIA/TensorRT-LLM/pull/12014 * [https://nvbugs/5948878][fix] Implement workaround for ClientPayloadError by @yingguo-trt in https://github.com/NVIDIA/TensorRT-LLM/pull/12018 * [TRTLLM-10407][feat] Integrate CuTE DSL top-k kernel for Blackwell by @limin2021 in https://github.com/NVIDIA/TensorRT-LLM/pull/11900 * [TRTLLM-11148][perf] _prepare_inputs host time optimization by @hyukn in https://github.com/NVIDIA/TensorRT-LLM/pull/11704 * [None][test] Fix model_name starcoder_15b is not in allowed_models issue by @yufeiwu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11981 * [None][infra] Waive 5 failed cases for main in post-merge 2578 by @ZhanruiSunCh in https://github.com/NVIDIA/TensorRT-LLM/pull/12023 * [None][chore] AutoDeploy: re-enable nvfp4 superv3 accuracy test by @galagam in https://github.com/NVIDIA/TensorRT-LLM/pull/11945 * [None][chore] Remove visual_gen benchmark test from YAML by @zhenhuaw-me in https://github.com/NVIDIA/TensorRT-LLM/pull/12027 * [None][fix] Fix the model list as it had a dup model by @tcherckez-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/12029 * [https://nvbugs/5863806][fix] Fix Python string truthiness bug in FMHA cubin selection by @luyiyun1021 in https://github.com/NVIDIA/TensorRT-LLM/pull/11909 * [None][feat] Upgrade xgrammar from 0.1.25 to 0.1.32 by @sunnyqgg in https://github.com/NVIDIA/TensorRT-LLM/pull/12016 * [https://nvbugs/5924144][test] unwaive cpp/test_unit_tests.py::test_unit_tests[kernels-80] by @Funatiq in https://github.com/NVIDIA/TensorRT-LLM/pull/11902 * [None][chore] limit tileiras to CUDA13.1 by @tburt-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/12042 * [None][feat] Add silu to trtllm-gen MoE by @IwakuraRein in https://github.com/NVIDIA/TensorRT-LLM/pull/11663 * [TRTLLM-11045][feat] Integrate SA with EAGLE3 and PARD by @cascade812 in https://github.com/NVIDIA/TensorRT-LLM/pull/11878 * [None][chore] waive test_visual_gen_quickstart by @tburt-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/12043 * [None][feat] NIXL support for hybrid model cache transfer by @NVShreyas in https://github.com/NVIDIA/TensorRT-LLM/pull/11608 ## New Contributors * @zyang-Modular made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/11689 * @slin1237 made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/11711 * @davidmlw made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/11493 * @marinayanov made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/11530 * @lazykyama made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/9453 * @capyun007 made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/11753 * @Bias92 made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/11862 * @ekou24 made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/11242 * @o-stoner made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/11964 * @torotoki made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/11505 * @IwakuraRein made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/11663 **Full Changelog**: https://github.com/NVIDIA/TensorRT-LLM/compare/v1.3.0rc6...v1.3.0rc7

## What's Changed * [https://nvbugs/5850094][fix] Fix MoE cost estimation for auto multi-stream scheduling by @yizhang-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11160 * [None][feat] update TRT-LLM Gen DS FP8 MoE cubins and optimize finalize kernel by @nekorobov in https://github.com/NVIDIA/TensorRT-LLM/pull/11104 * [None][chore] Bump version to 1.2.0rc6.post3 by @yiqingy0 in https://github.com/NVIDIA/TensorRT-LLM/pull/11224 * [None][fix] Fallback to NCCL instead of NCCL symmetric by @Tabrizian in https://github.com/NVIDIA/TensorRT-LLM/pull/11174 * [None][feat] fuse shared to sparse experts in TRT-LLM Gen MoE by @nekorobov in https://github.com/NVIDIA/TensorRT-LLM/pull/11143 **Full Changelog**: https://github.com/NVIDIA/TensorRT-LLM/compare/v1.2.0rc6.post2...v1.2.0rc6.post3

## What's Changed * [None][fix] fix TinyGemm accuracy issue. cherry-pick https://github.com/NVIDIA/TensorRT-LLM/pull/10619 and https://github.com/NVIDIA/TensorRT-LLM/pull/10873 by @bo-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10990 * [None][chore] Bump version to 1.2.0rc2.post2 by @chzblych in https://github.com/NVIDIA/TensorRT-LLM/pull/11012 * [None][chore] Upgrade starlette and FastAPI (#9319) by @chzblych in https://github.com/NVIDIA/TensorRT-LLM/pull/11027 * [None][fix] fix accuracy issue(cherry-pick https://github.com/NVIDIA/TensorRT-LLM/pull/11157 and https://github.com/NVIDIA/TensorRT-LLM/pull/9530) by @bo-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/11222 **Full Changelog**: https://github.com/NVIDIA/TensorRT-LLM/compare/v1.2.0rc2.post1...v1.2.0rc2.post2

## Highlights - Model Support - Add export patch for GraniteMoe MoE models to enable torch.export compatibility (#10169) - Eagle: qwen2 capture hidden states (#10091) - Add pp support for DeepSeek-v3.2 (#10449) - Pass lora_params through Qwen2/3 model forward (#10174) - Fix export for microsoft/Phi-3-medium-128k-instruct (#10455) - Mistral large 3 few code refine (#10405) - EPD for Qwen3 VL (#10470) - Remove some model support; add device constraint (#10563) - Enable AttentionDP on Qwen3-VL and fix test (#10435) - API - Add stability tags for serve subcommand (#10012) - Feature - Better align MLA chunking with indexer chunking when chunked prefill enabled for DSV32 (#10552) - Sm100 weight-only kernel (#10190) - AutoTuner Cache: Support cache file lock and merge all ranks into one (#10336) - Apply AutoTuner to AllReduce Op for strategy tuning (#8531) - Add transferAgent binding (step 1) (#10113) - Add the eos tokens in generation config to stop words in the sampler (#10389) - Apply fusion for W4AFP8_AWQ MoE (#9838) - Further reduce tuning time for cuteDSL nvFP4 dense gemm (#10339) - Run sample_async on extra stream (#10215) - Optimize qk rope/nope concat for DSA (#10571) - Fix - Fix bug of Mistral-Small-3.1-24B-Instruct-2503 (#10394) - Use 0 port as arbitrary port when disagg service discovery is enabled (#10383) - Fix buffer reuse for CUDA graph attention metadata (#10393) - Force release torch memory when LLM is destroyed (#10314) - Swap TP-CP grouping order (#10350) - TRTLLM MoE maps to lower tuning buckets when ep>1 (#9998) - Fix draft token tree chain crash and depth=1 corner case (#10386, #10385) - Fixed recursive node traversals (#10379) - Fix undefined tokens_per_block (#10438) - Skip spec dec for non-last rank (#10445) - Setup dist before using autotuner (#10491) - Fix broken cast (#9975) - Fix sm120 speculation (#10049) - Fix mamba_cache_manager when enabling cuda_graph_padding and let test cover this case (#9873) - Choose register model config over root config for VLM (#10553) - Documentation - Update SWA + spec dec support matrix (#10421) - Add --config preference over --extra_llm_api_options in CODING_GUIDELINES.md (#10426) - Adding parallelism types in feature combination matrix (#9849) - Update GPTOSS Doc (#10536) - Blog: Optimizing DeepSeek-V3.2 on NVIDIA Blackwell GPUs (#10565) - Update Qwen3-Next doc by adding known issues section (#10582) - Test & Infra - Add tests for DeepSeek v3.2 (#10561) - Add accuracy tests for super-v3 with multiple-gpus (#10234) - Layer-wise benchmarks: support TEP balance, polish slurm scripts (#10237) - Add disag-serving kimi k2 thinking tests (#10357) - Partition test_llm_pytorch.py for parallel execution (#10400) - Only Use Throughput Metrics to Check Regression (#10404) - Add vswa test cases coverage (#10146) - Use random port in container port section (#10432) - Remove redundant retries while binding to arbitrary port (#10452) - Add qwen3-4b accuracy test case (#10382) - Update kimi-k2-1k1k dataset (#10473) - Fix concurrency list in Wide-EP perf tests (#10529) - Restrict max_num_tokens in disagg mtp config (#10442) - Add kimi_k2 single node perf test (#10436) - Add MMMU test for mistral small (#10530) - Workaround OCI-NRT slowdown issue (#10587) ## What's Changed * [#8391][chore] added deepseek_r1_distill_qwen_32b AutoDeploy perf test to L0 by @MrGeva in https://github.com/NVIDIA/TensorRT-LLM/pull/10377 * [https://nvbugs/5670469][fix] Filter 0s and choose min of kv_head for Nemotron model by @farazkh80 in https://github.com/NVIDIA/TensorRT-LLM/pull/10206 * [https://nvbugs/5772363][fix] fix bug of Mistral-Small-3.1-24B-Instruct-2503 by @byshiue in https://github.com/NVIDIA/TensorRT-LLM/pull/10394 * [https://nvbugs/5649010][fix] use 0 port as arbitrary port when disagg service discovery is enabled by @reasonsolo in https://github.com/NVIDIA/TensorRT-LLM/pull/10383 * [TRTLLM-10065][feat] Add accuracy tests for super-v3 with multiple-gpus by @Wanli-Jiang in https://github.com/NVIDIA/TensorRT-LLM/pull/10234 * [https://nvbugs/5779534][fix] fix buffer reuse for CUDA graph attention metadata by @lfr-0531 in https://github.com/NVIDIA/TensorRT-LLM/pull/10393 * [None][feat] sm100 weight-only kernel by @Njuapp in https://github.com/NVIDIA/TensorRT-LLM/pull/10190 * [https://nvbugs/5701425][chore] Unwaive tests. by @yuxianq in https://github.com/NVIDIA/TensorRT-LLM/pull/10269 * [None][feat] Layer-wise benchmarks: support TEP balance, polish slurm scripts by @yuantailing in https://github.com/NVIDIA/TensorRT-LLM/pull/10237 * [None][infra] Waive failed cases in post-merge on 1/5 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/10399 * [TRTLLM-10185][feat] AutoTuner Cache: Support cache file lock and merge all ranks into one by @hyukn in https://github.com/NVIDIA/TensorRT-LLM/pull/10336 * [TRTLLM-8242][feat] Add stability tags for serve subcommand by @LinPoly in https://github.com/NVIDIA/TensorRT-LLM/pull/10012 * [https://nvbugs/5752521][fix] Unwaive test_trtllm_flashinfer_symbol_collision.py by @yihwang-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10227 * [None][infra] Waive failed cases again on 1/5 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/10403 * [https://nvbugs/5715568][fix] Force to release torch memory when LLM is destroyed by @HuiGao-NV in https://github.com/NVIDIA/TensorRT-LLM/pull/10314 * [TRTLLM-8821][feat] Apply AutoTuner to AllReduce Op for strategy tuning. by @hyukn in https://github.com/NVIDIA/TensorRT-LLM/pull/8531 * [None][feat] update deepgemm to the DeepGEMM/nv_dev branch by @lfr-0531 in https://github.com/NVIDIA/TensorRT-LLM/pull/9898 * [TRTLLM-9381][test] add disag-serving kimi k2 thinking tests by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10357 * [#10374][fix] fixed race condition in AutoDeploy's mp tests port acquisition by @MrGeva in https://github.com/NVIDIA/TensorRT-LLM/pull/10366 * [TRTLLM-9465][fix] Swap TP-CP grouping order by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10350 * [None][perf] TRTLLM MoE maps to lower tuning buckets when ep>1 by @rosenrodt in https://github.com/NVIDIA/TensorRT-LLM/pull/9998 * [TRTLLM-10053][feat] AutoDeploy: Add Super v3 config file, improve test runtime by @galagam in https://github.com/NVIDIA/TensorRT-LLM/pull/10397 * [https://nvbugs/5772521][fix] Fix draft token tree chain crash by @mikeiovine in https://github.com/NVIDIA/TensorRT-LLM/pull/10386 * [https://nvbugs/5772414][fix] Fix draft token tree depth=1 corner case by @mikeiovine in https://github.com/NVIDIA/TensorRT-LLM/pull/10385 * [TRTLLM-9767][feat] Fixed recursive node traversals by @greg-kwasniewski1 in https://github.com/NVIDIA/TensorRT-LLM/pull/10379 * [TRTLLM-9551][infra] Partition test_llm_pytorch.py for parallel execution by @Superjomn in https://github.com/NVIDIA/TensorRT-LLM/pull/10400 * [https://nvbugs/5695984][fix] Unwaive llama3 eagle test by @mikeiovine in https://github.com/NVIDIA/TensorRT-LLM/pull/10092 * [https://nvbugs/5745152][fix] Unwaive gpt oss spec decode test by @mikeiovine in https://github.com/NVIDIA/TensorRT-LLM/pull/10370 * [#10170][fix] Add export patch for GraniteMoe MoE models to enable torch.export compatibility by @karthikvetrivel in https://github.com/NVIDIA/TensorRT-LLM/pull/10169 * [https://nvbugs/5777044][chore] Remove solved bugs from waives.txt by @SimengLiu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10422 * [None][feat] precompiled installation from local src dir by @lucaslie in https://github.com/NVIDIA/TensorRT-LLM/pull/10419 * [TRTLLM-9527][feat] Add transferAgent binding (step 1) by @chuangz0 in https://github.com/NVIDIA/TensorRT-LLM/pull/10113 * [None][fix] Only Use Throughput Metrics to Check Regression by @chenfeiz0326 in https://github.com/NVIDIA/TensorRT-LLM/pull/10404 * [None][feat] add the eos tokens in generation config to stop words in the sampler by @JadoTu in https://github.com/NVIDIA/TensorRT-LLM/pull/10389 * [None][chore] Update SWA + spec dec support matrix by @mikeiovine in https://github.com/NVIDIA/TensorRT-LLM/pull/10421 * [None][feat] CuteDSL MOE FC1 Enhancement by @liyuhannnnn in https://github.com/NVIDIA/TensorRT-LLM/pull/10088 * [https://nvbugs/5726962][feat] Apply fusion for W4AFP8_AWQ MoE by @yumin066 in https://github.com/NVIDIA/TensorRT-LLM/pull/9838 * [#2511][fix] eagle: qwen2 capture hidden states by @XiaoXuan42 in https://github.com/NVIDIA/TensorRT-LLM/pull/10091 * [None][docs] Add `--config` preference over `--extra_llm_api_options` in CODING_GUIDELINES.md by @venkywonka in https://github.com/NVIDIA/TensorRT-LLM/pull/10426 * [#8460][feat] Revive and simplify Model Explorer visualization integration by @karthikvetrivel in https://github.com/NVIDIA/TensorRT-LLM/pull/10150 * [None][chore] unwaive qwen3 30b test by @kris1025 in https://github.com/NVIDIA/TensorRT-LLM/pull/10115 * [TRTLLM-8638][fix] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10384 * [None][test] update test case constraint by @crazydemo in https://github.com/NVIDIA/TensorRT-LLM/pull/10381 * [https://nvbugs/5769926] [fix] Add no container mount home WAR by @kaiyux in https://github.com/NVIDIA/TensorRT-LLM/pull/10431 * [TRTLLM-8638][fix] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10367 * [TRTLLM-9622][infra] Enable DGX_B300 multi-gpu testing in pre-merge pipeline by @yiqingy0 in https://github.com/NVIDIA/TensorRT-LLM/pull/9699 * [TRTLLM-9896][test] add vswa test cases coverage by @crazydemo in https://github.com/NVIDIA/TensorRT-LLM/pull/10146 * [None] [fix] Fix undefined tokens_per_block by @kaiyux in https://github.com/NVIDIA/TensorRT-LLM/pull/10438 * [https://nvbugs/5772361][ci] Unwaive tests that have been fixed by @2ez4bz in https://github.com/NVIDIA/TensorRT-LLM/pull/10424 * [https://nvbugs/5785206][infra] Waive TestQwen3_30B_A3B::test_fp8[latency-torch_compile=False]. by @bobboli in https://github.com/NVIDIA/TensorRT-LLM/pull/10441 * [None][infra] Waive failed cases on 1/6 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/10440 * [TRTLLM-8638][fix] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10427 * [https://nvbugs/5760726][fix] Use random port in container port section by @JunyiXu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10432 * [None][chore] remove redundant retries while binding to arbitrary port by @reasonsolo in https://github.com/NVIDIA/TensorRT-LLM/pull/10452 * [https://nvbugs/5748600][ci] Unwaive disagg guided decoding test by @syuoni in https://github.com/NVIDIA/TensorRT-LLM/pull/10409 * [https://nvbugs/5749988][fix] Remove redundant qwen3 spec dec test by @mikeiovine in https://github.com/NVIDIA/TensorRT-LLM/pull/10387 * [None][feat] precompiled installation from local src dir with fnmatch only by @lucaslie in https://github.com/NVIDIA/TensorRT-LLM/pull/10430 * [https://nvbugs/5732942][fix] AutoDeploy: handle transformers 4.57.1 upgrade fixes by @lucaslie in https://github.com/NVIDIA/TensorRT-LLM/pull/10466 * [None] [feat] Add test script and raster M for gather fc1 kernel by @zongfeijing in https://github.com/NVIDIA/TensorRT-LLM/pull/10429 * [https://nvbugs/5721907][fix] AutoDeploy: improve numerical stability of flashinfer attention test by @lucaslie in https://github.com/NVIDIA/TensorRT-LLM/pull/10467 * [https://nvbugs/5698434][test] add qwen3-4b accuracy test case by @crazydemo in https://github.com/NVIDIA/TensorRT-LLM/pull/10382 * [None][chore] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10365 * [https://nvbugs/5767223][feat] add pp support for DeepSeek-v3.2 by @lfr-0531 in https://github.com/NVIDIA/TensorRT-LLM/pull/10449 * [TRTLLM-8638][fix] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10457 * [https://nvbugs/5726086][fix] update kimi-k2-1k1k dataset by @yingguo-trt in https://github.com/NVIDIA/TensorRT-LLM/pull/10473 * [#4745][fix] Pass lora_params through Qwen2/3 model forward by @karljang in https://github.com/NVIDIA/TensorRT-LLM/pull/10174 * [TRTLLM-8638][fix] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10474 * [None][bug] fix export for microsoft/Phi-3-medium-128k-instruct by @tcherckez-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/10455 * [None][test] Remove most TRT-backend test cases in llm_perf_nim.yml by @yufeiwu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10487 * [None][doc] Adding parallelism types in feature combination matrix by @pcastonguay in https://github.com/NVIDIA/TensorRT-LLM/pull/9849 * [https://nvbugs/5781589][fix] Skip spec dec for non-last rank by @ziyixiong-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10445 * [https://nvbugs/5761665][fix] AutoDeploy: using Dim.DYNAMIC for robust dynamic shape by @lucaslie in https://github.com/NVIDIA/TensorRT-LLM/pull/10511 * [https://nvbugs/5707392][fix] unwaive test_fused_moe_fp8_blockwise_wide_ep[NotEnabled] by @xxi-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10428 * [TRTLLM-9661][chore] Further reduce tuning time for cuteDSL nvFP4 dense gemm. by @hyukn in https://github.com/NVIDIA/TensorRT-LLM/pull/10339 * [https://nvbugs/5784543][fix] Setup dist before using autotuner. by @yuxianq in https://github.com/NVIDIA/TensorRT-LLM/pull/10491 * [None][chore] Support multiple job submission at the same time by @yingguo-trt in https://github.com/NVIDIA/TensorRT-LLM/pull/10492 * [https://nvbugs/5747878][fix] unwaive llama4 scout tests by @lucaslie in https://github.com/NVIDIA/TensorRT-LLM/pull/10468 * [https://nvbugs/5775402][fix] Fix concurrency list in Wide-EP perf tests by @Barry-Delaney in https://github.com/NVIDIA/TensorRT-LLM/pull/10529 * [None][chore] Enable seg fault cases since one race condition is fixed by @HuiGao-NV in https://github.com/NVIDIA/TensorRT-LLM/pull/10398 * [None][doc] Update GPTOSS Doc by @dongfengy in https://github.com/NVIDIA/TensorRT-LLM/pull/10536 * [TRTLLM-9642][infra] Increase pytest verbosity for failed tests by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/9657 * [None][chore] Bump version to 1.2.0rc8 by @yiqingy0 in https://github.com/NVIDIA/TensorRT-LLM/pull/10542 * [None][fix] Mistral large 3 few code refine by @byshiue in https://github.com/NVIDIA/TensorRT-LLM/pull/10405 * [#10417][fix] AutoDepoloy - Reverted to direct computation of minusA by @MrGeva in https://github.com/NVIDIA/TensorRT-LLM/pull/10509 * [None][feat] EPD for Qwen3 VL by @2ez4bz in https://github.com/NVIDIA/TensorRT-LLM/pull/10470 * [TRTLLM-9522][fix] broken cast by @ixlmar in https://github.com/NVIDIA/TensorRT-LLM/pull/9975 * [#10513][fix] AutoDeploy: removed self.mlp_type leftovers from last moe refactor by @MrGeva in https://github.com/NVIDIA/TensorRT-LLM/pull/10512 * [https://nvbugs/5740075][fix] Fix sm120 speculation by @mikeiovine in https://github.com/NVIDIA/TensorRT-LLM/pull/10049 * [None][chore] Waive tests blocking premerge 01/08 by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10555 * [None][fix] revert https://github.com/NVIDIA/TensorRT-LLM/pull/10445. by @yuxianq in https://github.com/NVIDIA/TensorRT-LLM/pull/10547 * [None][test] restrict max_num_tokens in disagg mtp config by @ruodil in https://github.com/NVIDIA/TensorRT-LLM/pull/10442 * [None][chore] Add failed cases into waives.txt by @jieli-matrix in https://github.com/NVIDIA/TensorRT-LLM/pull/10541 * [None][fix] Setup dist for AutoTuner in Layerwise benchmarking. by @hyukn in https://github.com/NVIDIA/TensorRT-LLM/pull/10534 * [TRTLLM-9676][fix] Fix mamba_cache_manager when enabling cuda_graph_padding and let test cover this case by @JadoTu in https://github.com/NVIDIA/TensorRT-LLM/pull/9873 * [https://nvbugs/5785206][infra] unwaive the accuracy/test_llm_api_pytorch.py::TestQwen3_30B_A3B by @byshiue in https://github.com/NVIDIA/TensorRT-LLM/pull/10560 * [https://nvbugs/5787453][fix] Better align MLA chunking with indexer chunking when chunked prefill enabled for DSV32 by @chang-l in https://github.com/NVIDIA/TensorRT-LLM/pull/10552 * [https://nvbugs/5622938][feat] Run sample_async on extra stream. by @yuxianq in https://github.com/NVIDIA/TensorRT-LLM/pull/10215 * [None][doc] blog: Optimizing DeepSeek-V3.2 on NVIDIA Blackwell GPUs by @lfr-0531 in https://github.com/NVIDIA/TensorRT-LLM/pull/10565 * [TRTLLM-9932][test] add kimi_k2 single node perf test by @ruodil in https://github.com/NVIDIA/TensorRT-LLM/pull/10436 * [None][chore] remove some model support; add device constraint by @jieli-matrix in https://github.com/NVIDIA/TensorRT-LLM/pull/10563 * [https://nvbugs/5756008][fix] unwaive test by @Tabrizian in https://github.com/NVIDIA/TensorRT-LLM/pull/10523 * [TRTLLM-10309] [feat] Optimize qk rope/nope concat for DSA by @kaiyux in https://github.com/NVIDIA/TensorRT-LLM/pull/10571 * [None][fix] Enable AttentionDP on Qwen3-VL and fix test by @yechank-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/10435 * [None][feat] Add support for DeepSeek v3.2 tests by @yingguo-trt in https://github.com/NVIDIA/TensorRT-LLM/pull/10561 * [https://nvbugs/5752687][fix] Choose register model config over root config for VLM by @farazkh80 in https://github.com/NVIDIA/TensorRT-LLM/pull/10553 * [https://nvbugs/5628848][fix] Fix nanobind stub generation by @Linda-Stadter in https://github.com/NVIDIA/TensorRT-LLM/pull/10516 * [https://nvbugs/5548861][fix] AutoDeploy: Fix the test by @nvchenghaoz in https://github.com/NVIDIA/TensorRT-LLM/pull/10521 * [https://nvbugs/5669097][tests] Add MMMU test for mistral small by @2ez4bz in https://github.com/NVIDIA/TensorRT-LLM/pull/10530 * [None][chore] Update AutoDeploy model list by @tcherckez-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/10505 * [None][chore] Fix Gitlab CI termination issues by @fredricz-20070104 in https://github.com/NVIDIA/TensorRT-LLM/pull/10576 * [None][chore] waive test case by @HuiGao-NV in https://github.com/NVIDIA/TensorRT-LLM/pull/10581 * [None][doc] Update Qwen3-Next doc by adding known issues section by @nv-guomingz in https://github.com/NVIDIA/TensorRT-LLM/pull/10582 * [None][ci] Workaround OCI-NRT slowdown issue by @chzblych in https://github.com/NVIDIA/TensorRT-LLM/pull/10587 ## New Contributors * @karthikvetrivel made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/10169 * @XiaoXuan42 made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/10091 **Full Changelog**: https://github.com/NVIDIA/TensorRT-LLM/compare/v1.2.0rc7...v1.2.0rc8

## Highlights - Model Support - Optimize DeepSeek FP8 activation kernel for TRT-LLM Gen MoE (#9175) - Disable fp8 deep GEMM for EXAONE-4.0-32B-FP8 (#8429) - Fix output unpack issues for Llama3/4 NVFP4 models (#8679) - API - Support out-of-tree models in `trtllm-serve` (#9269) - Feature - Make sharing of `activation_type` across SW layers more robust (#9238) - Create communication related classes (#8968) - Integrate CuteDSL NVFP4 grouped GEMM with SwiGLU fusion and finalized fusion (#9288) - Add PostNorm and multilayer options for Eagle models (#9233) - Fix - Use fp32 for indexer `weight_proj` GEMM (#9243) - Fix multimodal `InputProcessor` dummy builder (#8916) - Set correct `lm_head_tp_size_upper_bound` (#9300) - Move `torch.cuda.Stream` out of critical `torch` computation region (#8494) - Fix `trtllm-llmapi-launch` port conflict (#8582) - Rework `DisaggPPTerminationHandler` to fix hang issue (#8519) - Overwrite only if `default_max_tokens` is legal (#8538) - Fix block range index (#8470) - Restrict FP8 blockscale MoE case to valid configurations (#8583) - Fix `L0_backend_trtllm` behavior (#9282) - Improve beam search request validation (#9228) - Avoid incorrectly filling tensors with 0 (#9296) - Fallback to greedy sampling in two-model overlap scheduler to improve stability (#9321) - Documentation - Revise the description of `enable_autotuner` (#9320) - Document the process for C++ dependencies (#9016) - Benchmark - Set `max_batch_size=1` to stabilize accuracy test results (#8609) - Test & Infra - Use greedy decoding in `test_openai_compatible_json_schema` (#9305) - Enable checking duplicate items in `waives.txt` in pre-commit (#9265) - Fix test case where chunked attention is not supported on `sm_120` (#9260) - Add `NCCL_DEBUG=INFO` flag to collect more information on CI failures (#8440) - Remove multimodal test cases using TRT backend (#8611) - Clean cache for easily hanging test cases (#8619) - Enable relaxed acceptance test on Blackwell (#8709) - Update linter rules for mass integration (#8918) - Upgrade `starlette` and `FastAPI` dependencies (#9319) - Update `goggles_action` repository (#9240) - Move third-party components to their own list file (#8986) - Add fallback when fetching wheel from build stage fails (#9290) - Add `--waives-file` flag in rerun `pytest` command (#8971) - Add periodic JUnit XML path in `conftest` (#9337) - Consume `SlurmCluster` `sshPort` for clusters with custom SSH port (#9313) - Add one-model and overlap-scheduling to Eagle tests for GPTOSS (#9312) ## What's Changed * [#9316][feat] AutoDeploy: Add the accuracy test for Nemotron MOE models by @nvchenghaoz in https://github.com/NVIDIA/TensorRT-LLM/pull/9317 * [#9096][feature] Auto Deploy: configurable fused MoE backend by @nzmora-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/9194 * [None][fix] Use fp32 for indexer weight_proj GEMM by @chang-l in https://github.com/NVIDIA/TensorRT-LLM/pull/9243 * [None][fix] Multimodal InputProcessor dummy builder fix by @yechank-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/8916 * [None][ci] waive test_disagg_server_restart by @QiJune in https://github.com/NVIDIA/TensorRT-LLM/pull/9326 * [None][chore] Revise the description of enable_autotuner. by @hyukn in https://github.com/NVIDIA/TensorRT-LLM/pull/9320 * [TRTLLM-9295][fix] use greedy decoding in test_openai_compatible_json_schema by @ixlmar in https://github.com/NVIDIA/TensorRT-LLM/pull/9305 * [TRTLLM-9164][infra] Enable checking duplicate items in waives.txt in pre-commit by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/9265 * [#9236][feature] Make sharing of activation_type across SW layers more robust by @nzmora-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/9238 * [https://nvbugs/5667687][fix] Set correct lm_head_tp_size_upper_bound by @lancelly in https://github.com/NVIDIA/TensorRT-LLM/pull/9300 * [https://nvbugs/5667454][test] Fix Test Case as Chunked Attention not Supported on sm_120 by @yufeiwu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/9260 * [None][chore] Weekly mass integration of release/1.1 by @mikeiovine in https://github.com/NVIDIA/TensorRT-LLM/pull/8918 * [None][chore] Upgrade starlette and FastAPI by @tburt-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/9319 * [None][infra] Update goggles_action repository by @karljang in https://github.com/NVIDIA/TensorRT-LLM/pull/9240 * [TRTLLM-9197][infra] Move thirdparty stuff to it's own listfile by @cheshirekow in https://github.com/NVIDIA/TensorRT-LLM/pull/8986 * [TRI-332] [fix] Fix L0_backend_trtllm by @yinggeh in https://github.com/NVIDIA/TensorRT-LLM/pull/9282 * [None][ci] waive test_llm_context_only_timed_out_kv_cache_exhausted by @QiJune in https://github.com/NVIDIA/TensorRT-LLM/pull/9351 * [None][infra] Add fallback when get wheel from build stage is fail by @ZhanruiSunCh in https://github.com/NVIDIA/TensorRT-LLM/pull/9290 * [TRTLLM-9183][infra] Add --waives-file in rerun pytest command by @yiqingy0 in https://github.com/NVIDIA/TensorRT-LLM/pull/8971 * [TRTLLM-8957][feat] create communication related classes by @xxi-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/8968 * [None][chore] Add periodic junit xml path in conftest by @crazydemo in https://github.com/NVIDIA/TensorRT-LLM/pull/9337 * [None][ci] waive a test case of test_ad_build_small_multi.py by @QiJune in https://github.com/NVIDIA/TensorRT-LLM/pull/9355 * [None][infra] Waive failed cases in main post-merge on 11/21 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/9360 * [None][chore] Bump version to 1.2.0rc4 by @yiqingy0 in https://github.com/NVIDIA/TensorRT-LLM/pull/9363 * [TRTLLM-8650][fix] beam search request validation (#8433) by @ixlmar in https://github.com/NVIDIA/TensorRT-LLM/pull/9228 * [TRTLLM-9191][feat] support out-of-tree models in trtllm-serve by @ixlmar in https://github.com/NVIDIA/TensorRT-LLM/pull/9269 * [https://nvbugs/5629833][fix] Don't fill tensors by @HuiGao-NV in https://github.com/NVIDIA/TensorRT-LLM/pull/9296 * [None][feat] TRT-LLM Gen MoE optimize DeepSeek Fp8 activation kernel by @nekorobov in https://github.com/NVIDIA/TensorRT-LLM/pull/9175 * [https://nvbugs/5590408][fix] Fallback to greedy sampling in two-model overlap scheduler by @ziyixiong-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/9321 * [TRTLLM-9208][infra] Document the process for C++ deps by @cheshirekow in https://github.com/NVIDIA/TensorRT-LLM/pull/9016 * [TRTLLM-9370][feat] Integration of CuteDSL NVFP4 grouped GEMM (Part 2: SwiGLU Fusion and Finalize Fusion) by @syuoni in https://github.com/NVIDIA/TensorRT-LLM/pull/9288 * [None][feat] Eagle: PostNorm and multilayer options by @IzzyPutterman in https://github.com/NVIDIA/TensorRT-LLM/pull/9233 * [TRTLLM-9082][feat] AutoDeploy: Move the moe Align kernel to AOT by @nvchenghaoz in https://github.com/NVIDIA/TensorRT-LLM/pull/9106 * [#9388][fix] AutoDeploy: Fix cutlass BF16 MoE kernel invocation by @nzmora-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/9339 * [TRTINFRA-7326][infra] - Consume SlurmCluster sshPort for clusters with custom SSH port by @mlefeb01 in https://github.com/NVIDIA/TensorRT-LLM/pull/9313 * [None][test] Add one-model and overlap-scheduling to eagle tests for GPTOSS by @dongfengy in https://github.com/NVIDIA/TensorRT-LLM/pull/9312 **Full Changelog**: https://github.com/NVIDIA/TensorRT-LLM/compare/v1.2.0rc3...v1.2.0rc4

## Security Vulnerabilities ### GnuPG Vulnerability A security vulnerability has been identified in GnuPG versions prior to 2.4.9, which is present in the Ubuntu 24.04 LTS utilized by the TensorRT LLM base image. For details regarding this vulnerability, please refer to the official Ubuntu advisory: [CVE-2025-68973](https://ubuntu.com/security/CVE-2025-68973). An official patched package for the Ubuntu system is currently pending. The fix will be included in the next release once the updated package is published and incorporated. To mitigate potential risks immediately, users are advised to manually upgrade GnuPG to version 2.4.9 or later. ### Hugging Face Transformers Vulnerabilities Several security vulnerabilities have been disclosed regarding the Hugging Face Transformers library used in TensorRT LLM. As these issues originate from an upstream dependency, remediation is dependent on the release of a patch by the Hugging Face team. We are actively monitoring the situation and will update TensorRT LLM to include the necessary fixes once a stable release of the Transformers library addressing these vulnerabilities becomes available. Affected CVEs: [CVE-2025-14920](https://www.cve.org/CVERecord?id=CVE-2025-14920), [CVE-2025-14921](https://www.cve.org/CVERecord?id=CVE-2025-14921), [CVE-2025-14924](https://www.cve.org/CVERecord?id=CVE-2025-14924), [CVE-2025-14927](https://www.cve.org/CVERecord?id=CVE-2025-14927), [CVE-2025-14928](https://www.cve.org/CVERecord?id=CVE-2025-14928), [CVE-2025-14929](https://www.cve.org/CVERecord?id=CVE-2025-14929), [CVE-2025-14930](https://www.cve.org/CVERecord?id=CVE-2025-14930) ## What's Changed * [None][chore] Bump version to 1.2.0rc2.post1 by @yiqingy0 in https://github.com/NVIDIA/TensorRT-LLM/pull/10286 * [TRTLLM-9752][fix] disable PDL for quant kernels by @bo-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10288 **Full Changelog**: https://github.com/NVIDIA/TensorRT-LLM/compare/v1.2.0rc2...v1.2.0rc2.post1

## Security Vulnerabilities ### GnuPG Vulnerability A security vulnerability has been identified in GnuPG versions prior to 2.4.9, which is present in the Ubuntu 24.04 LTS utilized by the TensorRT LLM base image. For details regarding this vulnerability, please refer to the official Ubuntu advisory: [CVE-2025-68973](https://ubuntu.com/security/CVE-2025-68973). An official patched package for the Ubuntu system is currently pending. The fix will be included in the next release once the updated package is published and incorporated. To mitigate potential risks immediately, users are advised to manually upgrade GnuPG to version 2.4.9 or later. ### Hugging Face Transformers Vulnerabilities Several security vulnerabilities have been disclosed regarding the Hugging Face Transformers library used in TensorRT LLM. As these issues originate from an upstream dependency, remediation is dependent on the release of a patch by the Hugging Face team. We are actively monitoring the situation and will update TensorRT LLM to include the necessary fixes once a stable release of the Transformers library addressing these vulnerabilities becomes available. Affected CVEs: [CVE-2025-14920](https://www.cve.org/CVERecord?id=CVE-2025-14920), [CVE-2025-14921](https://www.cve.org/CVERecord?id=CVE-2025-14921), [CVE-2025-14924](https://www.cve.org/CVERecord?id=CVE-2025-14924), [CVE-2025-14927](https://www.cve.org/CVERecord?id=CVE-2025-14927), [CVE-2025-14928](https://www.cve.org/CVERecord?id=CVE-2025-14928), [CVE-2025-14929](https://www.cve.org/CVERecord?id=CVE-2025-14929), [CVE-2025-14930](https://www.cve.org/CVERecord?id=CVE-2025-14930) ## Highlights - Model Support - Add Qwen3-VL-MoE (#9689) - Support DeepSeek-V32 chat template (#9814) - Support DeepSeek-V3.2, R1 and V3.1 tool parser (#10126, #10010) - Support Eagle3 on Mistral Large3 (#9971) - Support VLM part for Mistral Large 3 (#10188) - Support multi-gpu running for nemotron-v3-nano and super (#10118) - Support Qwen3-VL dense model in pytorch backend (#9060) - Support NVFP4 for gptoss (#8956) - Add MLA Based Eagle (#9677) - API - Migrate model registry to v2.0 format with composable configs (#9836) - Support multiple post process for Responses API (#9908) - Allow YAML config overwriting CLI args for trtllm-eval (#10296) - Standardize MoE weights interface (#10295) - Expose enable_trt_overlap in Triton backend (#10018) - Feature - Support NVFP4 weight and weight_scale padding for MoE cutlass (#9358) - Add routing support for the new model for cutlass and TRTLLM MoE backend (#9792) - Improve disagg-server prometheus metrics and synchronize dynamic workers’ clocks (#9726) - Update TRT-LLM Gen MoE for NvFp4 + bias with tileN=256 (#9734) - Add optimization options for MOE CuteDSL finalized kernel (#10042) - Add fp8 bmm on sm120 (#9687) - Reuse alltoall workspace for CuteDSL MoE output (#9840) - Support Mooncake transfer engine as cache transceiver backend (#8309) - Enable KV cache reuse for config database (#10094) - Enable PDL for CuteDSL kernels and overlap MoeOutputMemset (#10043) - Cudagraph updates for helix parallelism (#10141) - Custom AllToAll for helix parallelism (#9986) - Pass MRoPE tensors for EPD disagg (#9758) - Reuse previous draft requests if possible (#10263) - Make PDL enabled by default (#9695) - Enable 2CTA with autotune for CuteDSL MoE and Grouped GEMM optimizations (#10201) - Provide attention NVFP4 out support for torch compile (#9740) - Increase topk upper limit to 22 for NVLinkOneSided AlltoAll (#10229) - Deliver optimizations for two-model speculative decoding (#10208) - Fix - Fix PDL bug in trtllm-gen FMHA kernels (#9913) - Fix Illegal Memory Access for CuteDSL Grouped GEMM (#10008) - Disable tvm_ffi for CuteDSL nvFP4 dense GEMM (#10040) - Fix ready signal in NIXL backend (#10000) - Fix top_k=10 in NVLinkOneSided AlltoAll (#10197) - Fix race conditions in KV cache communication during unexpected termination (#10076) - Fix deepseek sharding (#9984) - Fix contiguous view usage in load_expert weights (#10136) - Fix detokenizer issue for DeepSeek-v3.2 (#10106) - Fix indice offset overflow in custom Top-K kernel and UT (#10027) - Fix draft_lengths for CUDA graph capture (#10004) - Fix port conflict handling for CI (#10392, #10175, #10035) - Fix NVFP4 linear method weight and weight_scale padding (#10148) - Fix VSWA block store/load scheme in KV cache manager (#10183) - Fix ready signal and execution_stream synchronization across components (#10060) - Fix PP+CP combination with helix parallelism (#10312) - Fix Gemma3 RoPE for local attention (#9961) - Make NCCL resource manager destructor exception-safe (#10166) - Fix detokenizer / tokenizer issues (use local tokenizer, cache vocab) (#10230, #10219) - Disable PDL for quant kernels to address accuracy (#10285) - Fix hilo: Avoid property with setter in nn modules (#10212) - Documentation - Add README for Nemotron Nano v3 (#10017) - Update CONTRIBUTING.md (#10023) - Update online benchmarking docs (#9611) - Update Dynamo Example document (#9619, #10368) - Update Perf_Overview.md with benchmarking results (#9723) - Add NIXL-Libfabric usage documentation (#10205) - Add Sparse Attention feature doc (#9648) - Update IFB performance guide & GPTOSS deployment guide (#10283) - Promote perfect MoE router feature documentation (#10303) - Test & Infra - Fix credential loading in lockfile generation pipeline (#10020) - Add Qwen3-4B-Eagle3 one-model perf test (#10041) - Add regression testing for config database (#9832) - Update tests for nemotron_h (#9993) - Use ucx as default backend (#10101) - Fix OpenSearch URL in slurm_launch.sh for multinode perf sanity (#9990) - Remove helix test from RTX test list (#10224) - Add ray test robustness and RL perf reproduce script (#9939) - Support multi-node disagg perf test in CI (#9138) - Enable single-gpu CI on spark (#9304) - Add disaggregated stress test (#9354) - Include LongBenchV1 in trtllm-eval (eval infra aspect) (#10265) - Fix port conflict avoidance in CI via get_free_port_in_ci (#10392) **Full Changelog**: https://github.com/NVIDIA/TensorRT-LLM/compare/v1.2.0rc7...v1.2.0rc7 ## What's Changed * [https://nvbugs/5708810][fix] Fix TRTLLMSampler by @moraxu in https://github.com/NVIDIA/TensorRT-LLM/pull/9710 * [TRTLLM-9641][infra] Use public triton 3.5.0 in SBSA by @ZhanruiSunCh in https://github.com/NVIDIA/TensorRT-LLM/pull/9652 * [TRTLLM-8638][fix] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/9979 * [TRTLLM-9794][ci] move more test cases to gb200 by @QiJune in https://github.com/NVIDIA/TensorRT-LLM/pull/9994 * [None][feat] Add routing support for the new model for both cutlass and trtllm moe backend by @ChristinaZ in https://github.com/NVIDIA/TensorRT-LLM/pull/9792 * [TRTLLM-8310][feat] Add Qwen3-VL-MoE by @yechank-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/9689 * [https://nvbugs/5731717][fix] fixed flashinfer build race condition during test by @MrGeva in https://github.com/NVIDIA/TensorRT-LLM/pull/9983 * [FMDL-1222][feat] Support weight and weight_scale padding for NVFP4 MoE cutlass by @Wanli-Jiang in https://github.com/NVIDIA/TensorRT-LLM/pull/9358 * [None][chore] Update internal_cutlass_kernels artifacts by @yihwang-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/9992 * [None][docs] Add README for Nemotron Nano v3 by @2ez4bz in https://github.com/NVIDIA/TensorRT-LLM/pull/10017 * [None][infra] Fixing credential loading in lockfile generation pipeline by @yuanjingx87 in https://github.com/NVIDIA/TensorRT-LLM/pull/10020 * [https://nvbugs/5727952][fix] a pdl bug in trtllm-gen fmha kernels by @PerkzZheng in https://github.com/NVIDIA/TensorRT-LLM/pull/9913 * [None][infra] Waive failed test for main branch on 12/16 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/10029 * [None][doc] Update CONTRIBUTING.md by @syuoni in https://github.com/NVIDIA/TensorRT-LLM/pull/10023 * [None][fix] Fix Illegal Memory Access for CuteDSL Grouped GEMM by @syuoni in https://github.com/NVIDIA/TensorRT-LLM/pull/10008 * [TRTLLM-9181][feat] improve disagg-server prometheus metrics; synchronize workers' clocks when workers are dynamic by @reasonsolo in https://github.com/NVIDIA/TensorRT-LLM/pull/9726 * [None][chore] Final mass integration of release/1.1 by @mikeiovine in https://github.com/NVIDIA/TensorRT-LLM/pull/9960 * [None][fix] Fix iteration stats for spec-dec by @achartier in https://github.com/NVIDIA/TensorRT-LLM/pull/9855 * [https://nvbugs/5741060][fix] Fix pg op test by @shuyixiong in https://github.com/NVIDIA/TensorRT-LLM/pull/9989 * [https://nvbugs/5635153][chore] Remove responses tests from waive list by @JunyiXu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10026 * [None] [feat] Enhancements to slurm scripts by @kaiyux in https://github.com/NVIDIA/TensorRT-LLM/pull/10031 * [None][infra] Waive failed tests due to llm model files by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/10068 * [None][fix] Enabled simultaneous support for low-precision combine and MTP. by @yilin-void in https://github.com/NVIDIA/TensorRT-LLM/pull/9091 * [https://nvbugs/5698434][test] Add Qwen3-4B-Eagle3 One-model perf test by @yufeiwu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10041 * [TRTLLM-9998][fix] Change trtllm-gen MoE distributed tuning strategy back to INDEPENDENT by @hyukn in https://github.com/NVIDIA/TensorRT-LLM/pull/10036 * [TRTLLM-9989][fix] Disable tvm_ffi for CuteDSL nvFP4 dense GEMM. by @hyukn in https://github.com/NVIDIA/TensorRT-LLM/pull/10040 * [None][chore] Remove unnecessary warning log for tuning. by @hyukn in https://github.com/NVIDIA/TensorRT-LLM/pull/10077 * [TRTLLM-9680][perf] Optimize TRTLLMSampler log_probs performance (Core fix has been merged via #9353) by @tongyuantongyu in https://github.com/NVIDIA/TensorRT-LLM/pull/9655 * [None][chore] Upgrade GitHub Actions for Node 24 compatibility by @salmanmkc in https://github.com/NVIDIA/TensorRT-LLM/pull/10045 * [None][fix] Autodeploy: fix some legacy flashinfer attention test errors by @nvchenghaoz in https://github.com/NVIDIA/TensorRT-LLM/pull/9928 * [None][fix] Revert GHA upgrade for blossom-ci workflow by @tburt-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10095 * [None][chore] Clarify copyright header guidance by @tburt-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/9882 * [TRTLLM-9381][feat] Add kimi k2 fp4 tests by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/9906 * [None][chore] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10025 * [#7532][feat] AutoDeploy: gather logits before lm head by @lucaslie in https://github.com/NVIDIA/TensorRT-LLM/pull/9962 * [https://nvbugs/5721644][fix] Update tests for nemotron_h by @Wanli-Jiang in https://github.com/NVIDIA/TensorRT-LLM/pull/9993 * [None][infra] Update allowlist 2025.12.17 by @yuanjingx87 in https://github.com/NVIDIA/TensorRT-LLM/pull/10097 * [None][fix] avoid ID conversion for non enable_configurable_moe cases. by @yuxianq in https://github.com/NVIDIA/TensorRT-LLM/pull/10003 * [None][infra] Waive failed cases for main branch on 12/18 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/10105 * [https://nvbugs/5753250][infra] Waive _test_openai_responses. by @bobboli in https://github.com/NVIDIA/TensorRT-LLM/pull/10110 * [None][infra] Fix slurm job does not catch cancelled jobs by @yuanjingx87 in https://github.com/NVIDIA/TensorRT-LLM/pull/9722 * [None][feat] update TRT-LLM Gen MoE for NvFp4 + bias with tileN=256 by @nekorobov in https://github.com/NVIDIA/TensorRT-LLM/pull/9734 * [None][perf] Add more optimization options for MOE CuteDSL finalized kernel by @sherry-1001 in https://github.com/NVIDIA/TensorRT-LLM/pull/10042 * [https://nvbugs/5456493][feat] Add fp8 bmm on sm120 by @CarstyYou in https://github.com/NVIDIA/TensorRT-LLM/pull/9687 * [TRTLLM-9819][perf] Reuse alltoall workspace for CuteDSL MoE output by @syuoni in https://github.com/NVIDIA/TensorRT-LLM/pull/9840 * [https://nvbugs/5726066][fix] fix auto-scaling related failures by @reasonsolo in https://github.com/NVIDIA/TensorRT-LLM/pull/9845 * [https://nvbugs/5747911][fix] Use offline data path for the unit test of mmencoder server by @chang-l in https://github.com/NVIDIA/TensorRT-LLM/pull/10135 * [https://nvbugs/5741331][fix] Fix helix accuracy test by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10021 * [TRTC-71][feat] Add regression testing for config database by @anish-shanbhag in https://github.com/NVIDIA/TensorRT-LLM/pull/9832 * [https://nvbugs/5721912][chore] Unwaive the test by @hyukn in https://github.com/NVIDIA/TensorRT-LLM/pull/10108 * [None][infra] Fix issue that lock file geneartion will skip dependency with comment by @yuanjingx87 in https://github.com/NVIDIA/TensorRT-LLM/pull/10144 * [None][fix] Fix ready signal in NIXL backend by @chuangz0 in https://github.com/NVIDIA/TensorRT-LLM/pull/10000 * [None][feat] Support Mooncake transfer engine as a cache transceiver backend by @wjueyao in https://github.com/NVIDIA/TensorRT-LLM/pull/8309 * [TRTLLM-9840][test] switch ucx backend to default backend by @crazydemo in https://github.com/NVIDIA/TensorRT-LLM/pull/10101 * [None][chore] Waive test blocking pre-merge 12/18 by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10145 * [#9230][refactor] Replace nemotron patches with custom model implementation by @2ez4bz in https://github.com/NVIDIA/TensorRT-LLM/pull/9751 * [None][chore] Update CODEOWNERS for test cases and test list by @LarryXFly in https://github.com/NVIDIA/TensorRT-LLM/pull/10119 * [TRTLLM-7736][feat] Incrementally update the inputs of target and draft models by @ziyixiong-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/9708 * [https://nvbugs/5722653][fix] Address port conflict by assigning different port section in the same node. by @JunyiXu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10035 * [TRTLLM-8638][fix] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10132 * [TRTLLM-8830][test] Overlap scheduler enhancement perf test: Add qwen3_0,8b and llama3.1 test cases by @yufeiwu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10114 * [TRTLLM-9654][feat] Support DeepSeek-V32 chat template by @chang-l in https://github.com/NVIDIA/TensorRT-LLM/pull/9814 * [TRTLLM-9604][feat] DS R1 & V3.1 tool parser by @LinPoly in https://github.com/NVIDIA/TensorRT-LLM/pull/10010 * [None][infra] Update waive and waive failed tests for main branch on 12/19 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/10151 * [TRTLLM-8638][fix] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10129 * [#9640][feat] Migrate model registry to v2.0 format with composable configs by @tcherckez-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/9836 * [None][fix] waive the failed test test_service_discovery[etcd-load_ba… by @xxi-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10161 * [https://nvbugs/5722653][fix] Unwaive fixed test by @JunyiXu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10157 * [TRTC-102][docs] `--extra_llm_api_options`->`--config` in docs/examples/tests by @venkywonka in https://github.com/NVIDIA/TensorRT-LLM/pull/10005 * [https://nvbugs/5720357][fix] Fix indice offset overflow in custom Top-K kernel and corresponding UT case by @longcheng-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10027 * [None][fix] Revert the change and remove device count guard for DSv32 by @chang-l in https://github.com/NVIDIA/TensorRT-LLM/pull/9631 * [#10056][test] AutoDeploy: Add accuracy test for Nemotron SuperV3 by @galagam in https://github.com/NVIDIA/TensorRT-LLM/pull/10131 * [None][chore] Waive timing out pre-merge test by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10167 * [None][fix] enable KV cache reuse for config database by @anish-shanbhag in https://github.com/NVIDIA/TensorRT-LLM/pull/10094 * [None][fix] fix draft_lengths for CUDA graph capture. by @yuxianq in https://github.com/NVIDIA/TensorRT-LLM/pull/10004 * [https://nvbugs/5747930][fix] Use offline tokenizer for whisper models. by @yuxianq in https://github.com/NVIDIA/TensorRT-LLM/pull/10121 * [TRTLLM-9992][perf] Enable PDL for CuteDSL kernels and overlap MoeOutputMemset by @syuoni in https://github.com/NVIDIA/TensorRT-LLM/pull/10043 * [https://nvbugs/5643631][fix] Fix hostfunc seg fault by @syuoni in https://github.com/NVIDIA/TensorRT-LLM/pull/10028 * [https://nvbugs/5753250][infra] Further waive all tests in _test_openai_responses.py by @bobboli in https://github.com/NVIDIA/TensorRT-LLM/pull/10176 * [https://nvbugs/5744427][fix] Fix accuracy test OOM by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10173 * [TRTLLM-9805][feat] Skip Softmax Attention. by @bobboli in https://github.com/NVIDIA/TensorRT-LLM/pull/9821 * [None][chore] removed duplicated test from l0_b200.yml by @MrGeva in https://github.com/NVIDIA/TensorRT-LLM/pull/10090 * [TRTLLM-9872][fix] clear the failed test at CI when enalbe_configurab… by @xxi-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10067 * [None][infra] Waive failed tests for main branch on 12/21 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/10184 * [None] [feat] Enhancements to slurm scripts by @kaiyux in https://github.com/NVIDIA/TensorRT-LLM/pull/10112 * [None][feat] Support Eagle3 on Mistral Large3 by @byshiue in https://github.com/NVIDIA/TensorRT-LLM/pull/9971 * [https://nvbugs/5702793][fix] Fix view operation on uncontiguous tensor by @shuyixiong in https://github.com/NVIDIA/TensorRT-LLM/pull/10147 * [None][feat] Cudagraph updates for helix parallelism by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10141 * [None][ci] Waive GPTOSS test case by @syuoni in https://github.com/NVIDIA/TensorRT-LLM/pull/10155 * [https://nvbugs/5701457][fix] Unwaive ray test. by @dominicshanshan in https://github.com/NVIDIA/TensorRT-LLM/pull/10175 * [None][fix] disable cuda ipc on device without nvlink (L40s) for disagg test by @chuangz0 in https://github.com/NVIDIA/TensorRT-LLM/pull/9735 * [https://nvbugs/5701445][chore] unwaive test. by @yuxianq in https://github.com/NVIDIA/TensorRT-LLM/pull/9949 * [TRTLLM-9880][feat] Include torch compile tests in QA test list by @liji-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10149 * [https://nvbugs/5684820][fix] fix the detokenizer issue for DeepSeek-v3.2 by @lfr-0531 in https://github.com/NVIDIA/TensorRT-LLM/pull/10106 * [https://nvbugs/5666821][chore] unwaive tests. by @yuxianq in https://github.com/NVIDIA/TensorRT-LLM/pull/9958 * [None][chore] Remove closed bugs by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10182 * [None][fix] NVFP4 linear method's weight and weight_scale padding by @JadoTu in https://github.com/NVIDIA/TensorRT-LLM/pull/10148 * [https://nvbugs/5762016][chore] Skip a ray test by @shuyixiong in https://github.com/NVIDIA/TensorRT-LLM/pull/10194 * [https://nvbugs/5503479][fix] update trtllm-gen kernels to address few bugs by @PerkzZheng in https://github.com/NVIDIA/TensorRT-LLM/pull/10089 * [None][refactor] simplify get_stats and get_kvcache_events with rpc by @Superjomn in https://github.com/NVIDIA/TensorRT-LLM/pull/9980 * [None][chore] NVLinkOneSided AlltoAll Support zero local_num_tokens. by @bobboli in https://github.com/NVIDIA/TensorRT-LLM/pull/9822 * [TRTLLM-9409][feat] Pass MRoPE tensors for EPD disagg by @2ez4bz in https://github.com/NVIDIA/TensorRT-LLM/pull/9758 * [None][chore] Remove logprobs constraint on trtllm-serve pytorch backend by @LinPoly in https://github.com/NVIDIA/TensorRT-LLM/pull/9911 * [None][infra] Waive failed cases on 12/22 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/10200 * [TRTLLM-7906][feat] Support multiple post process for Responses API by @JunyiXu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/9908 * [#9717][chore] Refactor MoE code to use enums by @tcherckez-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/9910 * [TRTLLM-9847][fix] WAR fix hanging fused allreduce. by @greg-kwasniewski1 in https://github.com/NVIDIA/TensorRT-LLM/pull/10087 * [TRTLLM-9677][feat] Support DeepSeek-V3.2 tool parser by @lfr-0531 in https://github.com/NVIDIA/TensorRT-LLM/pull/10126 * [https://nvbugs/5747674][fix] Add contiguous() before view() in load_expert_w3_w1_weight and load by @farazkh80 in https://github.com/NVIDIA/TensorRT-LLM/pull/10136 * [TRTLLM-9432][feat] Reduce synchronization and recompilation for qwen3-next by @yuantailing in https://github.com/NVIDIA/TensorRT-LLM/pull/9691 * [None][fix] avoid implicit cudaStreamSynchronize in sample_async. by @yuxianq in https://github.com/NVIDIA/TensorRT-LLM/pull/10120 * [TRTLLM-9989][fix] Fix tvm_ffi aaarch64 issue. by @limin2021 in https://github.com/NVIDIA/TensorRT-LLM/pull/10199 * [None][chore] Fix GB300 support issues by @fredricz-20070104 in https://github.com/NVIDIA/TensorRT-LLM/pull/10196 * [None][test] Add qa tests for RTX 6K by @pamelap-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/10210 * [TRTLLM-9091] [feat] Replace GenAI-Perf with AIPerf by @lkomali in https://github.com/NVIDIA/TensorRT-LLM/pull/9310 * [None][chore] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10177 * [https://nvbugs/5741884][fix] unwaive disagg sampler by @chuangz0 in https://github.com/NVIDIA/TensorRT-LLM/pull/10189 * [None][chore] Bump version to 1.2.0rc7 by @yiqingy0 in https://github.com/NVIDIA/TensorRT-LLM/pull/10216 * [None][fix] Fix the bug for top_k=10 in NVLinkOneSided AlltoAll. by @bobboli in https://github.com/NVIDIA/TensorRT-LLM/pull/10197 * [None][fix] Add OpenSearch URL in slurm_launch.sh for Multinode Perf Sanity Test by @chenfeiz0326 in https://github.com/NVIDIA/TensorRT-LLM/pull/9990 * [https://nvbugs/5729697][fix] MNNVL Allreduce: use CUDA runtime instead of Macro to get SM version. by @timlee0212 in https://github.com/NVIDIA/TensorRT-LLM/pull/10062 * [None][chore] Remove helix test from rtx test list by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10224 * [https://nvbugs/5764627][chore] waive the time-out test by @hyukn in https://github.com/NVIDIA/TensorRT-LLM/pull/10222 * [None] [feat] skip batch_tokenize_prompts in CustomDataset by @qiaoxj07 in https://github.com/NVIDIA/TensorRT-LLM/pull/10214 * [https://nvbugs/5702786][fix] Fix race conditions in KV cache communication during unexpected termination by @RoeyAzran1992 in https://github.com/NVIDIA/TensorRT-LLM/pull/10076 * [None][chore] Update AD coverage to use torch-cudagraph by @tcherckez-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/10233 * [None][infra] Waive flaky unittest/executor/test_rpc_proxy.py and unittest/executor/test_rpc_worker.py tests by @dongfengy in https://github.com/NVIDIA/TensorRT-LLM/pull/10209 * [None][infra] Waive failed cases on 12/23 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/10236 * [TRTLLM-9565][fix] Fix deepseek sharding by @greg-kwasniewski1 in https://github.com/NVIDIA/TensorRT-LLM/pull/9984 * [https://nvbugs/5680312][fix] Updated test waiving by @greg-kwasniewski1 in https://github.com/NVIDIA/TensorRT-LLM/pull/9630 * [None][feat] Expose enable_trt_overlap in Triton_backend brings 1.05x OTPS by @jhaotingc in https://github.com/NVIDIA/TensorRT-LLM/pull/10018 * [TRTLLM-9493][feat] Custom AllToAll for helix parallelism by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/9986 * [None][chore] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10204 * [None][infra] Waive failed cases om 12/24 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/10257 * [None][docs] Add NIXL-Libfabric Usage to Documentation by @zackyoray in https://github.com/NVIDIA/TensorRT-LLM/pull/10205 * [TRTLLM-9798][feat] Change to use new DeepGEMM MQA sm100 kernel for MTP-3 by @lfr-0531 in https://github.com/NVIDIA/TensorRT-LLM/pull/10226 * [TRTLLM-9615][feat] Support synchronization through PP ranks in the distributed tuning system by @hyukn in https://github.com/NVIDIA/TensorRT-LLM/pull/10011 * [None][chore] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10240 * [TRTLLM-9737][chore] Add rl perf reproduce script and enhance the robustness of Ray tests by @shuyixiong in https://github.com/NVIDIA/TensorRT-LLM/pull/9939 * [None][chore] Update tinygemm kernel name by @longlee0622 in https://github.com/NVIDIA/TensorRT-LLM/pull/10248 * [#10246][feature] Move AD dashboard to use cudagraph compile backend by @tcherckez-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/10267 * [None][infra] Check GB200 coherent GPU mapping by @yiqingy0 in https://github.com/NVIDIA/TensorRT-LLM/pull/10253 * [None][test] Add disag-serving auto scaling qa test by @StanleySun639 in https://github.com/NVIDIA/TensorRT-LLM/pull/10262 * [#10052][feat] AutoDeploy enable cudagraphs for flashinfer BatchDecode by @suyoggupta in https://github.com/NVIDIA/TensorRT-LLM/pull/10193 * [None][fix] fix: resolve GPU memory imbalance in concurrent weight loading by @Nekofish-L in https://github.com/NVIDIA/TensorRT-LLM/pull/6472 * [#10137][feat] AutoDeploy FP8 MoE refactor by @nzmora-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/10138 * [TRTLLM-10143][feat] Reuse previous draft requests if possible by @ziyixiong-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10263 * [TRTLLM-9862][infra] Move single-gpu tests on rtxpro6000d to pre-merge by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/9897 * [#9241][feat] AutoDeploy: Support Eagle3 Speculative Decoding by @govind-ramnarayan in https://github.com/NVIDIA/TensorRT-LLM/pull/9869 * [TRTC-121] [feat] Add recipe selector UI to complement the recipe database by @venkywonka in https://github.com/NVIDIA/TensorRT-LLM/pull/10125 * [None][doc] Add Sparse Attention feature doc by @heyuhhh in https://github.com/NVIDIA/TensorRT-LLM/pull/9648 * [None][chore] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10249 * [None][feat] Drop non-deepgemm fp8 block scale gemm by @lucifer1004 in https://github.com/NVIDIA/TensorRT-LLM/pull/10256 * [None][ci] Waive TestLlama3_1_8B::test_auto_dtype[False-2] for timeout by @syuoni in https://github.com/NVIDIA/TensorRT-LLM/pull/10293 * [None][chore] Remove NIM TRT-Backend Test Lists by @jieli-matrix in https://github.com/NVIDIA/TensorRT-LLM/pull/10232 * [None][fix] Fix pageable H2D memcopy issue on GB200 by @qiaoxj07 in https://github.com/NVIDIA/TensorRT-LLM/pull/10289 * [None][infra] Waive failed tests for main on 12/25 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/10298 * [None] [doc] Update IFB performance guide & GPTOSS deployment guide by @jgangani in https://github.com/NVIDIA/TensorRT-LLM/pull/10283 * [TRTLLM-9578][feat] make PDL enabled by default by @dc3671 in https://github.com/NVIDIA/TensorRT-LLM/pull/9695 * [None][infra] Move install_boost from install_triton.sh to install_base.sh by @Tabrizian in https://github.com/NVIDIA/TensorRT-LLM/pull/10055 * [https://nvbugs/5752516][chore] unwaive test; fix port conflicts in CI by @reasonsolo in https://github.com/NVIDIA/TensorRT-LLM/pull/10152 * [TRTLLM-9831][perf] Enable 2CTA with autotune for CuteDSL MoE and Grouped GEMM optimizations by @sherry-1001 in https://github.com/NVIDIA/TensorRT-LLM/pull/10201 * [https://nvbugs/5652062][fix] Rewind kv_cache and reset draft tokens by @ziyixiong-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10160 * [None][fix] Cherry-pick conflict changes for PR 7999 PR 8515 by @liji-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/9446 * [None][feat] Support VLM part for Mistral Large 3 by @byshiue in https://github.com/NVIDIA/TensorRT-LLM/pull/10188 * [None][chore] Small refactoring to auto-deploy MoE operator by @nzmora-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/10300 * [None][fix] Allow YAML config overwriting CLI args for trtllm-eval by @syuoni in https://github.com/NVIDIA/TensorRT-LLM/pull/10296 * [None][feat] Support multi-gpu running for nemotron-v3-nano and super by @Wanli-Jiang in https://github.com/NVIDIA/TensorRT-LLM/pull/10118 * [None] [doc] Document perfect MoE router feature for perf analysis by @jgangani in https://github.com/NVIDIA/TensorRT-LLM/pull/10303 * [https://nvbugs/5745152][fix] Fix some GPTOSS test setups by @dongfengy in https://github.com/NVIDIA/TensorRT-LLM/pull/10085 * [https://nvbugs/5633700][fix] Cache tiktoken vocab for gpt-oss by @LinPoly in https://github.com/NVIDIA/TensorRT-LLM/pull/10219 * [https://nvbugs/5747938][fix] Use local tokenizer by @LinPoly in https://github.com/NVIDIA/TensorRT-LLM/pull/10230 * [TRTLLM-8952][feat] Support Multi-Node Disagg Perf Test in CI by @chenfeiz0326 in https://github.com/NVIDIA/TensorRT-LLM/pull/9138 * [TRTLLM-7735][feat] Attention NVFP4 out support for torch compile by @liji-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/9740 * [None][fix] Fix request_id for best_of/n case by @evezhier in https://github.com/NVIDIA/TensorRT-LLM/pull/8368 * [TRTLLM-10126][feat] Increase topk upper limit to 22 for NVLinkOneSid… by @nv-guomingz in https://github.com/NVIDIA/TensorRT-LLM/pull/10229 * [TRTLLM-8577][feat] Clean the Qwen3-next code by removing Qwen3NextCo… by @nv-guomingz in https://github.com/NVIDIA/TensorRT-LLM/pull/10228 * [None][fix] [Gemma3] Fix RoPE for local attention for Gemma3 by @shivghai in https://github.com/NVIDIA/TensorRT-LLM/pull/9961 * [https://nvbugs/5753250][fix] Fix undefined local variable in responses utils by @JunyiXu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10154 * [TRTLLM-9962][feat] Some optimizations for two-model spec dec by @ziyixiong-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10208 * [None][ci] Move remaining DGX-B200 tests to LBD by @chzblych in https://github.com/NVIDIA/TensorRT-LLM/pull/9876 * [None][ci] Waive an intermittent test hang case by @chzblych in https://github.com/NVIDIA/TensorRT-LLM/pull/10324 * [https://nvbugs/5625990][fix] Respect VSWA scheme when doing block store for reuse and load block for reuse in KV cache manager by @eopXD in https://github.com/NVIDIA/TensorRT-LLM/pull/10183 * [None][infra] Some improvements for Slurm execution path in the CI by @chzblych in https://github.com/NVIDIA/TensorRT-LLM/pull/10316 * [#9626][feat] Add an auto-deploy transform for using cutlass FP4 MoE kernels by @nzmora-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/10304 * [None][chore] Waive tests failing in pre-merge 12/28 by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10311 * [None][infra] Remove duplicates in waives.txt by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/10333 * [None][chore] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10321 * [None][chore] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10301 * [TRTLLM-9455][feat] support for new checkpoint by @binghanc in https://github.com/NVIDIA/TensorRT-LLM/pull/10082 * [TRTLLM-9965][test] add long-context disagg test for GB300/GB200 and remove config_index in yaml by @ruodil in https://github.com/NVIDIA/TensorRT-LLM/pull/10225 * [None][chore] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10302 * [https://nvbugs/5594703][infra] Unwaive the failed case to test by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/10275 * [None][infra] Enable single-gpu CI on spark by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/9304 * [None][infra] Waive failed cases for main on 12/30 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/10338 * [None][infra] Add LongBenchV1 to trtllm-eval. by @bobboli in https://github.com/NVIDIA/TensorRT-LLM/pull/10265 * [https://nvbugs/5766986][fix] fixed the shard_all_unprocessed default value to align with the default.yml by @MrGeva in https://github.com/NVIDIA/TensorRT-LLM/pull/10271 * [https://nvbugs/5769890][fix] Import get_free_port. by @yuxianq in https://github.com/NVIDIA/TensorRT-LLM/pull/10341 * [None][feat] Implement send_object for TorchDist. by @yuxianq in https://github.com/NVIDIA/TensorRT-LLM/pull/10213 * [https://nvbugs/5707359][fix] Unwaive OOM case that should be fixed by #9446 by @liji-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10334 * [None][chore] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10344 * [None][fix] disable thread leak check for kimi by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10337 * [None][chore] Unify DS tool parser names by @LinPoly in https://github.com/NVIDIA/TensorRT-LLM/pull/10239 * [TRTLLM-10016][infra] Use SlurmPatition attribute time as timeout threshold by @yiqingy0 in https://github.com/NVIDIA/TensorRT-LLM/pull/10254 * [https://nvbugs/5774869][chore] waive tests. by @yuxianq in https://github.com/NVIDIA/TensorRT-LLM/pull/10356 * [https://nvbugs/5558516][test] add disaggregated stress test by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/9354 * [None][feat] support Qwen3-VL dense model in pytorch backend by @Nekofish-L in https://github.com/NVIDIA/TensorRT-LLM/pull/9060 * [None][infra] Waive failed cases on 12/31 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/10353 * [https://nvbugs/5727475][fix] Avoid use property with setter in nn.Mo… by @liji-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10212 * [#9717][chore] Standardize MoE weights interface by @tcherckez-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/10295 * [TRTLLM-9834][feat] Transfer to TRTLLM-INFRA Database and Fail post-merge tests if regression by @chenfeiz0326 in https://github.com/NVIDIA/TensorRT-LLM/pull/10282 * [TRTLLM-8638][fix] Add failed cases into waives.txt by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10354 * [https://nvbugs/5717993][fix] Add execution_stream across PyExecutor, KVCacheManager, PeftCacheManager to ensure proper CUDA stream synchronization between KV cache transfer operations and model forward kernels. by @SimengLiu-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10060 * [None][feat] Implement sampling for MTP 1-model by @mikeiovine in https://github.com/NVIDIA/TensorRT-LLM/pull/10019 * [#10244][feat] AutoDeploy: separate prefill/decode in flashinfer by @lucaslie in https://github.com/NVIDIA/TensorRT-LLM/pull/10252 * [https://nvbugs/5740359][chore] Unwaive tests. by @yuxianq in https://github.com/NVIDIA/TensorRT-LLM/pull/10260 * [https://nvbugs/5744427][fix] Make Gemma3 multimodal test fp8 by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10368 * [None][chore] Waive tests blocking pre-merge 12/31 by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10373 * [None][feat] Add export data to build and run script for AD by @tcherckez-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/10299 * [#10056][fix] AutoDeploy: Handle deletion of nested params in sharding by @galagam in https://github.com/NVIDIA/TensorRT-LLM/pull/10376 * [TRTLLM-9467][fix] Fix PP+CP combination with helix parallelism by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10312 * [None][test] Unified slurm extra args management and session collection logic by @fredricz-20070104 in https://github.com/NVIDIA/TensorRT-LLM/pull/10332 * [None][fix] Minor updates on Perf Test System by @chenfeiz0326 in https://github.com/NVIDIA/TensorRT-LLM/pull/10375 * [#10056][chore] AutoDeploy: Enable Nemo SuperV3 accuracy test by @galagam in https://github.com/NVIDIA/TensorRT-LLM/pull/10308 * [None][feat] Eagle: MLA Based Eagle by @IzzyPutterman in https://github.com/NVIDIA/TensorRT-LLM/pull/9677 * [None][doc] promote AutoDeploy to beta feature in docs by @lucaslie in https://github.com/NVIDIA/TensorRT-LLM/pull/10372 * [TRTLLM-9752][fix] WAR: Disable PDL for quant kernels to fix accuracy issues by @bo-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/10285 * [None][infra] Waive failed cases on 1/3 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/10391 * [None][fix] [fix] Make NCCL resource manager destructor exception-safe by @nv-lschneider in https://github.com/NVIDIA/TensorRT-LLM/pull/10166 * [TRTLLM-10358][feat] Added proper rescaling of FP4 weights by @greg-kwasniewski1 in https://github.com/NVIDIA/TensorRT-LLM/pull/10378 * [None][infra] add retry logic to get slurm sbatch job log when ssh dropped by @yuanjingx87 in https://github.com/NVIDIA/TensorRT-LLM/pull/9167 * [https://nvbugs/5748683][fix] Use get_free_port_in_ci to avoid port conflict. by @yuxianq in https://github.com/NVIDIA/TensorRT-LLM/pull/10392 * [TRTLLM-10171][fix] Correct attention handling in ModelConfig and KVCacheManager by @jaedeok-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/10330 * [TRTLLM-7138][feat] Support nvfp4 for gptoss by @dongfengy in https://github.com/NVIDIA/TensorRT-LLM/pull/8956 * [None][ci] Some tweaks for the CI pipeline by @chzblych in https://github.com/NVIDIA/TensorRT-LLM/pull/10359 * [None][fix] Decrease Pre Merge Perf Tests by @chenfeiz0326 in https://github.com/NVIDIA/TensorRT-LLM/pull/10390 ## New Contributors * @salmanmkc made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/10045 * @sherry-1001 made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/10042 * @LarryXFly made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/10119 * @longcheng-nv made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/10027 * @lkomali made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/9310 * @RoeyAzran1992 made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/10076 * @jgangani made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/10283 * @shivghai made their first contribution in https://github.com/NVIDIA/TensorRT-LLM/pull/9961 **Full Changelog**: https://github.com/NVIDIA/TensorRT-LLM/compare/v1.2.0rc6...v1.2.0rc7