[Bug]: Qwen3.5-397B-A17B (bf16) OOMs host RAM on the PyTorch backend; AutoDeploy loads fine on the same node

### System Info - GPU: 8× NVIDIA H200 (SM90), 2,015 GB host RAM - TensorRT-LLM: 1.3.0rc16 (verified against the source tree) - Container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc16 - Model: Qwen3.5-397B-A17B (bf16; hybrid full-attention + Gated DeltaNet) - Serving: trtllm-serve, TP8 ### Who can help? _No response_ ### Information - [x] The official example scripts - [ ] My own modified scripts ### Tasks - [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...) - [ ] My own task or dataset (give details below) ### Reproduction trtllm-serve Qwen3.5-397B-A17B --backend pytorch --tp_size 8 ### Expected behavior The model loads within available host RAM (it fits in GPU across 8×H200). The **AutoDeploy** backend loads the same checkpoint on the same node using ~69 GB committed host RAM (mmap-shared). ### actual behavior The PyTorch backend exhausts host RAM during weight load and OOMs the node. Measured peak **committed** host RAM (the OOM-relevant metric): | Deployment | Modeling path | Loader behavior | Peak committed host | Result | |---|---|---|---|---| | Qwen3.5 AutoDeploy (accelerate, mmap) | — | mmap/shared | 69 GB | served (~10 min) | | Qwen3.5 PyTorch (qwen3_5/qwen3_next) | accumulates, never frees | ~2,013 GB | OOM | never served (54 min) | ### additional notes Root cause (read against the 1.3.0rc16 tree): 1. The shared reader `HfWeightLoader._load_safetensors_file` (`tensorrt_llm/_torch/models/checkpoints/hf/weight_loader.py`) uses `safetensors.torch.load_file` (**no mmap**) → each of the 8 TP ranks materializes a private host copy. 2. Per-model `load_weights` diverges: - `deepseek_v3` (`modeling_deepseekv3.py`) drains the host dict as it streams to GPU via `.cuda()` + `weights.mark_consumed(...)` → host stays bounded and releases. - `qwen3_5` / `qwen3_next` (`modeling_qwen3_next.py`) does **not** call `.cuda()`/`mark_consumed` during load → the host weight dict is never drained → it accumulates → ~8× checkpoint → ~2 TB → OOM. ### Before submitting a new issue... - [x] Make sure you already searched for relevant issues, and checked the [documentation](https://nvidia.github.io/TensorRT-LLM/) and [examples](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples) for answers to frequently asked questions.