[Bug]: Clamp very small non-zero temperature values to avoid numerical instability

### System Info CPU architecture: x86_64 CPU: Intel Xeon Gold 6462C, 8 vCPUs (4 cores × 2 threads), max 3900 MHz GPU name: NVIDIA L20 GPU memory size: 48G TensorRT-LLM branch or tag: main TensorRT-LLM commit: https://github.com/NVIDIA/TensorRT-LLM/commit/28acbad14b442e89312cfe9e845a5f721a3e702f PyTorch: 2.10.0+cu130 CUDA: 13.0.88 cuDNN: 9.15.0 NVIDIA driver version: 550.163.01 OS: Ubuntu 24.04.3 LTS ### Who can help? _No response_ ### Information - [ ] The official example scripts - [x] My own modified scripts ### Tasks - [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...) - [x] My own task or dataset (give details below) ### Reproduction ```python from tensorrt_llm.sampling_params import SamplingParams # Before the fix: very small non-zero temperature bypasses validation # and enters the sampling backend, causing potential inf/nan in logits/temperature sp = SamplingParams(temperature=1e-12, top_k=50) print(sp.temperature) # prints 1e-12 — dangerously small # Simulating what happens inside the backend: import torch logits = torch.tensor([1.0, 2.0, 3.0]) temperature = 1e-12 scaled = logits / temperature print(scaled) # tensor([1.0e+12, 2.0e+12, 3.0e+12]) → leads to inf/nan in softmax print(torch.softmax(scaled, dim=-1)) # tensor([0., 0., 1.]) or nan depending on dtype ``` ### Expected behavior SamplingParams should reject or safely normalize temperature values that are positive but below a numerical stability threshold. Specifically: - temperature=0.0 → kept as 0.0 (greedy decoding) - temperature=1e-12 → clamped to MIN_SAMPLING_TEMPERATURE (1e-2) - temperature=1.0 → kept as 1.0 (normal sampling) ### actual behavior amplingParams only validates temperature >= 0, so values like 1e-12 pass validation unchanged and are forwarded to the sampling backend. Inside the backend, logits / temperature with temperature=1e-12 produces extremely large logit values (e.g., 1e+12), which overflow to inf or nan in float16/bfloat16 precision, leading to undefined sampling behavior. Example: ```python import torch logits = torch.tensor([1.0, 2.0, 3.0], dtype=torch.float16) scaled = logits / 1e-12 print(scaled) # tensor([inf, inf, inf], dtype=torch.float16) print(torch.softmax(scaled, dim=-1)) # tensor([nan, nan, nan], dtype=torch.float16) ``` ### additional notes 1. This fix mirrors the same numerical stability guard used by vLLM (temperature = max(temperature, 1e-2) for non-zero values). 2. The issue is most severe in float16/bfloat16: typical logit values (~10) divided by 1e-6 already exceed float16's max range (~65504), producing inf/nan in softmax. 3. temperature=0.0 (greedy decoding) is intentionally excluded from the clamp and its behavior is unchanged. ### Before submitting a new issue... - [x] Make sure you already searched for relevant issues, and checked the [documentation](https://nvidia.github.io/TensorRT-LLM/) and [examples](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples) for answers to frequently asked questions.