ITADN

Incorrect initialization of max_val in the softmax_cuda kernel function may lead to inaccurate calculation results.

#10ClosedjokerD888 创建于 2025-02-14
J
jokerD888commented
https://github.com/Infatoshi/cuda-course/blob/master/08_Triton/02_softmax.cu line 12 float max_val = input[offset + tid]; In the kernel softmax_cuda (line 12), each thread initializes max_val with its own element input[offset + tid]. This leads to incorrect results because each thread only compares its own value, not the global maximum for the entire batch. Problem: The max_val is calculated per thread, causing incorrect softmax calculations. Only the thread responsible for the maximum value computes it correctly, while others compute incorrect results. Solution: float max_val = input[offset];
关闭于 2026-03-11 1 条评论