ITADN

benchmark v2 gaudi2/gaudi3 results

#115Openhuijjj 创建于 2025-08-28
H
huijjjcommented
Hi, I’m an engineer at SqueezeBits Inc., focusing on optimizing AI workloads across various hardware platforms. First of all, thank you for the excellent work here — I’m sincerely glad to see such an impressive project. I’d like to contribute by filling in the numbers for Gaudi2 and Gaudi3 in compute/accelerator/README.md. As of now, benchmarking for BF16 on Gaudi3 is about 70% complete. The current MAMF is 1241 TFLOPS, which corresponds to roughly 74% of peak performance. I’ve also just started benchmarking BF16 on Gaudi2. I expect to deliver reliable BF16 results for both Gaudi2 and Gaudi3 using the existing benchmark code. Preparing the actual PR will likely take about a week. For FP8 benchmarking, however, I noticed that the SynapseAI compiler stack tends to eliminate the matrix multiplication entirely, replacing it with a simple transpose (see the attached device trace screenshot). To obtain accurate FP8 results, it seems that some code modifications will be required. Would you be open to me debugging this issue and making the necessary changes to properly benchmark FP8 performance on Gaudi, or would you prefer that I limit my contribution to providing BF16 results only? <img width="1594" height="518" alt="Image" src="https://github.com/user-attachments/assets/0033f8ec-c347-4f34-8711-33e2836f1f75" />
16 条评论