CUDA-PyTorch Extension Speed-up

Hi, Thanks much for the awesome video series! I just finished it and learned so much. I have a question. In the PyTorch extension using CUDA, you benchmarked a polynomial activation function implemented with CUDA against the native PyTorch API. I was surprised that the CUDA version was about 4x faster than PyTorch's native API. What could be the possible reasons for this speed-up? Doesn't PyTorch use cuBLAS and cuDNN under the hood, which are already super optimized? I remember in the "Faster Multiplication" chapter that it was very challenging to match cuBLAS's speed, but here, it seems pretty easy to surpass it with polynomial activation, even 4 times faster. Thanks again!