Example how to use with Python app
Would you be so kind as to provide an example of using this implementation of MetalFlashAttention compiled as a library as a replacement for the original flash_attn with Python code running a model such as https://huggingface.co/microsoft/Phi-4-multimodal-instruct?
And thanks for [your article](https://medium.com/engineering-draw-things/metal-flashattention-2-0-pushing-forward-on-device-inference-training-on-apple-silicon-fe8aac1ab23c)!
Thank you!
3 条评论