`bfloat16` support
Hi!
I was wondering if you would be interested in adding `bf16` support to MFA or at least the GEMM kernels? For `mlx` Apple defined a custom type: https://github.com/ml-explore/mlx/blob/76c919b4ecf0cccaa1cfef214d12be0ad71485cc/mlx/backend/metal/kernels/bf16.h (MIT licensed), so I understand supporting this is not easy and maybe not even desirable because it's not a native type and performance is not great anyway.
Btw, I came here via huggingface/candle. It uses libMFA for matmul and FA.
关闭于 2024-08-27 5 条评论