ITADN

Error in MFA with Flux.1

#59Closedbrappier 创建于 2024-08-09
B
brappiercommented
Thanks for the Flux implementaion. It works great with flux.1 model!! I had replaced the attention with MFA: removed: ``` keys = keys.permuted(0, 2, 1, 3).contiguous() queries = ((1.0 / Float(k).squareRoot()) * queries) .permuted(0, 2, 1, 3).contiguous() values = values.permuted(0, 2, 1, 3).contiguous() var dot = Matmul(transposeB: (2, 3))(queries, keys) dot = dot.reshaped([b * h * (t + hw), t + hw]) dot = dot.softmax() dot = dot.reshaped([b, h, (t + hw), t + hw]) var out = dot * values out = out.reshaped([b, h, (t + hw), k]).transposed(1, 2).reshaped([b, (t + hw), h * k]) ``` added ``` let scaledDotProductAttention = ScaledDotProductAttention(scale: 1.0 / Float(k).squareRoot()) queries = queries.reshaped(.HWC(b, (t + hw), k * h)) keys = keys.reshaped(.HWC(b, (t + hw), k * h)) values = values.reshaped(.HWC(b, (t + hw), k * h)) var out = scaledDotProductAttention(queries, keys, values).reshaped([b, (t + hw), k * h]) ``` But I got the following Error: ``` [Metal] Encountered unexpected error in: pipeline external/ccv/lib/nnc/mfa/ccv_nnc_mfa_attention.cpp:431: error: Compute function exceeds available temporary registers [Metal] Quitting now. ``` Do you have any idea on how to fix it?
关闭于 2024-08-09 1 条评论