## What's Changed * [Cute][Testing] Minor improvements on pytest-xdist workflow by @Alkaid-Benetnash in https://github.com/Dao-AILab/flash-attention/pull/2311 * Nicer headdim error message by @drisspg in https://github.com/Dao-AILab/flash-attention/pull/2227 * [Fwd,Sm100] Extract named barriers by @drisspg in https://github.com/Dao-AILab/flash-attention/pull/2309 * Change 2cta opt in to have min seqlen > 2*m_block_size by @drisspg in https://github.com/Dao-AILab/flash-attention/pull/2320 * [CuteDSL][SM90] varlen bwd works by @KareemMusleh in https://github.com/Dao-AILab/flash-attention/pull/2275 * Add Logging helper by @drisspg in https://github.com/Dao-AILab/flash-attention/pull/2327 * [CuTeDSL][Sm80] basic fix for new api by @zhuochenKIDD in https://github.com/Dao-AILab/flash-attention/pull/2297 * fix: duplicate softmax_scale param by @NanoCode012 in https://github.com/Dao-AILab/flash-attention/pull/2328 * Fix FA2 + FA4 co-existence by @drisspg in https://github.com/Dao-AILab/flash-attention/pull/2331 * [Cute,Sm100] Introduce a flexible lambda-based R2P masking by @Alkaid-Benetnash in https://github.com/Dao-AILab/flash-attention/pull/2313 * [Cute, SM90, bwd] Wire seqused_q/k through backward pass by @NJX-njx in https://github.com/Dao-AILab/flash-attention/pull/2315 * SM120 forward pass (Blackwell GeForce / DGX Spark) by @blake-snc in https://github.com/Dao-AILab/flash-attention/pull/2329 * [cutlass] Allow compilation of cutlass FA3 for sm100 via enable_sm90 by @henrylhtsang in https://github.com/Dao-AILab/flash-attention/pull/2332 * [Cute] fix: rename logging module to avoid circular import at building by @Luosuu in https://github.com/Dao-AILab/flash-attention/pull/2335 * BUG: SeqlenInfo.create has a tile parameter that defaults to 128 by @risan-raja in https://github.com/Dao-AILab/flash-attention/pull/2337 * [Fwd,SM100,CuTe] Fix split KV OOM with diff headdim + fix SM100 kwarg mismatch by @MatthewBonanni in https://github.com/Dao-AILab/flash-attention/pull/2338 * [AMD] Migrate to Triton Backend to Aiter by @micmelesse in https://github.com/Dao-AILab/flash-attention/pull/2230 * [Bwd,Sm120] Add SM120 backward pass support by @blake-snc in https://github.com/Dao-AILab/flash-attention/pull/2330 * [Bwd, SM80] Fix tdKrdS typo by @henrylhtsang in https://github.com/Dao-AILab/flash-attention/pull/2341 * Add SM120 varlen attention support by @blake-snc in https://github.com/Dao-AILab/flash-attention/pull/2333 * fix the create_ragged_tensor_for_tma issue by @rainj-me in https://github.com/Dao-AILab/flash-attention/pull/2345 * [Sm90] Fix test_mask_mod and bwd block-sparse kwarg mismatch by @henrylhtsang in https://github.com/Dao-AILab/flash-attention/pull/2365 * [Cute, Testing] Fix aot + tvm-ffi EnvStream related parameter mismatch by @Alkaid-Benetnash in https://github.com/Dao-AILab/flash-attention/pull/2369 * [Cute, Testing] Bump cutedsl to 4.4.2 and remove prior aot cache management workarounds by @Alkaid-Benetnash in https://github.com/Dao-AILab/flash-attention/pull/2370 * [Cute] fix: FA4 paged attention kv load for DeepSeek (192,128) on SM100 by @Luosuu in https://github.com/Dao-AILab/flash-attention/pull/2368 * [AMD ROCm] Update ROCm/CK backend to align with latest ComposableKernel API changes by @rocking5566 in https://github.com/Dao-AILab/flash-attention/pull/2363 * [ROCm] Auto-detect Triton backend if C++ extension is missing by @Soddentrough in https://github.com/Dao-AILab/flash-attention/pull/2343 * [Fwd,Sm90] Add paged KV attention support (tma and cp.async) by @henrylhtsang in https://github.com/Dao-AILab/flash-attention/pull/2360 * [CuTe,Flex] limit vec_size to 2 for score mod when not on Sm100 by @reubenconducts in https://github.com/Dao-AILab/flash-attention/pull/2371 * Support 2CTA for sliding window hdim 192 by @Inodayy in https://github.com/Dao-AILab/flash-attention/pull/2347 * [Cute,Fwd,Sm100] support irregular qhead / kvhead ratios by @timmy-feng in https://github.com/Dao-AILab/flash-attention/pull/2186 * benchmarks: add MFU% column to benchmark output by @Johnsonms in https://github.com/Dao-AILab/flash-attention/pull/2377 * Update flow to enable beta weekly releases by @drisspg in https://github.com/Dao-AILab/flash-attention/pull/2378 ## New Contributors * @NJX-njx made their first contribution in https://github.com/Dao-AILab/flash-attention/pull/2315 * @blake-snc made their first contribution in https://github.com/Dao-AILab/flash-attention/pull/2329 * @Luosuu made their first contribution in https://github.com/Dao-AILab/flash-attention/pull/2335 * @risan-raja made their first contribution in https://github.com/Dao-AILab/flash-attention/pull/2337 * @MatthewBonanni made their first contribution in https://github.com/Dao-AILab/flash-attention/pull/2338 * @rainj-me made their first contribution in https://github.com/Dao-AILab/flash-attention/pull/2345 * @Soddentrough made their first contribution in https://github.com/Dao-AILab/flash-attention/pull/2343 * @Inodayy made their first contribution in https://github.com/Dao-AILab/flash-attention/pull/2347 * @Johnsonms made their first contribution in https://github.com/Dao-AILab/flash-attention/pull/2377 **Full Changelog**: https://github.com/Dao-AILab/flash-attention/compare/fa4-v4.0.0.beta4...fa4-v4.0.0.beta6