Vulkan stability problem while training with fusion

**Describe the bug** I'm trying to train an AlphaGo-like network where I have value and policy heads. When training with Vulkan value loss is mostly fine, but policy loss doesn't decrease, and after some time even starts to increase until it explodes. Rocm doesn't have this problem, as well as Vulkan without autotune, both losses decrease as expected. The model is defined [here](https://github.com/pointsgame/oppai-rs/blob/33db62ae657be2926a2f8296aee3dd0bc8ac2b80/zero-burn/src/model.rs). **To Reproduce** 1. `git clone --revision=33db62ae657be2926a2f8296aee3dd0bc8ac2b80 https://github.com/pointsgame/oppai-rs.git && cd oppai-rs/zero-train` 2. `cargo build --release --features 'rocm,vulkan'` (or `cuda` if you like) 3. `../target/release/oppai-zero-train --backend Wgpu init --model ./model_0.mpk --optimizer ./optimizer_0.mpk` 4. `../target/release/oppai-zero-train --backend Wgpu train --model ./model_0.mpk --optimizer ./optimizer_0.mpk --model-new ./model_1.mpk --optimizer-new ./optimizer_1.mpk --batch-size 512 --learning-rate-start 0.01 --learning-rate-end 0.01 --width 18 --height 18 --games ./games.sgf` Download and extract these games: http://kropki.org/static/games.7z **Actual behavior** Policy loss doesn't decrease: ``` [2026-06-15T15:48:07Z INFO oppai_zero_burn::model] Loss: value 1.2461083 policy 5.563547 opponent policy 0.8347978 pdf 0.13816915 cdf 1.6666768 L2 norm 68.48315 [2026-06-15T15:48:11Z INFO oppai_zero_burn::model] Loss: value 1.1240056 policy 5.5585446 opponent policy 0.83435917 pdf 0.13816905 cdf 1.6667739 L2 norm 68.482956 [2026-06-15T15:48:13Z INFO oppai_zero_burn::model] Loss: value 1.0980561 policy 5.5456495 opponent policy 0.8328711 pdf 0.13816994 cdf 1.6669087 L2 norm 68.48277 [2026-06-15T15:48:15Z INFO oppai_zero_burn::model] Loss: value 1.1109565 policy 5.5586147 opponent policy 0.834342 pdf 0.13817103 cdf 1.6667825 L2 norm 68.48255 [2026-06-15T15:48:17Z INFO oppai_zero_burn::model] Loss: value 1.0351858 policy 5.559708 opponent policy 0.8349808 pdf 0.1381714 cdf 1.6670176 L2 norm 68.482216 [2026-06-15T15:48:19Z INFO oppai_zero_burn::model] Loss: value 0.97435343 policy 5.5602813 opponent policy 0.83469045 pdf 0.13817154 cdf 1.6668878 L2 norm 68.48187 [2026-06-15T15:48:22Z INFO oppai_zero_burn::model] Loss: value 0.95400274 policy 5.555773 opponent policy 0.8344088 pdf 0.13817221 cdf 1.6672219 L2 norm 68.481636 [2026-06-15T15:48:24Z INFO oppai_zero_burn::model] Loss: value 0.98619366 policy 5.552112 opponent policy 0.8331547 pdf 0.13817318 cdf 1.6668278 L2 norm 68.4815 [2026-06-15T15:48:26Z INFO oppai_zero_burn::model] Loss: value 1.0162333 policy 5.5528693 opponent policy 0.83376014 pdf 0.13817248 cdf 1.6667953 L2 norm 68.481415 [2026-06-15T15:48:28Z INFO oppai_zero_burn::model] Loss: value 0.89178014 policy 5.551481 opponent policy 0.8334638 pdf 0.13817152 cdf 1.6668042 L2 norm 68.48133 [2026-06-15T15:48:30Z INFO oppai_zero_burn::model] Loss: value 0.9294362 policy 5.5509453 opponent policy 0.83344144 pdf 0.13817163 cdf 1.6669475 L2 norm 68.481346 [2026-06-15T15:48:32Z INFO oppai_zero_burn::model] Loss: value 0.91246307 policy 5.5543694 opponent policy 0.83380455 pdf 0.1381715 cdf 1.6668826 L2 norm 68.48142 [2026-06-15T15:48:34Z INFO oppai_zero_burn::model] Loss: value 0.93387365 policy 5.55659 opponent policy 0.8343021 pdf 0.13817163 cdf 1.6672544 L2 norm 68.48151 [2026-06-15T15:48:36Z INFO oppai_zero_burn::model] Loss: value 0.88695407 policy 5.55254 opponent policy 0.83340037 pdf 0.13817114 cdf 1.6669981 L2 norm 68.481606 [2026-06-15T15:48:39Z INFO oppai_zero_burn::model] Loss: value 0.86959183 policy 5.547882 opponent policy 0.8333068 pdf 0.13817099 cdf 1.6668081 L2 norm 68.48174 [2026-06-15T15:48:41Z INFO oppai_zero_burn::model] Loss: value 0.95188105 policy 5.553667 opponent policy 0.83371264 pdf 0.13817117 cdf 1.6665397 L2 norm 68.48189 [2026-06-15T15:48:43Z INFO oppai_zero_burn::model] Loss: value 0.92090195 policy 5.557854 opponent policy 0.8344412 pdf 0.13817035 cdf 1.6666538 L2 norm 68.48203 [2026-06-15T15:48:45Z INFO oppai_zero_burn::model] Loss: value 0.8765797 policy 5.5513606 opponent policy 0.8335907 pdf 0.13817026 cdf 1.6669209 L2 norm 68.482155 [2026-06-15T15:48:47Z INFO oppai_zero_burn::model] Loss: value 0.8631395 policy 5.549721 opponent policy 0.83294207 pdf 0.1381701 cdf 1.6670537 L2 norm 68.4823 [2026-06-15T15:48:49Z INFO oppai_zero_burn::model] Loss: value 0.8947104 policy 5.5488667 opponent policy 0.833186 pdf 0.13816965 cdf 1.6667923 L2 norm 68.48248 [2026-06-15T15:48:51Z INFO oppai_zero_burn::model] Loss: value 0.8754648 policy 5.5558896 opponent policy 0.83368367 pdf 0.13816948 cdf 1.6667346 L2 norm 68.482635 [2026-06-15T15:48:53Z INFO oppai_zero_burn::model] Loss: value 0.8820871 policy 5.546515 opponent policy 0.83315426 pdf 0.138169 cdf 1.6665565 L2 norm 68.48279 [2026-06-15T15:48:56Z INFO oppai_zero_burn::model] Loss: value 0.8937218 policy 5.5549784 opponent policy 0.8338867 pdf 0.13816808 cdf 1.6665064 L2 norm 68.48296 [2026-06-15T15:48:58Z INFO oppai_zero_burn::model] Loss: value 0.93352807 policy 5.5565367 opponent policy 0.834227 pdf 0.13816778 cdf 1.6668507 L2 norm 68.483154 [2026-06-15T15:49:00Z INFO oppai_zero_burn::model] Loss: value 0.82966256 policy 5.5671535 opponent policy 0.83555126 pdf 0.13816704 cdf 1.6665063 L2 norm 68.48334 [2026-06-15T15:49:02Z INFO oppai_zero_burn::model] Loss: value 0.82685083 policy 5.5614877 opponent policy 0.83539927 pdf 0.13816638 cdf 1.666679 L2 norm 68.483604 ``` And eventually it will explode. **Expected behavior** Policy loss should decrease: ``` [2026-06-15T15:52:40Z INFO oppai_zero_burn::model] Loss: value 1.2442088 policy 5.546191 opponent policy 0.8311181 pdf 0.13816999 cdf 1.6667922 L2 norm 68.48315 [2026-06-15T15:52:44Z INFO oppai_zero_burn::model] Loss: value 1.1132236 policy 5.557571 opponent policy 0.83284026 pdf 0.13816941 cdf 1.6667738 L2 norm 68.48296 [2026-06-15T15:52:46Z INFO oppai_zero_burn::model] Loss: value 1.1062258 policy 5.556683 opponent policy 0.83275515 pdf 0.13816953 cdf 1.6665276 L2 norm 68.48278 [2026-06-15T15:52:49Z INFO oppai_zero_burn::model] Loss: value 1.1185349 policy 5.555852 opponent policy 0.83288425 pdf 0.13817057 cdf 1.666806 L2 norm 68.48254 [2026-06-15T15:52:51Z INFO oppai_zero_burn::model] Loss: value 1.0266815 policy 5.545779 opponent policy 0.8315447 pdf 0.13817124 cdf 1.666926 L2 norm 68.48218 [2026-06-15T15:52:53Z INFO oppai_zero_burn::model] Loss: value 0.9869168 policy 5.5529428 opponent policy 0.8330492 pdf 0.13817112 cdf 1.6667402 L2 norm 68.48185 [2026-06-15T15:52:55Z INFO oppai_zero_burn::model] Loss: value 0.98968506 policy 5.535971 opponent policy 0.8309017 pdf 0.13817234 cdf 1.6668082 L2 norm 68.48161 [2026-06-15T15:52:58Z INFO oppai_zero_burn::model] Loss: value 0.9796931 policy 5.551389 opponent policy 0.83375907 pdf 0.13817221 cdf 1.6670415 L2 norm 68.48147 [2026-06-15T15:53:00Z INFO oppai_zero_burn::model] Loss: value 0.948686 policy 5.534304 opponent policy 0.8313129 pdf 0.13817234 cdf 1.6671652 L2 norm 68.48136 [2026-06-15T15:53:02Z INFO oppai_zero_burn::model] Loss: value 0.92028415 policy 5.537883 opponent policy 0.8326097 pdf 0.13817167 cdf 1.6669214 L2 norm 68.48131 [2026-06-15T15:53:04Z INFO oppai_zero_burn::model] Loss: value 0.8701049 policy 5.535105 opponent policy 0.8326362 pdf 0.13817152 cdf 1.666951 L2 norm 68.48134 [2026-06-15T15:53:07Z INFO oppai_zero_burn::model] Loss: value 0.89616084 policy 5.5335526 opponent policy 0.83276707 pdf 0.13817142 cdf 1.6670594 L2 norm 68.481476 [2026-06-15T15:53:09Z INFO oppai_zero_burn::model] Loss: value 0.9020816 policy 5.5247617 opponent policy 0.83185124 pdf 0.13817199 cdf 1.66678 L2 norm 68.48162 [2026-06-15T15:53:11Z INFO oppai_zero_burn::model] Loss: value 0.85129917 policy 5.5210614 opponent policy 0.8318867 pdf 0.13817182 cdf 1.6670634 L2 norm 68.48175 [2026-06-15T15:53:13Z INFO oppai_zero_burn::model] Loss: value 0.8710419 policy 5.5203543 opponent policy 0.83223456 pdf 0.13817091 cdf 1.6667024 L2 norm 68.48194 [2026-06-15T15:53:15Z INFO oppai_zero_burn::model] Loss: value 0.89716905 policy 5.5138144 opponent policy 0.831761 pdf 0.13817103 cdf 1.6668032 L2 norm 68.48213 [2026-06-15T15:53:18Z INFO oppai_zero_burn::model] Loss: value 0.9183528 policy 5.4998097 opponent policy 0.83040696 pdf 0.13817154 cdf 1.6668886 L2 norm 68.48231 ``` **Desktop:** - OS: Linux - Graphics card: AMD 9700 XTX with Mesa - Burn 0.21.0