NUM_WARMUP_UPDATES multiplied by number of GPU's during multi-GPU training

### Checks - [x] This template is only for usage issues encountered. - [x] I have thoroughly reviewed the project documentation but couldn't find information to solve my problem. - [x] I have searched for existing issues, including closed ones, and couldn't find a solution. - [x] I am using English to submit this issue to facilitate community communication. ### Environment Details Hi, I’m finetuning F5 on ~47k hours of multilingual data using 8 H100s. I set: Base ckpt: https://huggingface.co/SWivid/F5-TTS/blob/main/F5TTS_Base/model_1200000.pt `NUM_WARMUP_UPDATES=20000` but during training the effective warmup appears to be 20k × 8 = 160k steps. Is warmup counted per-GPU or globally? Is this scaling with world size expected, or should warmup be specified as global steps? Also, do these params look reasonable for this scale? ``` BATCH_SIZE_PER_GPU=35000 LR=7e-5 EPOCHS=10 ``` Any guidance on warmup/LR for large multi-GPU runs would be appreciated. Thanks! ### Steps to Reproduce No response ### ✔️ Expected Behavior No response ### ❌ Actual Behavior No response