ITADN

Add Gradient Crop

#7Pull Requestbfloat16 创建于 2024-05-02已合并
B
bfloat16commented
Only tested on CUDA, MUSA compatibility unknown. If the loss still takes off, try changing the second parameter of ```clip_grad_value_(model.parameters(), 1)``` to something smaller (e.g. 1 to 0.8)
合并状态:已合并 合并于 2024-05-02 关闭于 2024-05-02 0 条评论