ITADN

Unusual Initializer Range default

#114Closedevanatyourservice 创建于 2024-08-13
Hello @young-geng! Thank you for this concise and easily hackable repo! Unfortunately I spent days wondering what was weird about training before I found the default init value is 1.0. I think a suitable value would be 0.02? Would be happy to submit a pull request! 1.0 is too unusual to be the default, and as 0.02 is so common elsewhere, I think it is hard to find the problem is the init because it's assumed it wouldn't be so different to begin with.
关闭于 2024-08-13 2 条评论