[Bug] dflash cross block data leak
### Checklist
- [x] 1. I have searched related issues but cannot get the expected help.
- [x] 2. The bug has not been fixed in the latest version.
- [x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
- [x] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/SpecForge/discussions/new/choose Otherwise, it will be closed.
- [x] 5. Please use English, otherwise it will be closed.
### Describe the bug
I tried training a dflash model using the script in this repo, and noticed that performance is much better when `num-anchors` is small. When `num-anchors` is large, the training loss/acc look abnormally good (loss unusually low / acc unusually high compared to expect), but all the evaluation results are much worse than reported in its paper.
So I dug into the code and found the `random-anchor` path: cross-block context visibility is determined purely by the `block index` (`k_ctx < q_block_id`). When the anchor spacing becomes `smaller than `block_size`, blocks can overlap. This overlap allows a later block to see future target hidden states from the previous block. So the larger `num-anchors` making more overlaps and training loss/acc look unusually better.
### Reproduction
same as https://github.com/sgl-project/SpecForge/issues/465
### Environment
same as https://github.com/sgl-project/SpecForge/issues/465
0 条评论