[Inference] Fix multibatch inference
<!-- Demo: https://github.com/PaddlePaddle/PaddleNLP/pull/26 -->
#### Before submitting
- [x] Lint code. If there are lint issues, please format the code first.
```shell
# Install and register `pre-commit` in the project folder
pip install pre-commit && pre-commit install
# Process previous code files separately
pre-commit run --file XXXX.py
```
- [ ] Add test cases into `tests` folder. If there are codecov issues, please add tests cases first.
### PR types
<!-- One of [ New features | Bug fixes | Function optimization | Performance optimization | Breaking changes | Others ] -->
Bug fixes
### PR changes
<!-- One of [ Models | APIs | Docs | Others ] -->
Others
### Description
<!-- Describe what this PR does -->
Fix multibatch infra for Qwen2
合并状态:已合并 合并于 2025-02-10 关闭于 2025-02-10 1 条评论