[Bug] Segfault encountered in training draft model
### Checklist
- [x] 1. I have searched related issues but cannot get the expected help.
- [x] 2. The bug has not been fixed in the latest version.
- [x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
- [x] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/SpecForge/discussions/new/choose Otherwise, it will be closed.
- [x] 5. Please use English, otherwise it will be closed.
### Describe the bug
After setting up the environment for the latest version and running the training script, the following errors are triggered immediately. When reverting to several versions from December, the same errors occur the moment epoch 0 starts. Additionally, the official environment does not seem to be directly compatible with the code, requiring numerous modifications.
!!!!!!! Segfault encountered !!!!!!!
File "<unknown>", line 0, in _PyEval_EvalFrameDefault
File "<unknown>", line 0, in _PyFunction_Vectorcall
File "<unknown>", line 0, in _PyObject_FastCallDictTstate
File "<unknown>", line 0, in _PyObject_MakeTpCall
File "<unknown>", line 0, in _PyEval_EvalFrameDefault
File "<unknown>", line 0, in PyObject_Call
File "<unknown>", line 0, in _PyEval_EvalFrameDefault
File "<unknown>", line 0, in _PyFunction_Vectorcall
File "<unknown>", line 0, in PyObject_Call
File "<unknown>", line 0, in _PyEval_EvalFrameDefault
File "<unknown>", line 0, in _PyFunction_Vectorcall
File "<unknown>", line 0, in PyObject_Call
File "<unknown>", line 0, in _PyEval_EvalFrameDefault
File "<unknown>", line 0, in PyEval_EvalCode
File "<unknown>", line 0, in _PyRun_SimpleFileObject
File "<unknown>", line 0, in _PyRun_AnyFileObject
File "<unknown>", line 0, in Py_RunMain
File "<unknown>", line 0, in Py_BytesMain
File "<unknown>", line 0, in _start
File "<unknown>", line 0, in 0xffffffffffffffff
### Reproduction
bash examples/run_qwen3_8b_eagle3_online.sh
use Qwen/Qwen3-8B
### Environment
uv pip list
(SpecForge-main) (base) idc@HGX-082:/Data2/jj/SpecForge-main$ uv pip list
Package Version Editable project location
------------------------- ------------- -------------------------
accelerate 1.13.0
aiohappyeyeballs 2.6.1
aiohttp 3.13.5
aiosignal 1.4.0
airportsdata 20260315
annotated-doc 0.0.4
annotated-types 0.7.0
anthropic 0.95.0
anyio 4.13.0
apache-tvm-ffi 0.1.10
asttokens 3.0.1
attrs 26.1.0
av 17.0.0
blobfile 3.0.0
build 1.4.3
certifi 2026.2.25
cffi 2.0.0
cfgv 3.5.0
charset-normalizer 3.4.7
click 8.3.2
cloudpickle 3.1.2
compressed-tensors 0.15.0.1
cuda-bindings 13.2.0
cuda-pathfinder 1.5.3
cuda-python 13.2.0
datasets 4.8.4
decorator 5.2.1
decord2 3.3.0
dill 0.4.1
diskcache 5.6.3
distlib 0.4.0
distro 1.9.0
docstring-parser 0.18.0
einops 0.8.2
executing 2.2.1
fastapi 0.135.3
filelock 3.28.0
flashinfer-cubin 0.5.3
flashinfer-python 0.5.3
frozenlist 1.8.0
fsspec 2026.3.0
gguf 0.18.0
gitdb 4.0.12
gitpython 3.1.46
grpcio 1.75.1
grpcio-health-checking 1.75.1
grpcio-reflection 1.75.1
grpcio-tools 1.75.1
h11 0.16.0
hf-transfer 0.1.9
hf-xet 1.4.3
httpcore 1.0.9
httpx 0.28.1
huggingface-hub 1.11.0
identify 2.6.18
idna 3.11
interegular 0.3.3
ipython 9.10.1
ipython-pygments-lexers 1.1.1
jedi 0.19.2
jinja2 3.1.6
jiter 0.14.0
jsonschema 4.26.0
jsonschema-specifications 2025.9.1
lark 1.3.1
llguidance 0.7.30
loguru 0.7.3
lxml 6.0.4
markdown-it-py 4.0.0
markupsafe 3.0.3
matplotlib-inline 0.2.1
mdurl 0.1.2
modelscope 1.35.4
mpmath 1.3.0
msgspec 0.21.1
multidict 6.7.1
multiprocess 0.70.19
nest-asyncio 1.6.0
networkx 3.6.1
ninja 1.13.0
nodeenv 1.10.0
numpy 2.4.4
nvidia-cublas-cu12 12.8.4.1
nvidia-cuda-cupti-cu12 12.8.90
nvidia-cuda-nvrtc-cu12 12.8.93
nvidia-cuda-runtime-cu12 12.8.90
nvidia-cudnn-cu12 9.10.2.21
nvidia-cudnn-frontend 1.22.1
nvidia-cufft-cu12 11.3.3.83
nvidia-cufile-cu12 1.13.1.3
nvidia-curand-cu12 10.3.9.90
nvidia-cusolver-cu12 11.7.3.90
nvidia-cusparse-cu12 12.5.8.93
nvidia-cusparselt-cu12 0.7.1
nvidia-cutlass-dsl 4.2.1
nvidia-ml-py 13.595.45
nvidia-nccl-cu12 2.27.5
nvidia-nvjitlink-cu12 12.8.93
nvidia-nvshmem-cu12 3.3.20
nvidia-nvtx-cu12 12.8.90
openai 2.6.1
openai-harmony 0.0.4
orjson 3.11.8
outlines 0.1.11
outlines-core 0.1.26
packaging 26.1
pandas 3.0.2
parso 0.8.6
partial-json-parser 0.2.1.1.post7
pexpect 4.9.0
pillow 12.2.0
platformdirs 4.9.6
pre-commit 4.5.1
prometheus-client 0.25.0
prompt-toolkit 3.0.52
propcache 0.4.1
protobuf 6.33.6
psutil 7.2.2
ptyprocess 0.7.0
pure-eval 0.2.3
py-spy 0.4.1
pyarrow 23.0.1
pybase64 1.4.3
pycountry 26.2.16
pycparser 3.0
pycryptodomex 3.23.0
pydantic 2.13.1
pydantic-core 2.46.1
pygments 2.20.0
pyproject-hooks 1.2.0
python-dateutil 2.9.0.post0
python-discovery 1.2.2
python-multipart 0.0.26
pyyaml 6.0.3
pyzmq 27.1.0
qwen-vl-utils 0.0.11
referencing 0.37.0
regex 2026.4.4
requests 2.33.1
rich 15.0.0
rpds-py 0.30.0
safetensors 0.7.0
scipy 1.17.1
sentencepiece 0.2.1
sentry-sdk 2.58.0
setproctitle 1.3.7
setuptools 82.0.1
sgl-kernel 0.3.18.post2
sglang 0.5.6
shellingham 1.5.4
six 1.17.0
smmap 5.0.3
sniffio 1.3.1
soundfile 0.13.1
specforge 0.1.1
stack-data 0.6.3
starlette 1.0.0
sympy 1.14.0
tabulate 0.10.0
tiktoken 0.12.0
timm 1.0.16
tokenizers 0.22.2
torch 2.9.1
torch-memory-saver 0.0.9
torchao 0.9.0
torchaudio 2.9.1
torchcodec 0.7.0
torchvision 0.24.1
tqdm 4.67.3
traitlets 5.14.3
transformers 5.5.4
triton 3.5.1
typer 0.24.1
typing-extensions 4.15.0
typing-inspection 0.4.2
urllib3 2.6.3
uvicorn 0.44.0
uvloop 0.22.1
virtualenv 21.2.4
wandb 0.26.0
wcwidth 0.6.0
xgrammar 0.1.27
xxhash 3.6.0
yarl 1.23.0
nvidia-smi
Fri Apr 17 11:09:57 2026
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.161.08 Driver Version: 535.161.08 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
0 条评论