ITADN

process_lifetime PDEATHSIG guard kills every child when Studio runs as PID 1

#6756OpenQuanqued 创建于 2026-06-30
Q
Quanquedcommented
1. Did you update? `pip install --upgrade --force-reinstall --no-cache-dir unsloth unsloth_zoo` — Yes, this is current code. The guard is in the Studio backend (`studio/backend/utils/process_lifetime.py`), installed via `unsloth.ai/install.sh` on 2026-06-28. 2. Colab/Kaggle/Local or Cloud? — Local, in Docker. NVIDIA DGX Spark (GB10, aarch64), Ubuntu 24.04 base. 3. Number GPUs — 1. 4. Which notebook? Please link! — None; this is the Studio backend, not a training notebook. It reproduces on startup with no model loaded. 5. Which Unsloth version, TRL version, transformers version and PyTorch version? — unsloth 2026.6.9, unsloth_zoo 2026.6.7, trl 0.23.1, transformers 4.57.6, torch 2.10.0+cu130. 6. Which trainer? `SFTTrainer`, `GRPOTrainer` etc — N/A; backend process management, not a trainer. --- If you run Studio in a container without an init process, the backend itself is PID 1, and every subprocess it spawns (the llama.cpp server, training jobs) gets killed the instant it starts. It comes down to the parent-death guard in `_pdeathsig_preexec` (`process_lifetime.py`): ```python def _pdeathsig_preexec() -> None: try: import ctypes ctypes.CDLL("libc.so.6", use_errno=True).prctl(_PR_SET_PDEATHSIG, signal.SIGTERM) if os.getppid() == 1: os._exit(1) except Exception: pass ``` The `getppid() == 1` check is there to close the race where the parent dies between `fork` and the `prctl` call — in that case the child gets reparented to init and should bail. The problem is that it treats "reparented to PID 1" and "my real parent is PID 1" as the same thing. When the Studio process is legitimately PID 1, every healthy child has `getppid() == 1` from birth, so the guard fires on all of them and they exit immediately via `os._exit(1)`. This hits any container that doesn't run an init as PID 1, which is the default for a plain `CMD`/`ENTRYPOINT`. **Workaround:** run an init so Studio isn't PID 1 — `docker run --init`, `init: true` in Compose, or bundling `tini`. With a real init in front, `getppid()` returns the init's PID instead of 1 and the guard behaves. This fixed it for us. **Suggested fix:** capture the actual parent PID before the fork and compare against that, rather than the literal `1` — or skip the check when the process is itself PID 1 (if Studio-as-PID-1 dies, the container is gone anyway, so there's nothing to guard against). PDEATHSIG already covers the normal parent-death case either way.
0 条评论