[bug] moon hangs indefinitely when ≥2 projects share a managed toolchain

>Apologies for the duplicate issues. Also, this is heavily mediated by Claude Code as I'm not a rust dev. I hope it's useful. **Describe the bug** CacheEngine::create_lock calls flock(LOCK_EX) (via starbase_utils::fs::lock_file) directly on a tokio worker thread, without spawn_blocking. The lock file name is a hash of the toolchain ID, which is shared across every project using the same toolchain. Then, two tokio tasks race to acquire it, and on linux the flock is per-OFD not per process, so the second flock blocks the OS thread. Tokio ends up single-threaded and parked in `futex_wait`, so the first task can't wake to release its lock. This happens for any two concurrent actions for projects on the same toolchain (e.g. `SetupEnvironment`, `InstallDependencies`). **Steps to reproduce** 1. On linux Create a workspace with 2+ projects sharing a managed toolchain (e.g. `unstable_python` or `javascript/node` in `.moon/toolchains.yml`) 2. Run `moon ci` or `moon run :install` from a cold cache on Linux 3. Observe moon printing initial action progress, then hanging indefinitely with no log output and no child processes I can reliably reproduce this using a linux docker container running on MacOS. We have 27 projects split across two toolchains and can pretty consistently reproduce this. Note: MOON_TOOLCHAIN_FORCE_GLOBALS=true likely causes us to hit this more than we would without it. When moon bootstraps proto toolchains, the concurrency pattern looks different. I've included an example self-contained reproducer at the end of the issue. **Expected behavior** Actions acquire the lock and proceed. At most one action per toolchain runs at a time, the rest queue and execute in turn. **Log output** `/proc/<pid>` snapshot taken during an 18-minute hang: ~~~shell State: S (sleeping) Wchan: futex_wait fd 46 → .moon/cache/locks/setup-environment-6664070644759123416.lock ← task 1 holds LOCK_EX fd 48 → .moon/cache/locks/setup-environment-6664070644759123416.lock ← task 2 blocked in flock() ~~~ **Environment** - moon 2.0.4 – 2.2.4 - Linux only (macOS uses per-process flock semantics and is not affected) **Fix** I've prototyped a fix with Claude Code here and I'm happy to turn it into a PR if it's useful: https://github.com/palexander/moon/commit/323768377ef1def45e2a74716f407fb31d346190 Running with this as our build in docker resolves the deadlocks. **Reproducer** Hangs until killed on Linux: ~~~rust // Cargo.toml: libc = "0.2", tokio = { version = "1", features = ["full"] } // [dev-dependencies]: tempfile = "3" fn raw_flock_ex(path: &Path) -> File { let f = OpenOptions::new().create(true).write(true).open(path).unwrap(); unsafe { libc::flock(f.as_raw_fd(), libc::LOCK_EX) }; f } #[tokio::test(flavor = "current_thread")] async fn deadlock() { let path = tempdir().unwrap().path().join("test.lock"); let p = path.clone(); tokio::spawn(async move { let _f = raw_flock_ex(&p); // fd1: acquires LOCK_EX tokio::time::sleep(Duration::from_millis(500)).await; // yields }); tokio::spawn(async move { let _f = raw_flock_ex(&path); // fd2: flock(LOCK_EX) → parks the thread }); tokio::time::sleep(Duration::from_secs(60)).await; // never reached } ~~~