deno lsp: 126s → 20s first response, 30+ second -> 300ms go to definition, 10-11GB res -> 3 GB res (6 patches) -- includes comprehensive repro, simulation, and evidence

Once again, authored by Claude but I validated everything and am running the patched binary right now in my IDE with noticeably improved performance. I suspect the reason that this one was missed up until now is because deno doesn't ship large package sources in the filesystem. But if deno is going to support mixed repos (please do, thank you!) large uncommitted resources are common place in most languages. I proposed some fixes here with meaningful evidence (defect 1 and 2, note that 3 and 4 are hacks I didn't think very much about) about their efficacy, but to be honest I don't recommend that any of them be merged. My hope is that this (and probably future issues because deno is still eating ~11gigs of memory for unknown reasons and is still painfully but usably slow) will inspire the maintainers to revisit the LSP performance. Most notably the root cause in this issue could probably do with some tinder love and care 🙃. --- On a 53-project monorepo, one `textDocument/documentSymbol` took **126.7 s**. Enumerating every file in that workspace takes 221 ms, so almost all of that is work nothing asked for. **To be precise about the headline, because two of these numbers mean different things:** - The **three mergeable fixes** take it to **38.7 s** — a 3.3x improvement. - Adding a **fourth change, which is a diagnostic and should not be merged**, takes it to **20.0 s** — 6.3x. That fourth one skips work rather than removing redundancy; it is included as evidence of where the remaining time goes, not as a proposal. So: 3.3x from fixes you could take today, and a measured pointer at roughly 20 more seconds that appears to be redundant. Full write-up, reproductions, patch scripts and raw evidence: https://github.com/alita-moore/deno-lsp-performance Workspace shape: 73 npm workspace members, 5,781 TS files totalling **6.1 MB** of source, ~96k `.d.ts` in `node_modules`. ## Measurements One session, one probe, unpatched vs patched builds of `v2.9.5`, both built with `--features lsp-tracing`: | | unpatched | +M5+R1+lazy | +export gate | |---|---|---|---| | `documentSymbol` | 126,734 ms | 38,650 ms | **19,989 ms** | | `lsp.did_change_configuration` | 41,504 ms | **1,871 ms** | 2,123 ms | | `tsc.op.op_script_names` | 42,136 ms | 35,625 ms | **15,906 ms** | | peak RSS | 2,943 MB | 2,316 MB | 2,298 MB | Directory opens on the same workspace, unpatched vs the two enumeration fixes: **39,014 → 5,910** (−84.9%). ## Four defects **1 & 2 — two callers of the same line.** Both bottom out at `libs/config/glob/collector.rs:178`, the `fs_read_dir` inside `FileCollector::collect_file_patterns`: - **Workspace member-glob expansion** (`discovery.rs:898`), reached from `refresh_config_tree`. Expanding `"workspaces": ["packages/*"]` descends every matched member in full, pruning only `node_modules`, `.git` and one exact vendor path. A `.venv` inside a matched member is walked; the identical tree at the repository root is not — 2,587 opens vs 1, everything else held fixed. No user configuration reaches this walk at all. - **tsconfig root-set collection** (`compiler_options.rs:87` → `collect_specifiers`), reached from `refresh_compiler_options_resolver`. Same walk, and `collect_specifiers` is called once per distinct `FilePatterns`, so each member's tsconfig gets its own. The same mechanism is **unsafe at the first and safe at the second**, which is why they need different fixes. Honouring the ignore set during member expansion loses workspace members whose directory is gitignored (measured: 2 of 2 in one shape). During root-set collection it does not, because `exclude` governs seeds rather than reachability — an imported file still enters the graph, verified against the binary. Fixes: bound member-glob descent by whether the pattern can still match below the directory (the predicate, `can_match_under`, already exists and is already trusted by `split_by_base`); honour the ignore set for root-set collection. **3 — eager per-scope npm dependency resolution.** `ConfiguredDepResolutions` is built eagerly for every scope in `cli/lsp/resolver.rs`. With 73 members that is 73 constructions, most never used. Holding it in a `OnceLock` per scope takes `did_change_configuration` from **41,504 ms to 1,871 ms**. Related: #36662. **4 — redundant export-key enumeration.** Gating the `export_keys` enumeration behind an env var halves `documentSymbol` and takes `op_script_names` from 42 s to 16 s. **This is a diagnostic, not a proposed patch** — it skips the work rather than removing the redundancy, and should not be merged as-is. It is included as evidence that ~20 s there appears to be redundant, consistent with #36662. ## What a reader will assume, which is false - **tsconfig `include`/`exclude` do not affect defect 1 at all.** Different subsystem. 46 configurations were measured against it; none moves this cost. - **`walk_workspace` is not the problem.** It is capped at 1,000 entries (`language_server.rs:1065`) and accounted for 91 of 42,666 opens. - **Deleting `workspaces` from `package.json` "fixes" it** by skipping the expansion entirely. That is why this can look like an npm-workspaces problem rather than an enumeration one. Every claim above was located by capturing a call stack at each `opendir` and symbolising it against the binary, not by reading source. The tooling is in the repo and runs against any workspace. ## Still unexplained With all four patches the server reaches **10.6 GB resident in 72 seconds** on this workspace, up from ~8 GB unpatched, against 6.1 MB of source. RSS is flat once reached, so it is not a leak — it allocates once and holds. The reproduction never gets near this (peaks ~2.3 GB), so I have not identified the cause. Possibly the patches remove the work that was throttling allocation, but that is unverified.