ITADN

--incremental: after a pnpm dependency version change, the cached run is slower than a cold run and most of the time is unattributed

#63731Openmushan0x0 创建于 15 天前
Needs InvestigationFix Available
M
mushan0x0commented
# `--incremental`: after a pnpm dependency version change, the cached run is slower than a cold run, and most of the time is not attributed to any reported phase ## Summary When `.tsbuildinfo` is invalidated by a **dependency path change** (pnpm moves `node_modules/x` to a new `.pnpm/x@<version>/…` target on every version bump), `tsc --noEmit` does *more* work than a cold build with no cache at all — and the extra time is not attributed to parse, bind, or check. Minimal repro (80 files, zod): the post-change run takes **2.89s vs 1.63s cold**, with `Check time: 0.07s`. Roughly 2.5s of a 2.89s run is unaccounted. On a real 8773-file application the same shape becomes severe: **114.6M instantiations vs 7.7M cold (14.9x a full build)**, **202.4s vs 24.7s (8.2x slower than no cache)**, **5.45 GB peak RSS vs 2.54 GB**, and it **OOMs at node's default heap**. Editing source files does *not* trigger this — invalidating a module that 75 files transitively import is cheaper than a cold build, as expected. Only the dependency-path case regresses. ## Minimal reproduction The trigger is **pnpm's content-addressed layout**, not "a dependency changed" as such. With npm, `npm i zod@<other>` replaces `node_modules/zod` in place and `tsc` does not invalidate at all (it takes the fast path, `Instantiations: 0`). With pnpm, `node_modules/zod` is a symlink into `.pnpm/zod@<version>/node_modules/zod`, so a version change moves every resolved path — and that is what perturbs the cache. ```bash mkdir repro && cd repro printf '{ "name": "repro", "private": true, "type": "module" }\n' > package.json pnpm add zod@3.25.76 typescript@5.9.3 # tsconfig.json: { "compilerOptions": { "strict": true, "noEmit": true, # "incremental": true, "skipLibCheck": true, "moduleResolution": "bundler", # "module": "esnext", "target": "ESNext" }, "include": ["src/**/*.ts"] } # src/shared.ts re-exports zod + a generic `make<B, R>({ body, response, handler })` # src/route0..79.ts each build a nested z.object and call make() (see gist/attachment) ./node_modules/.bin/tsc --noEmit --extendedDiagnostics -p tsconfig.json # A: cold ./node_modules/.bin/tsc --noEmit --extendedDiagnostics -p tsconfig.json # B: fresh cache pnpm add zod@3.24.4 # symlink moves ./node_modules/.bin/tsc --noEmit --extendedDiagnostics -p tsconfig.json # D ``` | run | Types | Instantiations | Check time | Total time | | --- | ---: | ---: | ---: | ---: | | A cold | 84,190 | 368,421 | — | 1.63s | | B fresh cache | 85 | 0 | — | 0.23s | | D after pnpm version change | 84,190 | 378,157 | **0.07s** | **2.89s** | Two things to note in D, both of which also hold in the large project below: 1. **The incremental run is slower than the cold run** (2.89s vs 1.63s). The cache is not merely useless here, it is a net cost. 2. **`Check time` is 0.07s while `Total time` is 2.89s.** Parse and bind are ~0.2s. Roughly 2.5s — the large majority of the run — is not attributed to any phase `--extendedDiagnostics` reports. ## The same signature at scale In a real Next.js application (8773 files, ~136k lines of TS, ~672k lines of definitions; zod + drizzle-orm + next-rest-framework generic chains), the same trigger produces a far more severe version of the same shape: | run | state | Types | Instantiations | Check time | Total time | peak RSS | | --- | --- | ---: | ---: | ---: | ---: | ---: | | A | no `.tsbuildinfo` (cold) | 964,787 | 7,687,382 | 18.93s | 24.72s | 2.54 GB | | B | fresh `.tsbuildinfo` | 86 | 0 | — | 5.18s | 0.99 GB | | C | fresh, then one widely-imported **source** file edited (75 direct dependents) | 627,108 | 5,095,840 | 8.52s | 13.66s | 1.80 GB | | D | fresh, then **pnpm dependency version changed** | **3,965,709** | **114,589,573** | **9.35s** | **202.39s** | **5.45 GB** | - D does **14.9x the instantiations of a full cold build**, and is **8.2x slower than having no cache at all**. - `Check time` in D (9.35s) is *lower* than in the cold build (18.93s), yet total time is 202s. ~193s is unaccounted. - At node's default heap D does not complete: it OOMs after 254s at 4.4 GB. The figures above required `--max-old-space-size=8192`. - Run C is the control: invalidating a module that 75 files import transitively is **cheaper** than a cold build (13.66s / 1.80 GB). Incremental behaves correctly for source edits. Only the dependency-path case regresses. I could not reproduce the instantiation blow-up at minimal scale — it appears to need the real project's type complexity. The minimal repro does reproduce the two structural facts: the cached run is slower than cold, and the time is not attributed to parse/bind/check. ## Why this matters The failure mode is silent and counter-intuitive: enabling `incremental` makes the *common CI / post-`npm install` case* an order of magnitude worse than disabling it, and can push a project that type-checks fine in 2.5 GB over the default heap limit. Teams hit it as "our type-check randomly OOMs" and, not suspecting the cache, raise `--max-old-space-size` — which masks it, since the run then merely takes 200s instead of 25s. There appears to be no heuristic along the lines of "the cached state is mostly invalid, discard it and do a cold check", which is what the numbers say should happen here. ## Questions 1. What phase accounts for the ~193s that is neither parse, bind, nor check? 2. Why do Types/Instantiations exceed a full cold build by 4x/15x when the amount of code to check is, at most, all of it? 3. Would it be reasonable for `tsc` to drop the cached program when the module-resolution inputs (installed package versions / resolved paths) have changed, rather than reconciling it? ## Environment - TypeScript: 5.9.3 - Node: v22.16.0 - OS: macOS (darwin arm64) - `tsconfig`: `incremental: true`, `skipLibCheck: true`, `strict: true`, `moduleResolution: "bundler"`, `noEmit: true`
1 条评论