Sync v2 stream has no backpressure — `send()` ignores `response.write()`'s return value, so a large `AssetOcrV1` backlog OOMs the api worker (V8 heap FATAL) and crash-loops it with exit code 0

### The bug `/api/sync/stream`'s send helper writes serialized rows to the HTTP response without checking `response.write()`'s return value. When the client drains the socket slower than Postgres produces rows, `write()` starts returning `false` — but the `for await` loop keeps running at cursor speed, so every untransmitted row accumulates in process memory until the api worker hits V8's default heap ceiling (~2240 MB without `NODE_OPTIONS`) and dies: ``` FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory ``` `start.sh` then exits **0**, so `restart: always` restarts the container in ~250 ms with `OOMKilled=false` — from the outside it looks like a healthy container that quietly loses every long sync session, and it defeats restart/OOM monitoring (no non-zero exit, no cgroup OOM kill, sub-second gap). The sync checkpoint makes each session resume, so the loop repeats every time a client with a large backlog connects. The trigger type in our case is `AssetOcrV1` (new in v3; not present in v2.7.5), simply because it is the biggest payload: our instance has 4,320,180 `asset_ocr` rows ≈ 1,137 MB of serialized JSON in a single stream. The missing-backpressure pattern itself is shared by the other sync types — `AssetOcrV1` is just the first one big enough to hit the ceiling. Your Sync v2 blog post says the design goal was "end-to-end streaming … avoiding serializing huge chunks of JSON in memory" — the design does stream end-to-end (the Kysely cursor genuinely paginates), but the last hop never asks the socket whether it is ready, which un-does the memory-boundedness whenever the client is slower than the database. ### Environment - immich server **v3.0.2** (also byte-identical in `main` as of 2026-07-14: the send helper still discards `write()`'s return value) - Docker Compose deployment, default `NODE_OPTIONS` (none) - Postgres on the same host → the producer is much faster than any real client ### Evidence (measured, not inferred) - Sampling the resident worker's memory during the ratchet shows the bytes are post-serialize/pre-transmit sync frames: resident arenas are full of `{"type":"AssetOcrV1",…,"ack":"AssetOcrV1|<uuid>"}` strings — and `ack` is not a DB column; it is synthesized in JS at send time, so the accumulation can only be the unflushed write backlog. - One 22-second client session ratcheted the worker 2.67 GB → 7.33 GB, which persisted after the client disconnected; the worker died at the heap ceiling ~an hour later. 4 crash-loops in one afternoon, 24 h of zero sync progress. - Isolated reproduction (same code, same driver, same table, throwaway container): pushing all 4.32 M rows through the unmodified send path with a slow reader grows the socket write queue to **1,571 MB and climbing**; the identical run with a drain-await added holds the write queue at **0.0 MB** (worker RSS 118 MB). - With a drain-await deployed on our instance, a real client streamed **+414,279 rows in a 38-minute session with zero restarts and zero V8 FATALs** (peak 2,088 MiB); before the fix the same workload produced the 7.33 GiB ratchet and worker death. - Raising the container memory cap does not help (we measured before understanding the mechanism): the accumulation lives largely outside the V8 heap (allocator arenas ~3× heap in our samples), and the V8 ceiling is independent of the cgroup limit anyway. ### The fix needs one subtlety: the disconnect path The obvious fix — `if (!response.write(x)) await once(response, 'drain')` — **hangs on client disconnect** (measured): no `'drain'` ever fires, the generator parks at its `yield`, so Kysely's `finally { release() }` never runs and the reserved Postgres connection leaks for the life of the process. That trades a self-healing crash for a permanent connection leak. The await must be abortable on `'close'`/`destroyed`, and the handler must **stop the whole handler** on disconnect (not just break one loop — the deletes loop otherwise falls through and streams upserts to the dead socket). ### Reference: the hotfix we run locally We bind-mount a patched compiled `dist/services/sync.service.js` over v3.0.2 (2 behavioral call sites in `syncAssetOcrV1` only; all other `send()` call sites untouched). Shape: ```js const sendWithBackpressure = async (response, item) => { if (response.write(serialize(item))) { return true; } if (response.destroyed || response.writableEnded) { return false; } const ac = new AbortController(); const onClose = () => ac.abort(); response.once('close', onClose); try { await once(response, 'drain', { signal: ac.signal }); return true; } catch { return false; // disconnected while waiting — caller must stop } finally { response.off('close', onClose); } }; // call sites (deletes + upserts loops): if (!(await sendWithBackpressure(response, { type, ids, data }))) { return; // return from the handler, not break — see above } ``` We verified `return` unwinds the generator and releases the Postgres connection (1 release / 0 hangs across break/throw/return variants; the bare drain-await variant is the only one that leaks). A proper upstream fix presumably wants this in `sync.service.ts` for **all** sync types, with the send helper itself made async. Happy to provide any of the measurements in more detail. *Disclosure: all pre-fix evidence above was collected on an unpatched v3.0.2; the instance has run the bind-mount hotfix since 2026-07-14.*