`@mediapipe/tasks-genai@0.10.27` — `gemma-4-E4B-it-web.task` loads but decode output is numeric tokens, not text

### Have I written custom code (as opposed to using a stock example script provided in MediaPipe) No ### OS Platform and Distribution macOS 26.5 ### MediaPipe Tasks SDK version @mediapipe/tasks-genai@0.10.27 (latest). Also reproduced on 0.10.36-rc.20260518 (current nightly, published 2026-05-18) — byte-identical output. ### Task name (e.g. Image classification, Gesture recognition etc.) LLM Inference (tasks-genai) ### Programming Language and version (e.g. C++, Python, Java) JavaScript / TypeScript — Web (browser, WebGPU). Node 24.14.0 for the local dev server only; runtime is browser. ### Describe the actual behavior On `@mediapipe/tasks-genai@0.10.27`, the `litert-community/gemma-4-E4B-it-litert-lm` web variant (`gemma-4-E4B-it-web.task`, 2.83 GB) loads successfully and runs inference, but the streamed output is numeric tokens / digit sequences rather than detokenised text. The same code path on `gemma-4-E2B-it-web.task` (1.91 GB, same repo family) and `gemma3-1b-it-int4-web.task` (668 MB) produces correct text output. This looks like a tokenizer / detokenizer mismatch specific to 4 B-param Gemma in this version, not a corrupt download (file loads, inference completes, no errors thrown). Inference completes (95 streaming chunks delivered, no errors thrown), but the rendered output is numeric tokens — looks like raw vocab IDs rather than decoded text. Live sample, captured 2026-05-19 from <https://expo-litert-lm.pages.dev/>, GPU backend, prompt = anti-scam SMS classification ("Someone got a text message saying their bank account..."), `gemma-4-E4B-it-web.task` (2827 MB): ``` 1145161728942394020115200128942394020115200128942394020115923402011520012894239402011520012894239402011592340201152001289423940201152001289423940201159234020115200128942394020115200128942394020115923402011520012894239402011520012894239402011592340201152001289423940 ``` The output is 284 characters of digit runs with a few short repeating motifs (`894239402011520012`, `592340201152001`) — no whitespace, no punctuation, no detokenised text. Same shape on every retry; same shape on the CPU backend. Run metrics for the sample above: GPU backend, peak 464.1 MB JS heap, TTFT 382 ms, total 7152 ms, 95 chunks, 40 chars/s (≈ 9.9 tok/s). Earlier matrix runs on a different prompt landed in the same ballpark (GPU 16 chars/s peak 424 MB; CPU 10 chars/s peak 382 MB). ### Describe the expected behaviour Streamed text response in English, similar to what Gemma 4 E2B produces on this exact code path: > "Hi there! How can I help you today? ..." ### Standalone code/steps you may have used to try to get what you need ```html <input type="file" id="picker"> <button id="run">Run</button> <pre id="out"></pre> <script type="module"> import { FilesetResolver, LlmInference, } from 'https://cdn.jsdelivr.net/npm/@mediapipe/tasks-genai@0.10.27/+esm'; document.getElementById('run').onclick = async () => { const file = document.getElementById('picker').files[0]; const fileset = await FilesetResolver.forGenAiTasks( 'https://cdn.jsdelivr.net/npm/@mediapipe/tasks-genai@0.10.27/wasm' ); // Stream reader, not arrayBuffer — keeps heap ~10x lower on multi-GB files const reader = file.stream().getReader(); const inference = await LlmInference.createFromOptions(fileset, { baseOptions: { modelAssetBuffer: reader }, maxTokens: 256, }); inference.generateResponse('hi', (partial, done) => { document.getElementById('out').textContent += partial; }); }; </script> ``` ### Other info / Complete Logs ```shell ### Positive controls (same page, same code, smaller models work correctly) On the same page, same `LlmInference.createFromOptions` call, with `modelAssetBuffer` fed the same `ReadableStreamDefaultReader`: | Model | File size | Backend | Output | Decode | |---|---|---|---|---| | `gemma3-1b-it-int4-web.task` | 668 MB | GPU | ✅ clean English text | 468 chars/s | | `gemma-4-E2B-it-web.task` | 1.91 GB | GPU | ✅ clean English text | 234 chars/s | | `gemma-4-E4B-it-web.task` | **2.83 GB** | GPU | ⚠️ numeric gibberish | 16 chars/s | | `gemma-4-E4B-it-web.task` | 2.83 GB | CPU | ⚠️ numeric gibberish | 10 chars/s | So the failure is specific to the 4 B-parameter variant on this version, on both GPU and CPU backends. Not a heap issue (model loads, inference runs to completion). Not a magic-byte / routing issue (`.task` ZIP signature recognised, MediaPipe accepts the buffer). Looks like tokenizer / vocab / detokenizer mismatch. ### Reproduced on the latest nightly (`0.10.36-rc.20260518`) To rule out "this is already fixed in nightly," I re-ran the same page with the only change being the CDN URL bumped to `@mediapipe/tasks-genai@0.10.36-rc.20260518` (current `nightly` dist-tag, published 2026-05-18 — the same day as Google's LiteRT-LM v0.12.0 release). Same model file, same prompt, same browser: | Version | Peak heap | TTFT | total | chars | chars/s | chunks | Output | |---|---|---|---|---|---|---|---| | `@mediapipe/tasks-genai@0.10.27` (latest) GPU | 464.1 MB | 382 ms | 7152 ms | 284 | 40 (≈ 9.9 tok/s) | 95 | digit-run gibberish | | `@mediapipe/tasks-genai@0.10.36-rc.20260518` (nightly) GPU | 393.5 MB | 752 ms | 7441 ms | 284 | 38 (≈ 9.5 tok/s) | 95 | **byte-identical** digit-run gibberish | Output string is character-for-character identical across the two versions. So this isn't a stale `latest` — the most recent code Google ships still reproduces it. ### Context This came up while extending [`expo-litert-lm`](https://github.com/helenkwok/expo-litert-lm) (an Expo Modules wrapper for LiteRT-LM on React Native / Expo) to a web target. The `.task` path is the fast web path on smaller Gemmas (1 B and E2B both run cleanly here), so the E4B-specific failure stands out. Live page reproducing this: <https://expo-litert-lm.pages.dev/> (pick the E4B `.task` file). Branch: <https://github.com/helenkwok/expo-litert-lm/tree/feat/litert-v0.12> Happy to test a follow-up `.task` build or a different engine option if you can suggest one. ```