ITADN

Threshold compaction never fires for zero-usage providers: estimateContextTokens pure estimate discarded when lastUsageIndex === null

#8328Closedischindl 创建于 10 天前
untriaged
I
ischindlcommented
### What happened? For OpenAI-compatible providers whose streaming responses omit the final `usage` block (despite pi sending `stream_options: { include_usage: true }`), threshold auto-compaction **never fires**: the threshold case of `_checkCompaction` bails out when no assistant message carries non-zero usage — even though `estimateContextTokens` has already computed a pure chars/4 estimate over the whole message list, which the caller discards. ### Mechanism (verified 2026-08-18) Threshold case of `packages/coding-agent/src/core/agent-session.ts` (`_checkCompaction`): ```ts const directContextTokens = assistantMessage.usage ? calculateContextTokens(assistantMessage.usage) : 0; if (assistantMessage.stopReason === "error" || directContextTokens === 0) { const estimate = estimateContextTokens(this.agent.state.messages); if (estimate.lastUsageIndex === null) return false; // No usage data at all ... contextTokens = estimate.tokens; } ... if (shouldCompact(contextTokens, contextWindow, settings)) { ... } ``` Verified line references: - pi-coding-agent **0.84.1** dist: `dist/core/agent-session.js:1567-1572` (early return at `:1571-1572`) - **v0.84.2** tag (`914cf1472`): `agent-session.ts:2033` - **main** @ `59a71b235dad` (2026-08-18): `agent-session.ts:2102`; the pre-prompt call site `await this._checkCompaction(lastAssistant, false)` at `:1217` shares the same path, so the blind spot holds pre-prompt too `estimateContextTokens` (`packages/coding-agent/src/core/compaction/compaction.ts`; 0.84.1 dist `compaction.js:131-152`) already handles the no-usage case: it sums `estimateTokens` (chars/4) across all messages and returns `{ tokens: <estimate>, usageTokens: 0, trailingTokens: <estimate>, lastUsageIndex: null }`. The pure estimate is computed — then thrown away by the `return false`. ### Impact Any provider that omits usage (observed against an OpenAI-compatible gateway for a DeepSeek model, `contextWindow` 128,000) runs from ~87% of the window all the way to hard overflow with **zero threshold compaction events** (13 consecutive calls at 112K-124K input, threshold 111,616, no compaction). The overflow endgame (compact-and-retry itself overflowing a 128K model, sticky `_overflowRecoveryAttempted`) then wedges the session at 1-token output. Net effect: provider-agnostic overflow protection is unavailable for zero-usage providers, and every integrating application must ship its own pre-overflow gate. ### Suggested fix In the threshold case, fall back to the already-computed pure estimate instead of returning false when no usage data exists: ```ts if (estimate.lastUsageIndex === null) { contextTokens = estimate.tokens; // pure chars/4 fallback — deterministic, monotonic per message } ``` The estimate grows monotonically as messages are appended, so the `shouldCompact(contextTokens, contextWindow, settings)` threshold (default `reserveTokens: 16384`) becomes usable for zero-usage providers without changing any other code path. ### Related issues - #8192 (closed, not planned) — crash in `getLastAssistantUsageInfo` on unvalidated session files; different defect. - #8196 (closed, not planned) — compact-and-retry overflow endgame; different defect. - #6879 (open) — adjacent variant: compaction check cadence during long turns / models whose backend accepts beyond the configured window; not the zero-usage early return. ### Version 0.84.1 (incident); still present in v0.84.2 and main @ `59a71b235dad` (2026-08-18).
关闭于 10 天前 1 条评论