[Bug]: Responses API — reasoning items (incl. encrypted_content) are silently dropped for providers without a native ResponsesAPIConfig (anthropic, bedrock/converse, non-gpt databricks)
llm translationclaude code
### What happened?
When a **Responses API** request carrying `reasoning` items is routed to a provider that has **no native `ResponsesAPIConfig`** (`anthropic/`, `bedrock/converse/`, non-`gpt` `databricks/`), LiteLLM converts the request to that provider's chat dialect and **silently drops every reasoning item**, including `encrypted_content`.
Tool plumbing survives the conversion correctly — this is specifically about reasoning state.
The practical effect is that a reasoning model loses its reasoning across every tool-call turn and re-derives it from scratch. That is the exact penalty the Responses API exists to avoid ([OpenAI measures it at +5% on TAUBench and 40–80% better cache utilization](https://developers.openai.com/blog/responses-api)). Because nothing errors, a caller migrating to `/v1/responses` for reasoning persistence gets none of it on these providers and has no signal that anything was lost.
Interestingly, LiteLLM *does* translate the reasoning **effort knob** on this path — it emits `thinking: {"type": "enabled", "budget_tokens": 2048}` (or `additionalModelRequestFields.thinking` for Bedrock). So the request tells Claude to think fresh while discarding the prior thinking. That combination is arguably worse than dropping both, because it spends tokens re-deriving reasoning that was already paid for.
### Relevant log output / evidence
Measured by pointing `api_base` at a local HTTP server that records the request body. Config:
```yaml
model_list:
- model_name: p-anthropic
litellm_params:
model: anthropic/claude-sonnet-4-5-20250929
api_key: sk-ant-fake
api_base: http://127.0.0.1:9099
- model_name: p-bedrock
litellm_params:
model: bedrock/converse/us.anthropic.claude-sonnet-4-5-20250929-v1:0
aws_access_key_id: AKIAFAKEFAKEFAKEFAKE
aws_secret_access_key: fakefakefakefakefakefakefakefakefakefake
aws_region_name: us-east-1
aws_bedrock_runtime_endpoint: http://127.0.0.1:9099
general_settings:
master_key: sk-probe-local
```
Request to `POST /v1/responses`:
```json
{
"model": "p-anthropic",
"reasoning": {"effort": "medium"},
"input": [
{"role": "user", "content": "Check CWE-79 then CWE-89 and summarize."},
{"type": "reasoning", "id": "rs_001",
"summary": [{"type": "summary_text", "text": "First I should look up CWE-79."}],
"encrypted_content": "ENCRYPTED_BLOB_ONE"},
{"type": "function_call", "call_id": "call_1", "name": "lookup", "arguments": "{\"cwe\":\"79\"}"},
{"type": "function_call_output", "call_id": "call_1", "output": "XSS details"},
{"type": "reasoning", "id": "rs_002",
"summary": [{"type": "summary_text", "text": "Now CWE-89."}],
"encrypted_content": "ENCRYPTED_BLOB_TWO"},
{"type": "function_call", "call_id": "call_2", "name": "lookup", "arguments": "{\"cwe\":\"89\"}"},
{"type": "function_call_output", "call_id": "call_2", "output": "SQLi details"}
],
"tools": [{"type": "function", "name": "lookup", "description": "Look up a CWE",
"parameters": {"type": "object", "properties": {"cwe": {"type": "string"}},
"required": ["cwe"]}}]
}
```
What each provider actually receives upstream, counting how many of the two `encrypted_content` blobs survive:
| `litellm_params.model` | Upstream route | Reasoning blobs surviving | Tool pairs |
| --- | --- | --- | --- |
| `openai/gpt-5.4-mini` | `POST /responses` | **2 / 2** | intact |
| `azure/gpt-5.4-mini` | `POST /openai/responses?api-version=…` | **2 / 2** | intact |
| `databricks/pixee-gpt54-mini` | `POST /responses` | **2 / 2** | intact |
| `anthropic/claude-sonnet-4-5-…` | `POST /v1/messages` | **0 / 2** | intact |
| `bedrock/converse/us.anthropic.claude-sonnet-4-5-…` | `POST /model/{id}/converse` | **0 / 2** | intact |
| `databricks/claude-sonnet-4-5` (no `gpt` in name) | `POST /chat/completions` | **0 / 2** | intact |
The reasoning `summary` text is dropped along with `encrypted_content`. Identical results on **v1.86.1** and **v1.95.0**.
### Root cause
`ProviderConfigManager.get_provider_responses_api_config` (`litellm/utils.py`, ~L8214) has no entry for `ANTHROPIC` or for `bedrock/converse`, so those requests take the chat-conversion fallback, and that conversion has no representation for a reasoning item.
The mapping exists, but only in the opposite direction: `LiteLLMAnthropicToResponsesAPIAdapter` (`litellm/llms/anthropic/experimental_pass_through/responses_adapters/transformation.py`) implements `translate_thinking_to_reasoning` for the Anthropic-in → Responses-out path. There is no Responses-in → Anthropic-out counterpart that replays a `reasoning` item as a `thinking` block with its `signature`.
### Suggested fix
Replay reasoning items as Anthropic `thinking` / Bedrock `reasoningContent` blocks, preserving the signature, rather than dropping them. Anthropic's extended thinking already has the matching concept, and pydantic-ai's native `AnthropicModel` does exactly this round-trip today (`BetaThinkingBlockParam` with `signature`), so there is a working reference for the shape.
If a full mapping is not desirable, **failing loudly would be better than dropping silently** — today there is no way for a caller to detect the loss.
Two related notes:
1. **Please consider capability-driven dispatch rather than model-name substrings.** The Databricks branch still gates on `if model and "gpt" in model.lower()`, so renaming a model silently changes the wire format and the fidelity of the request. `BEDROCK_MANTLE` in the same function already does this the better way, and the comment there states the principle explicitly: *"Both decisions are data-driven from the model's price-map entry, with no model-name logic."* Extending `mantle_supports_responses`-style capability lookup to the other providers would remove a whole class of surprise.
2. This is adjacent to #35677, which is a different symptom of the same hardcoded-model-name pattern on the Anthropic Responses path.
### Are you a ML Ops Team?
No
### What LiteLLM version are you on?
v1.86.1 and v1.95.0 (both reproduce)
---
*(Filed on Johnathan Gilday's behalf by Claude Code.)*
0 条评论