ITADN

XAI :livekit-plugins-xai: RealtimeSession treats in-progress input transcriptions (status="in_progress") as final, duplicating user conversation items

#6271Openareebkhan-tech 创建于 2026-06-29
bug
A
areebkhan-techcommented
### Bug Description xAI's realtime API streams partial input transcriptions through the `conversation.item.input_audio_transcription.completed` channel with `status: "in_progress"`, and sends the final with `status: "completed"`. (OpenAI, by contrast, streams partials via `...transcription.delta` and sends `...completed` only once.) The xAI RealtimeSession overrides this handler but still delegates to the OpenAI base implementation, which ignores `status` and always emits `InputTranscriptionCompleted(is_final=True)`: # livekit/plugins/xai/realtime/realtime_model.py def _handle_conversion_item_input_audio_transcription_completed(self, event): if remote_item := self._remote_chat_ctx.get(event.item_id): if remote_item.item.type == "message" and remote_item.item.text_content == event.transcript: remote_item.item.content.clear() super()._handle_conversion_item_input_audio_transcription_completed(event) # emits is_final=True So every in-progress partial is surfaced as a *final* transcription. Downstream, AgentActivity finalizes a user ChatMessage and emits `conversation_item_added` on each `is_final=True`, so one user utterance produces many `conversation_item_added` (and `user_input_transcribed(is_final=True)`) events with the same item_id and a growing transcript. Observed (grok-voice realtime), single utterance: completed status=in_progress "What is" completed status=in_progress "What is my name? What is your" completed status=in_progress "What is my name? What is your Wi-Fi?" completed status=completed "What is my name? What is your Wi-Fi?" -> 4 user conversation items emitted instead of 1. This is xAI-specific (xAI's status-based streaming), so the fix belongs in the xAI plugin's RealtimeSession, not in the shared OpenAI base handler. ### Expected Behavior The xAI RealtimeSession should honor xAI's `status` field: an "in_progress" transcription is an interim and should be emitted as is_final=False (like a delta), so it does not finalize a conversation item; only status="completed" should yield is_final=True (and the single final conversation item). Suggested handling in the xAI override (its own implementation for the realtime session, without changing OpenAI base behavior): def _handle_conversion_item_input_audio_transcription_completed(self, event): # xAI sends interim transcripts as "completed" with status="in_progress" if getattr(event, "status", None) == "in_progress": self.emit( "input_audio_transcription_completed", llm.InputTranscriptionCompleted( item_id=event.item_id, transcript=event.transcript, is_final=False, ), ) return # status == "completed" (or absent): treat as final via base handling if remote_item := self._remote_chat_ctx.get(event.item_id): if remote_item.item.type == "message" and remote_item.item.text_content == event.transcript: remote_item.item.content.clear() super()._handle_conversion_item_input_audio_transcription_completed(event) Result: - in_progress -> is_final=False -> interim only, no duplicate conversation item. - completed -> is_final=True -> exactly one final conversation item. Interim user_input_transcribed events are preserved for UI placeholders. ### Reproduction Steps ```bash 1. Start an AgentSession with livekit-plugins-xai RealtimeModel (e.g. grok-voice-think-fast-1.0), input audio transcription enabled. 2. Subscribe to conversation_item_added (and/or input_audio_transcription_completed). 3. Speak one sentence. 4. Observe multiple is_final=True / conversation_item_added events for the same item_id with a growing transcript, instead of a single final one. ``` ### Operating System macOS ### Models Used _No response_ ### Package Versions ```bash livekit-agents === 1.5.17 livekit-plugins-xai == 1.5.17 livekit-plugins-openai == 1.5.17 ``` ### Session/Room/Call IDs _No response_ ### Proposed Solution ```python ``` ### Additional Context _No response_ ### Screenshots and Recordings _No response_
1 条评论