Prevent update_chat_ctx from deleting in-flight function calls using function call attributes#5021
Open
StianHanssen wants to merge 3 commits intolivekit:mainfrom
Conversation
…they have arrived in local context
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
update_chat_ctxcan delete in-flightfunction_callitems from the OpenAI Realtime server, causing cascading"failed to insert item: previous_item_id not found"corruption of_remote_chat_ctx.The root cause is a timing gap between two context-tracking structures:
_remote_chat_ctx: Updated immediately when the server sendsconversation.item.added_agent._chat_ctx: Updated later, only when tool execution starts (_tool_execution_started_cb)If
update_chat_ctxruns during this window (e.g. from context management), the diff sees thefunction_callin remote but not in local, treats it as intentionally removed, and sends a delete event. The existing_is_content_emptyguard only protectsmessageitems —function_callitems pass through unconditionally.A unit test gist replicating the exact pipeline demonstrates how
update_chat_ctxdeletes in-flightfunction_callitems.Fix
Use a shared-object flag (
extra["dispatched"]) onFunctionCallitems to distinguish in-flight from intentionally removed function calls.openai_item_to_livekit_itemsetsextra["dispatched"] = Falsewhen creating aFunctionCallfrom a server event._handle_function_callreuses the sameFunctionCallobject from_remote_chat_ctxinstead of creating a new one. This is safe becauseconversation.item.addedalways precedesresponse.output_item.doneon the websocket. The same Python object is now shared across both contexts._create_update_chat_ctx_eventsskips deletion for anyfunction_callwithextra["dispatched"] == False. Once the flag isTrue, summarization and other callers can delete the item normally.agent_activity.pysetsextra["dispatched"] = Truewhen tool execution starts or when all function calls for a generation are finalized (afterawait exe_task). Since the object is shared, this is visible to the diff guard immediately — no cross-package signaling needed.Future consideration
This fix only tracks
function_callitems.function_call_outputitems are currently client-initiated (manual_function_calls=True), so they enter_agent._chat_ctxbefore_remote_chat_ctxand are not vulnerable to this race. Ifauto_tool_reply_generationis enabled in a future configuration (server-generated outputs), a guard should be added to coverfunction_call_outputitems as well.