Skip to content

Prevent update_chat_ctx from deleting in-flight function calls using function call attributes#5021

Open
StianHanssen wants to merge 3 commits intolivekit:mainfrom
StianHanssen:fix-tool-sync-can-cause-item-loss-function-call-approach
Open

Prevent update_chat_ctx from deleting in-flight function calls using function call attributes#5021
StianHanssen wants to merge 3 commits intolivekit:mainfrom
StianHanssen:fix-tool-sync-can-cause-item-loss-function-call-approach

Conversation

@StianHanssen
Copy link
Contributor

Summary

update_chat_ctx can delete in-flight function_call items from the OpenAI Realtime server, causing cascading "failed to insert item: previous_item_id not found" corruption of _remote_chat_ctx.

The root cause is a timing gap between two context-tracking structures:

  • _remote_chat_ctx: Updated immediately when the server sends conversation.item.added
  • _agent._chat_ctx: Updated later, only when tool execution starts (_tool_execution_started_cb)

If update_chat_ctx runs during this window (e.g. from context management), the diff sees the function_call in remote but not in local, treats it as intentionally removed, and sends a delete event. The existing _is_content_empty guard only protects message items — function_call items pass through unconditionally.

A unit test gist replicating the exact pipeline demonstrates how update_chat_ctx deletes in-flight function_call items.

Fix

Use a shared-object flag (extra["dispatched"]) on FunctionCall items to distinguish in-flight from intentionally removed function calls.

  1. openai_item_to_livekit_item sets extra["dispatched"] = False when creating a FunctionCall from a server event.

  2. _handle_function_call reuses the same FunctionCall object from _remote_chat_ctx instead of creating a new one. This is safe because conversation.item.added always precedes response.output_item.done on the websocket. The same Python object is now shared across both contexts.

  3. _create_update_chat_ctx_events skips deletion for any function_call with extra["dispatched"] == False. Once the flag is True, summarization and other callers can delete the item normally.

  4. agent_activity.py sets extra["dispatched"] = True when tool execution starts or when all function calls for a generation are finalized (after await exe_task). Since the object is shared, this is visible to the diff guard immediately — no cross-package signaling needed.

Future consideration

This fix only tracks function_call items. function_call_output items are currently client-initiated (manual_function_calls=True), so they enter _agent._chat_ctx before _remote_chat_ctx and are not vulnerable to this race. If auto_tool_reply_generation is enabled in a future configuration (server-generated outputs), a guard should be added to cover function_call_output items as well.

devin-ai-integration[bot]

This comment was marked as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant