-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Race condition: background compaction causes stale session ValueError on next request #5194
Copy link
Copy link
Open
Labels
services[Component] This issue is related to runtime services, e.g. sessions, memory, artifacts, etc[Component] This issue is related to runtime services, e.g. sessions, memory, artifacts, etc
Description
Description
When EventsCompactionConfig is enabled on an App, the compaction runs as a fire-and-forget asyncio.create_task at the end of each invocation in runners.py (line ~436):
asyncio.create_task(
_run_compaction_for_sliding_window(
self.app, session, self.session_service
)
)This causes a race condition with DatabaseSessionService.append_event, which checks:
if storage_session.update_timestamp_tz > session.last_update_time:
raise ValueError("The last_update_time provided in the session object ... is earlier than the update_time in the storage_session ... Please check if it is a stale session.")Steps to reproduce
- Configure an
AppwithEventsCompactionConfigand useDatabaseSessionService - Send enough messages to trigger compaction (
compaction_intervalinvocations) - Send the next message shortly after compaction fires
What happens
- Agent finishes a turn → all events appended →
session.last_update_timeset to DB'supdate_time(e.g.14:26:43) - Compaction fires as background task → calls
session_service.append_event()for the summary event → DBupdate_timebecomes14:26:51 - Next user message arrives → runner uses same in-memory
sessionobject withlast_update_time = 14:26:43 append_eventfails the stale check:14:26:51 > 14:26:43→ValueError
The background task updates the DB timestamp but the in-memory session object is never refreshed.
Expected behavior
Compaction should not cause subsequent requests to fail. Either:
- The compaction task should update
session.last_update_timeafter its DB write (it does via line 687, but the caller's reference to the session may already be stale by the time the next request picks it up) - Or the session should be re-fetched from DB at the start of each invocation
- Or the stale check should account for compaction events
Environment
google-adk==1.18.0DatabaseSessionService(PostgreSQL)- Python 3.12
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
services[Component] This issue is related to runtime services, e.g. sessions, memory, artifacts, etc[Component] This issue is related to runtime services, e.g. sessions, memory, artifacts, etc