Skip to content

fix(ingest): run memorize in background when run_in_background=True#161

Open
relunctance wants to merge 1 commit intoFlowElement-ai:mainfrom
relunctance:fix/ingest-async-background
Open

fix(ingest): run memorize in background when run_in_background=True#161
relunctance wants to merge 1 commit intoFlowElement-ai:mainfrom
relunctance:fix/ingest-async-background

Conversation

@relunctance
Copy link
Copy Markdown

Fix: ingest memorize async (#158)

Problem: POST /api/v1/ingest with run_in_background=True still blocks for 30-120s because only the memorize stage ran in background; add remained sync.

Root cause: The run_in_background flag was only passed to memorize(), not applied to add() which does the heavy LLM entity extraction + LanceDB write.

Solution: When run_in_background=True:

  1. Run add() synchronously (memorize needs the dataset_id from add result)
  2. Fire memorize() as asyncio.create_task() — return BACKGROUND_STARTED immediately
  3. Log a warning if background memorize fails (fire-and-forget with error logging)

This unblocks hawk-memory's captureDualWrite() from timing out on mFlow calls.

Closes #158

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome to the M-flow project, @relunctance! 🎉
Thank you for opening this pull request. A core maintainer will review your changes shortly. In the meantime, please make sure you've read our Contributing Guide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

P0: ingest 延迟 30-120s — run_in_background=true 仍阻塞 add 阶段

1 participant