Skip to content

✨ Add ScheduledTaskTool with global scheduler and frontend polling#3216

Open
2862282695gjh-afk wants to merge 9 commits into
ModelEngine-Group:developfrom
2862282695gjh-afk:feature/scheduled-task-tool
Open

✨ Add ScheduledTaskTool with global scheduler and frontend polling#3216
2862282695gjh-afk wants to merge 9 commits into
ModelEngine-Group:developfrom
2862282695gjh-afk:feature/scheduled-task-tool

Conversation

@2862282695gjh-afk

Copy link
Copy Markdown

Summary

Add a ScheduledTaskTool that enables Nexent agents to create, list, and cancel cron-based or one-shot scheduled tasks.

Architecture

ScheduledTaskTool (thin CRUD)
  ├── create → write to scheduled_tasks_t
  ├── list   → read by agent_id + tenant_id
  └── cancel → update status

Global SchedulerService (singleton, survives agent runs)
  └── 10s DB poll → find due tasks → new agent run → save messages

Frontend Polling
  ├── Layer 1: 5s poll active conversation (refresh + scroll)
  └── Layer 2: 10s batch-poll background conversations (silent cache update)

Why not just a tool module?

Nexent creates a new agent instance per message — tools, scheduler threads, and all state are GCd after each run. A scheduled task tool needs to survive beyond a single agent run, which requires:

  1. DB persistence — task data outlives any agent instance
  2. Global scheduler — independent singleton thread, started at runtime_service startup
  3. Frontend polling — SSE is request-response only, no persistent connection for push

Key Design Decisions

Decision Rationale
Global scheduler, not per-tool Agent instances are ephemeral — per-tool schedulers die with the agent
Scheduler strips ScheduledTaskTool from triggered agent Prevents recursive task creation (agent sees the prompt and tries to create another task)
message_idx from DB max() Original save_messages computes idx from history=[] → always 0, causing collisions
Two-layer frontend polling Active conversation gets fast updates; background conversations get silent cache refresh
Ownership check in polling API get_new_messages_service validates conversation.created_by == user_id

Multi-Tenant Compatibility

  • ScheduledTaskRecord includes tenant_id and user_id fields
  • query_tasks_by_agent and cancel_task filter by tenant_id
  • Scheduler is global (polls all pending tasks) but uses each tasks own tenant_id/user_id for agent runs
  • Frontend polling API validates conversation ownership

Files Changed (15 files, +877 lines, all additive)

New files (3):

  • backend/database/scheduled_task_db.py — 5 CRUD functions
  • backend/services/scheduled_task_scheduler.py — global scheduler singleton
  • sdk/nexent/core/tools/scheduled_task_tool.py — tool with cron parser

Modified files (12, no existing logic changed):

  • backend/database/db_models.pyScheduledTaskRecord model
  • backend/database/conversation_db.pyget_max_message_index()
  • backend/services/conversation_management_service.pyget_new_messages_service()
  • backend/apps/conversation_management_app.py — polling endpoints
  • backend/apps/runtime_app.py — scheduler lifecycle
  • backend/agents/create_agent_info.pyconversation_id passthrough
  • sdk/nexent/core/agents/nexent_agent.py — ScheduledTaskTool metadata injection
  • sdk/nexent/core/tools/__init__.py — export ScheduledTaskTool
  • docker/init.sqlscheduled_tasks_t table
  • frontend/services/api.ts — new endpoints
  • frontend/services/conversationService.ts — polling methods
  • frontend/app/[locale]/chat/internal/chatInterface.tsx — polling useEffect

Testing

Tested with Docker (v2.2.0 image + bind mounts):

  • Created "remind me every 5 minutes" task → agent created task in DB ✅
  • Scheduler detected due task → triggered agent run → wrote messages to DB ✅
  • Frontend polling detected new messages → refreshed display ✅
  • Switched conversation → background polling updated cache ✅
  • No ScheduledTaskTool not found errors ✅
  • No recursive task creation ✅

@2862282695gjh-afk 2862282695gjh-afk force-pushed the feature/scheduled-task-tool branch 3 times, most recently from 963cdbe to 66f0180 Compare June 11, 2026 02:01
except (ValueError, IndexError):
return None

@staticmethod

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cron 解析器只支持精确数字和 *,不支持 */151-51,3,5 等标准 cron 语法。用户尝试这些时会抛 ValueError。建议在 description 中说明支持的语法子集,或使用 croniter 库。


if month_match and dom_match and hour_match and minute_match:
return candidate

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

性能问题:逐分钟遍历计算下次触发时间,最多 525,960 次循环。建议直接用 datetime 算术计算下一个匹配时间点。

t = threading.Thread(
target=_run_scheduled_task_from_db,
args=(task_dict,),
daemon=True,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

串行阻塞:t.join(timeout=300) 让调度器为每个任务阻塞最多 5 分钟。10 个任务同时到期时,最后一个要等 50 分钟。建议用 ThreadPoolExecutor 并行执行。

query=user_content,
history=[],
tenant_id=tenant_id,
user_id=user_id,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

asyncio.run() 和手动 new_event_loop() 混用会导致事件循环冲突和资源泄漏。建议统一在一个 async 函数中执行:

async def _execute():
    agent_run_info = await create_agent_run_info(...)
    async for chunk in agent_run(agent_run_info):
        chunks.append(chunk)
asyncio.run(_execute())

user_id, tenant_id = get_current_user_id(authorization)
result = get_new_messages_service(conversation_id, user_id, since_index)
return JSONResponse(status_code=HTTPStatus.OK, content=result)
except Exception as e:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[代码规范] except Exception: 过于宽泛,建议捕获更具体的异常类型,避免掩盖潜在错误。

if cid is not None:
results[str(cid)] = get_new_messages_service(cid, user_id, since)
return JSONResponse(status_code=HTTPStatus.OK, content={"results": results})
except Exception as e:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[代码规范] except Exception: 过于宽泛,建议捕获更具体的异常类型,避免掩盖潜在错误。


logger.info(f"Scheduled task {task_uuid} executed successfully")

except Exception as e:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[代码规范] except Exception: 过于宽泛,建议捕获更具体的异常类型,避免掩盖潜在错误。

if t.class_name != "ScheduledTaskTool"
]

# Run agent and collect response chunks

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[逻辑漏洞] _run_scheduled_task_from_db 中先使用 asyncio.run() 创建了一个事件循环,随后又用 asyncio.new_event_loop() 创建了另一个事件循环。asyncio.run() 会自动创建并管理事件循环,在同一函数中混用两种方式可能导致事件循环冲突。建议统一使用 asyncio.new_event_loop() + loop.run_until_complete() 的方式,或统一使用 asyncio.run()

Comment thread backend/apps/runtime_app.py Outdated
app.include_router(skill_creator_router)


@app.on_event("startup")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[代码规范] @app.on_event("startup")@app.on_event("shutdown") 在 FastAPI 中已被标记为弃用(deprecated),建议使用 lifespan 上下文管理器替代。参考:https://fastapi.tiangolo.com/advanced/events/

@router.post("/batch_new_messages", response_model=Dict[str, Any])
async def batch_check_new_messages_endpoint(request: Dict[str, Any], authorization: Optional[str] = Header(None)):
"""
Batch check for new messages across multiple conversations.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[代码规范] batch_check_new_messages_endpoint 的请求体使用了 Dict[str, Any],缺少类型约束和输入验证。建议使用 Pydantic 模型定义请求结构(如 BatchMessageCheckRequest),确保 checks 列表中的每个元素都包含必要的字段和类型校验。

self,
action: str,
task_name: Optional[str] = None,
task_prompt: Optional[str] = None,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ScheduledTaskTool 使用 asyncio.run() 在同步 forward 方法中执行异步任务,但如果 forward 已经在异步上下文中被调用(如通过 asyncio.to_thread),会抛出 RuntimeError。建议检查调用上下文,或使用 asyncio.get_event_loop().run_until_complete() 替代。

Add a scheduled task tool that enables agents to create, list, and cancel
cron-based or one-shot scheduled tasks. The architecture uses a global
scheduler singleton (independent of agent lifecycle), DB persistence, and
frontend polling for real-time message delivery.

Changes:
- SDK: ScheduledTaskTool as thin CRUD wrapper with cron parser
- Backend: global ScheduledTaskScheduler (10s DB poll), scheduled_task_db
- Backend: new_messages polling API + batch endpoint
- Backend: conversation_id passthrough for task-to-session binding
- Backend: scheduler strips ScheduledTaskTool from triggered agent to
  prevent recursive task creation
- Frontend: two-layer polling (5s active, 10s background conversations)
- DB: scheduled_tasks_t table with multi-tenant fields

No existing Nexent logic is modified — all changes are additive.
@WMC001

WMC001 commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Good addition. The ScheduledTaskTool integration with cron support looks reasonable. Please ensure the cron expression validation is covered by tests before merging.

@2862282695gjh-afk 2862282695gjh-afk force-pushed the feature/scheduled-task-tool branch from 66f0180 to 499442f Compare June 24, 2026 07:32
@2862282695gjh-afk

Copy link
Copy Markdown
Author

Thanks @WMC001! Added unit tests covering cron expression validation and next-fire-time computation in test/sdk/core/tools/test_scheduled_task_tool.py (27 test cases, all passing).

Coverage includes:

  • _expand_cron_field: *, exact values, ranges (1-5), steps (*/15, 1-30/10), comma lists (1,3,5, mixed), Sunday normalization (70), and invalid inputs (out-of-range, zero step)
  • _parse_cron: valid 5-field expressions, complex expressions (*/15 9-18 * * 1-5), and 7 invalid-expression cases (out-of-range fields, wrong field count, non-numeric, zero step)
  • _compute_next_fire: daily scheduling, step intervals, weekday filtering, skipping to next allowed day/month, rolling to next year, impossible-expression fallback, and past-hour jump-to-tomorrow

The tests load the tool module via importlib with a stubbed smolagents.tools.Tool base class, so they run without the full SDK/memory dependency chain.

@WMC001

WMC001 commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Bug 1: batch_check_new_messages_endpoint has no error isolation

backend/apps/conversation_management_app.py line 196-201 — the loop over request.checks has no try/except. If get_new_messages_service raises for any single conversation, the entire batch fails and the caller gets a 500 with no partial results:

for check in request.checks[:50]:
    results[str(check.conversation_id)] = get_new_messages_service(  # <-- no try/except
        check.conversation_id, user_id, check.since_index
    )

Fix: wrap the call in a try/except and record an error indicator per conversation instead of propagating the exception.

@WMC001

WMC001 commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Bug 2: cron _compute_next_fire incorrect for day_of_month and day_of_week interaction

sdk/nexent/core/tools/scheduled_task_tool.py line 1165-1168 — when candidate.day not in allowed_dom, the code advances to tomorrow at 00:00. But this skips over days that might match day_of_week. For example, cron "0 9 15 * 1" (9am on the 15th OR any Monday): advancing day-by-day from June 24 could skip past the next Monday if it lands on a non-matching day. The algorithm should consider both day-of-month and day-of-week simultaneously, or follow standard cron OR logic (if both fields are restricted, either matching satisfies the condition).

Note: Python's croniter library and standard cron treat day-of-month and day-of-week with OR semantics by default (unless L or # modifiers are used). The current logic treats them as AND, which is incorrect.

Fix: when both day_of_month and day_of_week have restricted values (non-*), use OR logic — accept the day if it matches either field.

…batch endpoint errors

Address WMC001 review:
- _parse_cron now tracks day_of_month/day_of_week restriction flags
- _compute_next_fire uses standard cron OR semantics: when both day
  fields are restricted, a day matches if it satisfies either field
  (previously used AND, which is non-standard)
- batch_check_new_messages_endpoint isolates per-conversation failures
  so one error returns an error marker instead of failing the whole batch
- add tests covering OR semantics and the restriction flags
@2862282695gjh-afk

Copy link
Copy Markdown
Author

Thanks @WMC001, both issues fixed in the latest commit:

Bug 1 (batch error isolation)batch_check_new_messages_endpoint now wraps each conversation check in its own try/except. A failing conversation returns {"error": "check_failed"} instead of propagating and failing the whole batch with a 500.

Bug 2 (day-of-month / day-of-week OR semantics) — good catch, the previous AND logic was non-standard. Now _parse_cron records day_of_month_restricted / day_of_week_restricted flags (true when the field was not a bare *), and _compute_next_fire applies standard cron OR semantics: when both day fields are restricted, a day matches if it satisfies either field; when only one is restricted, that field must match.

Added 5 new tests covering this:

  • 0 9 15 * 1 (15th OR Monday) → matches the next Monday (06-29)
  • 0 9 26 * 1 (26th OR Monday) → matches the earlier day-of-month hit (06-26)
  • 0 9 29 * * (only dom restricted) → still requires the 29th
  • 0 9 * * 1 (only dow restricted) → next Monday
  • parse records the restriction flags correctly

All 32 tests passing locally.

Address several issues that blocked the scheduled-task feature from
working in practice, found while driving the full flow:

- task_type auto-inference (_resolve_task_type): passing cron_expression
  alone was silently treated as oneshot and demanded delay_seconds.
  Now inferred from whichever time field is present; schema rewritten
  to drop examples and state the mutual-exclusion clearly.
- cancel accepts task_uuid OR task_name: callers often pass the task id
  via task_name; now it cancels by uuid first, then falls back to a
  name lookup, so the first attempt succeeds.
- assistant message rendering: _save_assistant_chunks parses/merges
  agent_run chunks (drops metadata, folds same-type units) instead of
  dumping raw JSON strings; also fixed a NameError (MESSAGE_ROLE import).
- trigger message wording: split the display text (short 'task fired'
  line) from the agent query (full instruction), so internal prompts
  are never shown in the chat bubble.
- frontend polling races: skip refresh while a conversation is actively
  streaming, and reuse the standard message extractors in refresh so
  tool-call steps survive (previously wiped mid-stream or on finish).
- tool descriptions simplified: pure contract, no scenario examples,
  so complex tasks (e.g. scheduled briefings) aren't biased to 'reminder'.

Add unit tests for _resolve_task_type and _handle_cancel (44 total).
@2862282695gjh-afk

Copy link
Copy Markdown
Author

Pushed a follow-up commit (f67afb71) fixing usability bugs found while driving the full scheduled-task flow end-to-end. These are the issues that actually blocked the feature from working in practice:

1. task_type auto-inference (_resolve_task_type)
Passing only cron_expression was silently treated as oneshot (the default) and then demanded delay_seconds, so cron-task creation kept failing with a confusing error. The task type is now inferred from whichever time field is present, and task_type can be omitted entirely. Tool input descriptions rewritten to be pure contract (no scenario examples) and to state the cron/delay mutual-exclusion clearly.

2. cancel accepts task_uuid OR task_name
Callers frequently pass the task id via task_name. Cancel now tries the uuid directly, then falls back to a name lookup, so the first attempt succeeds instead of erroring.

3. Assistant message rendering (_save_assistant_chunks)
The scheduler was dumping raw JSON chunk strings as the assistant content. Now chunks are parsed, metadata chunks (step_count/token_count/agent_new_run) filtered out, and same-type units folded — matching save_conversation_assistant. Also fixed a NameError (MESSAGE_ROLE import).

4. Trigger message split into display vs. query
The internal instruction prompt was being shown in the chat bubble. Now a short human-readable line is stored as the user message, while the full instruction is passed only to the agent as the run query.

5. Frontend polling races
Polling no longer overwrites a conversation that is actively streaming (which wiped in-progress tool-call rendering), and the refresh path reuses the standard message extractors so tool-call steps survive refresh instead of collapsing to just the final answer.

Tests: added coverage for _resolve_task_type (inference + rejection cases) and _handle_cancel (uuid / name-fallback / not-found). 44 unit tests passing locally.

…escription

- cancel_task now cancels tasks in 'pending' OR 'fired' state, so a task can
  be cancelled even while it is executing mid-run.
- Add reschedule_if_active: cron tasks are re-armed only while still 'fired',
  so a task cancelled mid-run is never resurrected on its next cycle.
- _run_and_reschedule uses the atomic reschedule guard instead of
  unconditionally resetting to 'pending'.
- Trim the tool description to a concise, codebase-consistent length and make
  the cancel flow explicit (list -> cancel by uuid -> verify).
- Add unit tests for the cancel_task and reschedule_if_active DB functions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@2862282695gjh-afk

Copy link
Copy Markdown
Author

Heads-up for reviewers on the cancel reliability of this tool, based on end-to-end testing:

This feature's reliability depends on the backing LLM's tool-use discipline. The scheduled_task tool is invoked via smolagents' prompt-based tool calling (the model emits Python code to call it), so there is no framework-level tool_choice forcing a call. With a weaker model we observed that "cancel this task" could occasionally be short-circuited: the model emitted a final_answer claiming success without actually calling the tool (a hallucination). The tool code, DB layer, and scheduler all behave correctly when the tool is called — this is purely an LLM-behavior issue.

Mitigations already in this PR (code-side, model-independent):

  • cancel_task accepts both pending and fired, so a task can be cancelled even mid-execution.
  • reschedule_if_active re-arms a cron task only while still fired, so a cancelled task is never resurrected on its next cycle.
  • The tool description keeps the cancel flow explicit and concise (list → cancel by uuid → verify).

In testing, switching the agent from qwen-plus to qwen3-max eliminated the hallucination — confirming the root cause is model tier, not the tool.

If a deployment uses a weaker model and needs a hard guarantee, the next step would be a verification hook in the agent run: intercept a final_answer that claims a scheduled_task outcome when no matching tool call exists in the step trace, and force a retry. That is intentionally out of scope for this PR (it touches the agent run pipeline and is model-agnostic infra), but I wanted to flag it as the recommended follow-up rather than expand this PR's surface.

2862282695gjh-afk and others added 4 commits June 25, 2026 16:15
Address review feedback (YehongPan) on overly broad except clauses:

- Polling endpoints (check_new_messages / batch_check_new_messages): catch
  SQLAlchemyError instead of Exception, so non-DB errors are no longer masked
  and can surface to FastAPI's default handler.
- Scheduler: the inner best-effort status updates now catch SQLAlchemyError.
  The daemon-loop and per-task handlers stay broad on purpose (a worker must
  survive any single failure), but each is now commented to state that intent
  and relies on exc_info so the real cause is logged, not hidden.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…task-tool

# Conflicts:
#	backend/agents/create_agent_info.py
…task-tool

# Conflicts:
#	backend/agents/create_agent_info.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants