Skip to content

feat(background_tasks): worker status admin page - #99

Merged
antosubash merged 15 commits into
mainfrom
feature/ecstatic-roentgen-c86488
May 1, 2026
Merged

feat(background_tasks): worker status admin page#99
antosubash merged 15 commits into
mainfrom
feature/ecstatic-roentgen-c86488

Conversation

@antosubash

Copy link
Copy Markdown
Owner

Summary

Adds a read-only /admin/background-tasks/workers sub-page so operators can see whether a Celery worker is connected to the broker and what each worker is doing — closes the visibility gap where "tasks stuck in pending" was indistinguishable from "no tasks enqueued yet."

  • New WorkerInspector wraps celery.control.inspect() with a probe-then-inspect flow that disambiguates broker-down from broker-up-with-no-workers. Catches OperationalError, RedisError, ConnectionError, OSError and surfaces them as WorkerSnapshot(broker_reachable=False, error=...) instead of 5xx-ing the page.
  • Two endpoints: Inertia view at /admin/background-tasks/workers (initial render) and JSON at /api/background_tasks/admin/workers (Refresh button uses plain fetch). Both run the inspector via asyncio.to_thread so the broker round-trip never blocks the event loop.
  • Workers.tsx page with three states — broker unreachable, no workers connected, list of worker cards (hostname, online dot, queue badges, active/pool/processed counts, software). Refresh button shows a sonner toast on failure, mirroring the existing retry pattern.
  • Discoverability: a "Workers" button in the existing Index toolbar (no new sidebar entry per the design).

Design choices worth flagging

  • Why a separate page (not a banner on Index): keeps the executions table focused on tasks; lets us show per-worker detail without crowding the toolbar.
  • Why manual Refresh, not auto-poll: inspect.ping() is a broadcast over the broker; polling every few seconds adds steady chatter for a page operators only glance at. One-click verify is intentional.
  • Why probe + inspect (extra round-trip): without the probe, inspect.*() returns None for both "broker down" and "no workers replied" — operators couldn't tell which.
  • Why redis.RedisError is caught explicitly: redis.exceptions.TimeoutError / ConnectionError don't inherit from built-in OSError or ConnectionError, so they'd otherwise escape and become 500s. Verified by an authentication-error test case.

Test plan

  • make test — 1054 Python + 8 JS tests pass (33 in modules/background_tasks/tests/ including 4 new inspector unit tests + 4 new endpoint tests covering auth, snapshot payload, broker-unreachable response, and Inertia render).
  • make lint — ruff format/check, ty, biome, tsc, 300-line cap, doctor checks (SM003/SM004/SM018/SM019), metadata, READMEs all clean.
  • make doctor — no new warnings.
  • Manual smoke (deferred to reviewer): with make dev running, exercise the three UI states by stopping the broker / starting it without workers / starting uv run python scripts/run_worker.py. Confirm the Index toolbar "Workers" link round-trips.

Notes for the reviewer

  • The plan and spec are committed at docs/superpowers/{specs,plans}/2026-05-01-background-tasks-worker-status-page-*.md for context.
  • A small refactor commit at the end consolidates the _stub_celery test fixture into a module conftest (opt-in via pytestmark.usefixtures so signal tests still get an unstarted app), and lifts WorkerInfo/WorkerSnapshot/formatTs into pages/constants.ts so Detail.tsx and Workers.tsx share them.

antosubash added 15 commits May 1, 2026 01:00
Adds a read-only /background-tasks/workers sub-page that surfaces Celery worker presence (hostname, queues, active count, pool, processed) via celery.control.inspect(). Manual refresh button, no auto-poll, reuses background_tasks.view permission, no sidebar entry.
…and formatTs

- Move _stub_celery autouse fixture to a module-scoped opt-in via
  pytestmark.usefixtures, dedupe across test_admin_api.py and the new
  test_workers_endpoints.py.
- Hoist WorkerInfo/WorkerSnapshot interfaces and formatTs into
  pages/constants.ts so Detail.tsx and Workers.tsx share them.
- Hoist `from background_tasks import worker_inspector as wi` to module
  scope in test_workers_endpoints.py.
@antosubash
antosubash merged commit 9bdb52b into main May 1, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant