Skip to content

DOC-7314 Add RQ and Dramatiq integration pages - #4602

Merged
andy-stark-redis merged 1 commit into
mainfrom
DOC-7314-add-rq-and-dramatiq-integration-docs
Nov 17, 2026
Merged

andy-stark-redis merged 1 commit into
mainfrom
DOC-7314-add-rq-and-dramatiq-integration-docs

Conversation

@andy-stark-redis

@andy-stark-redis andy-stark-redis commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Adds two integration pages under content/integrate/: RQ and Dramatiq, following the Celery page's shape.
  • Links both from the job-queue use case's Ecosystem section.
  • Follow-up to DOC-7313; part of epic DOC-7308; closes DOC-7314.

Verification

Every snippet was run against Redis 7.2.7 with RQ 3.0.0 and Dramatiq 2.2.1. The pages record what Redis held and how each library behaved after kill -9:

  • RQ: the job stays in the started registry as STARTED, a new worker leaves it alone, and registry cleanup later moves it to the failed registry as AbandonedJobError. Nothing re-runs it.
  • Dramatiq: with default settings, a new worker did not re-run the killed worker's job in 204 seconds (60 s heartbeat_timeout, maintenance_chance of 1,000 in a million). With heartbeat_timeout=5000 and maintenance_chance=1000000, it re-ran the job 6 seconds after the kill.

For review

  • Not run (and not claimed): RQ's failed-job requeue API and cron scheduling, Dramatiq's dead-letter queue and RabbitMQ broker. Links to upstream docs are site roots.
  • Neither library is tracked on the Confluence "Popular ecosystems" page, so the pages describe behavior and don't recommend either. RQ was suggested by a reviewer on DOC-7313 Add Celery, Sidekiq, and BullMQ integration pages #4600.
  • Weights (15 and 16) follow the Celery, Sidekiq, and BullMQ pages (12 to 14).

🤖 Generated with Claude Code


Note

Low Risk
Documentation-only additions and internal link updates; no application or infrastructure code changes.

Overview
Adds first-party integration docs for RQ and Dramatiq, modeled on existing Python queue guides like Celery.

Each new page covers install/requirements, enqueue-and-worker usage, retries and scheduling, a Redis key layout table, and empirically tested worker kill -9 behavior (RQ leaves jobs in the started registry until cleanup marks them abandoned; Dramatiq recovery depends heavily on heartbeat_timeout and maintenance_chance).

The job-queue use case Ecosystem section now links to these pages instead of external RQ/Dramatiq URLs.

Reviewed by Cursor Bugbot for commit 24ccf17. Bugbot is set up for automated code reviews on this repo. Configure here.

Adds content/integrate/rq and content/integrate/dramatiq, following the Celery page,
and links both from the job-queue use case's Ecosystem section in place of the external
project sites. A reviewer on the Celery, Sidekiq, and BullMQ PR suggested RQ. Dramatiq is
the other Python queue the section listed.

Every snippet was run against Redis 7.2.7 with RQ 3.0.0 and Dramatiq 2.2.1, and the
pages record observed behavior. RQ stores its results in streams, keeps a running job in
a started registry (rq:wip:<queue>), and needs a worker started with --with-scheduler to
run enqueue_in jobs. After kill -9 the job stays STARTED and a new worker leaves it
alone; registry cleanup later moves it to the failed registry as AbandonedJobError, and
nothing re-runs it. Dramatiq takes a message, runs it, and acknowledges afterward, so a
killed worker's message sits in its __acks__ set. With the default 60 second
heartbeat_timeout and 0.1% maintenance_chance, a new worker did not re-run the job in
204 seconds. With heartbeat_timeout=5000 and maintenance_chance=1000000 it re-ran the job
6 seconds after the kill. Both pages say to write jobs so that running one twice is safe.

Two of my first Dramatiq kill tests were invalid. One passed zsh a single string for
several PIDs, so the worker was never killed; another killed the worker before the job had
started. A kill test is only valid once the log shows the job running, and kill must
receive real PIDs.

Learned: A kill-recovery measurement is valid only if the job is confirmed running at kill time and the kill reached the process; Dramatiq recovery depends on heartbeat_timeout and a probabilistic maintenance pass
Constraint: RQ and Dramatiq kill-recovery claims come from single-worker runs against RQ 3.0.0 and Dramatiq 2.2.1; recheck if either changes how it recovers abandoned jobs
Directive: Do not describe Dramatiq as promptly redelivering a killed worker's messages without the heartbeat_timeout and maintenance_chance qualifiers
Gaps: RQ failed-job requeue API and scheduler cron jobs not run; Dramatiq dead-letter queue and RabbitMQ broker not run; links to upstream docs are site roots, not deep pages
Recheck: when RQ 4 or Dramatiq 3 ships
Ticket: DOC-7314
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

DOC-7314

@github-actions

Copy link
Copy Markdown
Contributor

🧠 Redis Memory

Found 5 related items from repository history (5 new this commit):

Memory updated at 24ccf17

@andy-stark-redis
andy-stark-redis requested a review from a team November 17, 2026 15:36
@andy-stark-redis andy-stark-redis self-assigned this Nov 17, 2026
@andy-stark-redis andy-stark-redis added the ecosystem Docs for external Redis integrations and resources label Nov 17, 2026
@@ -0,0 +1,131 @@
---
LinkTitle: RQ
Title: Redis with RQ

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

\0/

@dwdougherty dwdougherty left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@andy-stark-redis

Copy link
Copy Markdown
Contributor Author

Thanks @dwdougherty (for both the RQ suggestion and the review).

@andy-stark-redis
andy-stark-redis merged commit 48be16a into main Nov 17, 2026
114 checks passed
@andy-stark-redis
andy-stark-redis deleted the DOC-7314-add-rq-and-dramatiq-integration-docs branch November 17, 2026 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ecosystem Docs for external Redis integrations and resources

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants