-
Notifications
You must be signed in to change notification settings - Fork 173
Automatically clean-up old, finished background jobs #562
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
By default, finished background jobs are now deleted during the night. This ensures old, finished background job records don't linger around.
lib/generators/solid_queue/install/templates/config/recurring.yml
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I ran into a very similar issue and was thinking about opening a similar PR. 😄
- `clear_finished_jobs_after`: period to keep finished jobs around, in case `preserve_finished_jobs` is true—defaults to 1 day. **Note:** Right now, there's no automatic cleanup of finished jobs. You'd need to do this by periodically invoking `SolidQueue::Job.clear_finished_in_batches`, which can be configured as [a recurring task](#recurring-tasks). | ||
- `clear_finished_jobs_after`: period to keep finished jobs around, in case `preserve_finished_jobs` is true — defaults to 1 day. **Note:** `SolidQueue` automatically cleans up the finished jobs at 1 AM. To disable this, remove the `automatically_cleanup_finished_jobs` periodic job from the `recurring.yml` configuration. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think these changes to the README could be a little misleading. Even with these changes SolidQueue won't actually clean up the jobs. Instead when you install SolidQueue a job will be created that will clean up jobs.
I could see someone that ran the generator before this change being confused because their jobs aren't being automatically cleaned up and that code won't be in the recurring.yml
.
Thanks both! I'm going to go with something intermediate to this, maybe running this every hour, and with a sleep interval as well. I'm worried this might introduce some unexpected load for people by surprise 😅 |
This PR is a suggestion to automatically clean up old, finished background jobs. This prevents people from shooting themselves in the foot as I did.
See the discussion here: #560.