Automatically clean-up old, finished background jobs #562
Conversation
By default, finished background jobs are now deleted during the night. This ensures old, finished background job records don't linger around.
lib/generators/solid_queue/install/templates/config/recurring.yml
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
I ran into a very similar issue and was thinking about opening a similar PR. 😄
| - `clear_finished_jobs_after`: period to keep finished jobs around, in case `preserve_finished_jobs` is true—defaults to 1 day. **Note:** Right now, there's no automatic cleanup of finished jobs. You'd need to do this by periodically invoking `SolidQueue::Job.clear_finished_in_batches`, which can be configured as [a recurring task](#recurring-tasks). | ||
| - `clear_finished_jobs_after`: period to keep finished jobs around, in case `preserve_finished_jobs` is true — defaults to 1 day. **Note:** `SolidQueue` automatically cleans up the finished jobs at 1 AM. To disable this, remove the `automatically_cleanup_finished_jobs` periodic job from the `recurring.yml` configuration. |
There was a problem hiding this comment.
I think these changes to the README could be a little misleading. Even with these changes SolidQueue won't actually clean up the jobs. Instead when you install SolidQueue a job will be created that will clean up jobs.
I could see someone that ran the generator before this change being confused because their jobs aren't being automatically cleaned up and that code won't be in the recurring.yml.
|
Thanks both! I'm going to go with something intermediate to this, maybe running this every hour, and with a sleep interval as well. I'm worried this might introduce some unexpected load for people by surprise 😅 |
|
I ended up doing this but with some changes, with a different interval, and different wording in the README, in 6437617. Thank you! |
This PR is a suggestion to automatically clean up old, finished background jobs. This prevents people from shooting themselves in the foot as I did.
See the discussion here: #560.