Skip to content

Commit 8db019c

Browse files
jordanhunt22Convex, Inc.
authored andcommitted
[Retention] Make retention go even slower (#25357)
Increases the interval between scanning the documents log for retention from 10 mins -> 30 mins. The number of documents scanned per run was also decreased from `10000` - > `1000`. With these changes, we should be good to rollout retention again on monday. The changes made since the rollout have been: - removed `documents_to_delete` which should decrease the load by ~1/3 - cursor on scanned documents instead of deleted documents - increased time between batches -- should decrease load by ~2/3 - decreased documents scanned -- should decrease load by ~9/10 Let me know if there are any more optimizations we should consider. GitOrigin-RevId: b9e05b1031b7d33ac72cbce6cb25bcbfb78846b5
1 parent 4b60f3d commit 8db019c

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

crates/common/src/knobs.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -388,13 +388,13 @@ pub static RETENTION_FAIL_ALL_MULTIPLIER: LazyLock<usize> =
388388
pub static DOCUMENT_RETENTION_BATCH_INTERVAL_SECONDS: LazyLock<Duration> = LazyLock::new(|| {
389389
Duration::from_secs(env_config(
390390
"DOCUMENT_RETENTION_BATCH_INTERVAL_SECONDS",
391-
10 * 60,
391+
30 * 60,
392392
))
393393
});
394394

395395
/// Maximum scanned documents within a single run for document retention
396396
pub static DOCUMENT_RETENTION_MAX_SCANNED_DOCUMENTS: LazyLock<usize> =
397-
LazyLock::new(|| env_config("DOCUMENT_RETENTION_MAX_SCANNED_DOCUMENTS", 10000));
397+
LazyLock::new(|| env_config("DOCUMENT_RETENTION_MAX_SCANNED_DOCUMENTS", 1000));
398398

399399
/// Whether or not we run document retention in dry run mode
400400
pub static DOCUMENT_RETENTION_DRY_RUN: LazyLock<bool> =

0 commit comments

Comments
 (0)