Skip to content

Major compaction API #105

Closed
Closed
@marvin-j97

Description

@marvin-j97

This is already partially implemented as a hidden function, however it needs to be exposed as a method in AbstractTree (so it can be called for BlobTree as well, which will require a full rewrite of the entire value log (...maybe not)).

lsm-tree/src/tree/mod.rs

Lines 462 to 478 in 3a2e3e1

// TODO: Expose as public function, however:
// TODO: Right now this is somewhat unsafe to expose as
// major compaction needs ALL segments, right now it just takes as many
// as it can, which may make the LSM inconsistent.
// TODO: There should also be a function to partially compact levels and individual segments
/// Performs major compaction, blocking the caller until it's done.
///
/// # Errors
///
/// Will return `Err` if an IO error occurs.
#[doc(hidden)]
pub fn major_compact(&self, target_size: u64, seqno_threshold: SeqNo) -> crate::Result<()> {
log::info!("Starting major compaction");
let strategy = Arc::new(crate::compaction::major::Strategy::new(target_size));
self.compact(strategy, seqno_threshold)
}

It's important that the major compaction can get all segments, so it needs a write lock and wait until it can hide all segments.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions