Skip to content

Tech debt: fix caching  #681

@fulmicoton

Description

@fulmicoton

In #680 we replugged the long term cache, caching fast fields.

The solution is however not great as

  • it relies on the fact that segment ids are universally unique
  • it uses a global.

It would be worth putting more thought on how this should be implemented.

/// Wraps the given directory with a slice cache that is actually global
/// to quickwit.
///
/// FIXME The current approach is quite horrible in that:
/// - it uses a global
/// - it relies on the idea that all of the files we attempt to cache
/// have universally unique names. It happens to be true today, but this might be very error prone
/// in the future.
pub fn wrap_storage_with_long_term_cache(storage: Arc<dyn Storage>) -> Arc<dyn Storage> {
    static SINGLETON: OnceCell<Arc<dyn Cache>> = OnceCell::new();
    let cache = SINGLETON
        .get_or_init(|| Arc::new(QuickwitCache::default()))
        .clone();
    Arc::new(StorageWithCache { storage, cache })
}

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions