Skip to content

Latest commit

 

History

History
63 lines (51 loc) · 2.37 KB

retention.md

File metadata and controls

63 lines (51 loc) · 2.37 KB

Loki Storage Retention

Retention in Loki is achieved through the Table Manager. In order to enable the retention support, the Table Manager needs to be configured to enable deletions and a retention period. Please refer to the table_manager_config section of the Loki configuration reference for all available options. Alternatively, the table-manager.retention-period and table-manager.retention-deletes-enabled command line flags can be used. The provided retention period needs to be a duration represented as a string that can be parsed using Go's time.Duration.

WARNING: The retention period must be a multiple of the index and chunks table period, configured in the period_config block. See the Table Manager documentation for more information.

When using S3 or GCS, the bucket storing the chunks needs to have the expiry policy set correctly. For more details check S3's documentation or GCS's documentation.

Currently, the retention policy can only be set globally. A per-tenant retention policy with an API to delete ingested logs is still under development.

Since a design goal of Loki is to make storing logs cheap, a volume-based deletion API is deprioritized. Until this feature is released, if you suddenly must delete ingested logs, you can delete old chunks in your object store. Note, however, that this only deletes the log content and keeps the label index intact; you will still be able to see related labels but will be unable to retrieve the deleted log content.

For further details on the Table Manager internals, refer to the Table Manager documentation.

Example Configuration

Example configuration with GCS with a 30 day retention:

schema_config:
  configs:
  - from: 2018-04-15
    store: bigtable
    object_store: gcs
    schema: v9
    index:
      prefix: loki_index_
      period: 168h

storage_config:
  bigtable:
    instance: BIGTABLE_INSTANCE
    project: BIGTABLE_PROJECT
  gcs:
    bucket_name: GCS_BUCKET_NAME

table_manager:
  retention_deletes_enabled: true
  retention_period: 720h