Skip to content

Conversation

greatsharp
Copy link
Contributor

@greatsharp greatsharp commented Aug 7, 2025

Brief Introduction
Realtime hotkey analyze, implemented via LRU cache and deque, LRU cache is implemented by list and map.
Using one second as a counting cycle, when the key is accessed, accumulate the counter within each second; when crossing a second boundary, determine whether the access counter exceeds the threshold, If it does, identify it as a hot key. Then push the key into the deque.

Commands

  1. hotkey enable {capacity} {deque_size} {threshold}

Enable hotkey analyze, capacity is the LRU cache capacity, deque_size is the hotkey deque max size, if the key accessed counter is equal or greater than threshold within each second, then identify it as a hotkey, then put the key into the deque.
Snipaste_2025-08-07_22-27-09

  1. hotkey disable
    Disable the hotkey analyze, it will clear the LRU cache and hotkey deque.

  2. hotkey stats
    Display the hotkey stats

Snipaste_2025-08-07_22-34-36
  1. hotkey threshold {threshold}
    Change the hotkey threshold

  2. hotkey getbykey {key} {begin_timestamp_ms} {end_timestamp_ms}
    Search hotkey entries of {key} in hotkey deque

getbykey
  1. hotkey getbythreshold {threshold} {begin_timestamp_ms} {end_timestamp_ms}
    Search hotkey entries which equal to or larger than {threshold} in hotkey deque
getbythreshold
  1. hotkey dumplogfile {off|info|warning}
    Dump the hotkey entries to log file with level info or warning, off means do not dump.

  2. hotkey timerange {begin_timestamp_ms} {end_timestamp_ms}
    Search hotkey entries of it's last_access_time between begin_timestamp_ms and end_timestamp_ms in hotkey deque

timerange

Test Report
Base on v2.11.1
Env: Tencent CVM SA5.4XLARGE32(16C32G), 4*500GB CLOUD_TSSD disks with 4 stripes. cluster mode with 1 shard 2replicas.
Preset data: 20millions keys including string, hash, zset, list and zset types, nearly 900GB. Key length is 32 Bytes.
Enable hotkey analyze with command: hotkey enable 500000 1000000 1.
Test result: CPU usage increase does not exceed 5%,Memory usage increase is negligible. There is no significant decrease in QPS, P95 and P99 latency.

@greatsharp greatsharp changed the title feat(hotkey):hotkey analyze feat(hotkey): hotkey analyze Aug 7, 2025
@greatsharp greatsharp changed the title feat(hotkey): hotkey analyze feat(hotkey): Realtime hotkey analyze Aug 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant