feat(hotkey): Realtime hotkey analyze #3105
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Brief Introduction
Realtime hotkey analyze, implemented via LRU cache and deque, LRU cache is implemented by list and map.
Using one second as a counting cycle, when the key is accessed, accumulate the counter within each second; when crossing a second boundary, determine whether the access counter exceeds the threshold, If it does, identify it as a hot key. Then push the key into the deque.
Commands
Enable hotkey analyze, capacity is the LRU cache capacity, deque_size is the hotkey deque max size, if the key accessed counter is equal or greater than threshold within each second, then identify it as a hotkey, then put the key into the deque.

hotkey disable
Disable the hotkey analyze, it will clear the LRU cache and hotkey deque.
hotkey stats
Display the hotkey stats
hotkey threshold {threshold}
Change the hotkey threshold
hotkey getbykey {key} {begin_timestamp_ms} {end_timestamp_ms}
Search hotkey entries of {key} in hotkey deque
Search hotkey entries which equal to or larger than {threshold} in hotkey deque
hotkey dumplogfile {off|info|warning}
Dump the hotkey entries to log file with level info or warning, off means do not dump.
hotkey timerange {begin_timestamp_ms} {end_timestamp_ms}
Search hotkey entries of it's last_access_time between begin_timestamp_ms and end_timestamp_ms in hotkey deque
Test Report
Base on v2.11.1
Env: Tencent CVM SA5.4XLARGE32(16C32G), 4*500GB CLOUD_TSSD disks with 4 stripes. cluster mode with 1 shard 2replicas.
Preset data: 20millions keys including string, hash, zset, list and zset types, nearly 900GB. Key length is 32 Bytes.
Enable hotkey analyze with command: hotkey enable 500000 1000000 1.
Test result: CPU usage increase does not exceed 5%,Memory usage increase is negligible. There is no significant decrease in QPS, P95 and P99 latency.