Skip to content

Batch SSUBSCRIBE by Redis Cluster slot to avoid CROSSSLOT errors #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 12, 2025

Conversation

bellatoris
Copy link

@bellatoris bellatoris commented Aug 1, 2025

…ered bursts

Pull Request check-list

Please make sure to review and check all of these items:

  • Does $ tox pass with this change (including linting)?
  • Do the CI tests pass with this change (enable it first in your forked repo and wait for the github action build to finish)?
  • Is the new or changed code fully tested?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?
  • Is there an example added to the examples folder (if applicable)?
  • Was the change added to CHANGES file?

NOTE: these things are not required to open a PR and can be done
afterwards / while the PR is open.

Description of change

This PR changes how SSUBSCRIBE commands are issued in Redis Cluster mode.

Previously, each channel was subscribed individually in a loop, resulting in one SSUBSCRIBE per channel. In Redis Cluster, this increases the risk of CROSSSLOT errors and introduces unnecessary overhead when multiple channels belong to the same hash slot.

This update improves the logic by grouping channels by their Redis hash slot, and sending a single SSUBSCRIBE per slot group, subscribing to all relevant channels at once.

Motivation

  • Redis Cluster requires that all keys in a multi-key command reside in the same hash slot.
    Subscribing to multiple channels across slots in a single command causes CROSSSLOT errors.
  • The previous implementation avoided this by sending individual commands per channel, but this is inefficient.
  • Batching subscriptions by slot reduces command overhead and aligns with Redis Cluster's intended usage pattern.

Changes

  • Group channels by their hash slot (key_slot(self.encoder.encode(channel)))
  • Issue one SSUBSCRIBE command per slot group (i.e., SSUBSCRIBE channel1 channel2 ...)

Example

channels_by_slot = defaultdict(list)
for channel in new_s_channels:
    slot = key_slot(self.encoder.encode(channel))
    channels_by_slot[slot].append(channel)

@bellatoris bellatoris requested a review from longcoding August 1, 2025 14:58
@bellatoris bellatoris force-pushed the doogie/fix-pubsub-read branch from 99b97f5 to 61b8604 Compare August 2, 2025 01:48
Copy link

@daniel-lee-sb daniel-lee-sb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bellatoris bellatoris force-pushed the doogie/fix-pubsub-read branch from 61b8604 to 559c6b8 Compare August 7, 2025 04:56
@bellatoris bellatoris force-pushed the doogie/fix-pubsub-read branch from 559c6b8 to 08d9608 Compare August 8, 2025 05:21
@bellatoris bellatoris changed the title choose a node randomly when subscribing Batch SSUBSCRIBE by Redis Cluster slot to avoid CROSSSLOT errors Aug 12, 2025
@bellatoris bellatoris merged commit 7b31ee3 into 4.4 Aug 12, 2025
2 checks passed
@bellatoris bellatoris deleted the doogie/fix-pubsub-read branch August 12, 2025 01:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants