-
Notifications
You must be signed in to change notification settings - Fork 2.6k
Fix RuntimeError in ClusterPubSub sharded message generator #3889
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR fixes a RuntimeError: dictionary changed size during iteration that occurs when subscribing to multiple sharded channels across different cluster nodes. The issue arose because the generator was iterating directly over node_pubsub_mapping.values() while concurrent subscriptions could modify the dictionary.
Key changes:
- Modified
_pubsubs_generator()to create a snapshot of pubsub instances before iteration - Introduced cycle boundaries using
Nonemarkers to properly track iteration completion - Added comprehensive tests validating multi-node sharded pubsub behavior
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| redis/cluster.py | Fixed generator to snapshot mapping values before iteration and added cycle markers |
| tests/test_pubsub.py | Added three cluster-specific tests for multi-node sharded pubsub scenarios |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
d76f170 to
50f8ab3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Fix RuntimeError in ClusterPubSub sharded message generator
Problem:
RuntimeError: dictionary changed size during iterationwhen subscribing to multiple sharded channels on different cluster nodes.Solution: Create a snapshot of node_pubsub_mapping.values() before iteration in _pubsubs_generator() to prevent errors from concurrent modifications.
Changes:
Fixed _pubsubs_generator() in ClusterPubSub class
Added 3 cluster-only tests validating multi-node sharded pubsub and the concurrent mapping changes fix