Skip to content

Fix Redis to reconnect in Sentinel (Chris Staite)#2190

Open
MarcusSorealheis wants to merge 2 commits intomainfrom
redis_reconnect
Open

Fix Redis to reconnect in Sentinel (Chris Staite)#2190
MarcusSorealheis wants to merge 2 commits intomainfrom
redis_reconnect

Conversation

@MarcusSorealheis
Copy link
Collaborator

@MarcusSorealheis MarcusSorealheis commented Feb 26, 2026

The switch from Fred to Redis-rs caused auto-reconnect in Sentinel to not occur when the primary fails over. Instead, we have to manually take care of detecting a ReadOnly error and performing the re-discovery of the primary and re-connecting.

Re-work the Redis store to catch the ReadOnly error in update_data and reconnect. Take a UUID for each connection to ensure we don't spam reconnect attempts from multiple tasks at the same time. Switch the pubsub over to using the ConnectionManager and manage the psubscribe there.

Description

Please include a summary of the changes and the related issue. Please also
include relevant motivation and context.

Fixes # (issue)

Type of change

Please delete options that aren't relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to
    not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Please also list any relevant details for your test configuration

Checklist

  • Updated documentation if needed
  • Tests added/amended
  • bazel test //... passes locally
  • PR is contained in a single commit, using git amend see some docs

This change is Reviewable

chrisstaite-menlo and others added 2 commits February 23, 2026 16:32
The switch from Fred to Redis-rs caused auto-reconnect in Sentinel to not occur when the primary fails over.  Instead, we have to manually take care of detecting a ReadOnly error and performing the re-discovery of the primary and re-connecting.

Re-work the Redis store to catch the ReadOnly error in update_data and reconnect.  Take a UUID for each connection to ensure we don't spam reconnect attempts from multiple tasks at the same time.  Switch the pubsub over to using the ConnectionManager and manage the psubscribe there.
@MarcusSorealheis
Copy link
Collaborator Author

We clearly need to roll something back or fix some test that I broke. My apologies @chrisstaite

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants