Skip to content

Support Redis Cluster (createCluster, KEYS[] script declarations, shard-routed SCAN/TIME) #63

Description

@bgeils

Summary

y/hub builds its Redis connections with redis.createClient() only (src/stream.js:166 in v0.7.0), which cannot follow MOVED redirects. Managed clustered Redis offerings (AWS MemoryDB, clustered ElastiCache, most enterprise Redis) therefore cannot host y/hub even when all keys are hash-tagged into one slot — the standalone client still needs to discover which node owns the slot.

A second, subtler blocker: the Lua scripts interpolate the worker-stream and disabled-set key names into the script text with NUMBER_OF_KEYS: 1 (src/stream.js:171-276), so Redis Cluster cannot validate the keys the script touches; they must be declared via KEYS[].

What we run in production (offered as a PR)

  • redis.clusterMode?: boolean config; when set, the primary connection is built with createCluster() using the configured URL as root node and the normalized standalone options as per-node defaults (credentials, TLS/SNI, reconnect strategy carry over).
  • The blocking subscription connection is created with primary.duplicate() so it preserves standalone vs cluster mode.
  • Startup validation: cluster mode requires database 0 and a non-empty hash tag in redis.prefix (all y/hub keys must share a slot because _runSub reads every subscribed room in one multi-key XREAD).
  • Every Lua-accessed key declared in KEYS[] (addMessage, trimMessages, disableCompaction, enableCompaction).
  • Keyless / pattern commands routed to the shard that owns the namespace: SCAN-based key listing routed via a hash-tagged key, and TIME routed the same way so minted timestamps share the stream-id clock domain.

Standalone mode remains the default with byte-identical behavior. We also have a test harness that runs the suite against a real three-primary Redis Cluster with a deliberately wrong seed node (to exercise MOVED-following), plus a standalone regression smoke — happy to contribute both.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions