You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
| Hash ring (with vnodes) | O(log N) binary search over N points; O(1) with specialized structures | O(log N) to insert/remove points | O(N) points | Yes: take next R distinct successors; O(log N + R) |
21
-
| Rendezvous | O(N) score per node; top-1 | O(1) (no state to rebalance) | O(N) node list | Yes: pick top R scores; O(N log R) |
| Hash ring (with vnodes) | O(log N): binary search over N points; O(1): with specialized structures | O(log N) | O(N) | O(log N + R): Take next R distinct successors |
22
+
| Rendezvous | O(N): max score | O(1) | O(N) node list | O(N log R): pick top R scores |
- Hash ring: replicate by walking clockwise to the next R distinct nodes. Virtual nodes help spread replicas evenly and avoid hotspots.
30
+
- Hash ring: replicate by walking clockwise to the next R distinct nodes. Virtual nodes help spread replicas more evenly. Replicas are not independently distributed.
29
31
- Rendezvous hashing: replicate by selecting the top R nodes by score for the key. This naturally yields R distinct owners and supports weights.
30
-
- Jump consistent hash: the base function returns one bucket. Replication can be achieved by hashing (key, replica_index) and collecting R distinct buckets; this is simple but lacks the single-pass global ranking HRW provides.
32
+
- Jump consistent hash and variatns: the base function returns one bucket. Replication can be achieved by hashing (key, replica_index) and collecting R distinct buckets; this is simple but loses the consistency property!
33
+
- ConsistentChooseK: Faster and more memory efficient than all other solutions.
31
34
32
35
Why replication matters
33
36
- Tolerates node failures and maintenance without data unavailability.
34
37
- Distributes read/write load across multiple owners, reducing hotspots.
35
38
- Enables fast recovery and higher tail-latency resilience.
36
39
37
-
## N-Choose-R replication
38
-
39
-
We define the consistent `n-choose-k` replication as follows:
40
-
41
-
1. For a given number `n` of nodes, choose `k` distinct nodes `S`.
42
-
2. For a given `key` the chosen set of nodes must be uniformly chosen from all possible sets of size `k`.
43
-
3. When `n` increases by one, exactly one node in the chosen set will be changed.
44
-
4. and the node will be changed with probability `k/(n+1)`.
40
+
## ConsistentChooseK algorithm
45
41
46
-
For simplicity, nodes are represented by integers `0..n`.
47
-
Given `k` independent consistent hash functions `consistent_hash(key, k, n)` for a given `key`, the following algorithm will have the desired properties:
42
+
The following functions summarize the core algorithmic innovation as a minimal Rust excerpt.
43
+
`n` is the number of nodes and `k` is the number of desired replica.
44
+
The chosen nodes are returned as distinct integers in the range `0..n`.
`consistent_choose_k` makes `k` calls to `consistent_choose_max` which calls `consistent_hash` another `k` times.
61
+
In total, `consistent_hash` is called `k * (k+1) / 2` Utilizing a `O(1)` solution for `consistent_hash` leads to a `O(k^2)` runtime.
62
+
This runtime can be further improved by replacing the max operation with a heap where popped elements are updated according to the new arguments `n` and `k`.
63
+
With this optimization, the complexity reduces to `O(k log k)`.
64
+
With some probabilistic bucketing strategy, it should be possible to reduce the expected runtime to `O(k)`.
65
+
For small `k` neither optimization is probably improving the actual performance though.
66
+
67
+
The next section proves why this simple code works.
68
+
69
+
## N-Choose-R replication
70
+
71
+
We define the consistent `n-choose-k` replication as follows:
72
+
73
+
1. For a given number `n` of nodes, choose `k` distinct nodes `S`.
74
+
2. For a given `key` the chosen set of nodes must be uniformly chosen from all possible sets of size `k`.
75
+
3. When `n` increases by one, exactly one node in the chosen set will be changed.
76
+
4. and the node will be changed with probability `k/(n+1)`.
77
+
78
+
In the remainder of this section we prove that the `consistent_choose_k` algorithm satisfies those properties.
79
+
63
80
Let's define `M(k,n) = consistent_choose_max(_, k, n)` and `S(k, n) := consistent_choose_k(_, k, n)` as short-cuts for some arbitrary fixed `key`.
81
+
We assume that `consistent_hash(key, k, n)` computes `k` independent consistent hash functions.
64
82
65
83
Since `M(k, n) < n` and `S(k, n) = {M(k, n)} ∪ S(k - 1, M(k, n))` for `k > 1`, `S(k, n)` constructs a strictly monotonically decreasing sequence. The sequence outputs exactly `k` elements which therefore must all be distinct which proves property 1 for `k <= n`.
0 commit comments