Skip to content

Optimize PrecomputeReplicas for the case when rack count <= 1 #673

Open
@havaker

Description

@havaker

Memory footprint of precomputing replica sets for NTS strategies can be reduced when datacenters have at most one rack.

NTS dc-filtered replica lists computed for rf = n are prefixes of replica lists computed for rf = n+1 (if rack_count <= 1), so they can be "compressed" - only the largest list can be computed.

let rack_count = dc_rep_data.get_rack_count();
let compressed_replica_ring_rf = repfactors.range(..=rack_count).next_back();
let replica_ring_rf_above_rack_count = repfactors.range((rack_count + 1)..);
let produce_replica_ring_iter = |rf| {
let ring_iter = dc_rep_data.get_dc_ring().iter().map(|(token, _)| {
let cur_replicas: Replicas = replication_data
.nts_replicas_in_datacenter(*token, dc_name, rf)
.cloned()
.collect();
(*token, cur_replicas)
});
TokenRing::new(ring_iter)
};
let compressed_replica_ring =
compressed_replica_ring_rf.map(|rf| PrecomputedReplicasRing {
replicas_for_token: produce_replica_ring_iter(*rf),
max_rep_factor: *rf,
});
let above_rack_count_replica_rings = replica_ring_rf_above_rack_count
.map(|rf| (*rf, produce_replica_ring_iter(*rf)))
.collect();
let dc_precomputed_replicas = DatacenterPrecomputedReplicas {
compressed_replica_ring,
above_rack_count_replica_rings,
};

Activity

added this to the 1.1.0 milestone on Mar 28, 2023
self-assigned this
on Nov 15, 2023
removed their assignment
on Jul 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

      Participants

      @piodul@Lorak-mmk@havaker

      Issue actions

        Optimize `PrecomputeReplicas` for the case when `rack count <= 1` · Issue #673 · scylladb/scylla-rust-driver