Skip to content

Optimize PrecomputeReplicas for the case when rack count <= 1 #673

@havaker

Description

@havaker

Memory footprint of precomputing replica sets for NTS strategies can be reduced when datacenters have at most one rack.

NTS dc-filtered replica lists computed for rf = n are prefixes of replica lists computed for rf = n+1 (if rack_count <= 1), so they can be "compressed" - only the largest list can be computed.

let rack_count = dc_rep_data.get_rack_count();
let compressed_replica_ring_rf = repfactors.range(..=rack_count).next_back();
let replica_ring_rf_above_rack_count = repfactors.range((rack_count + 1)..);
let produce_replica_ring_iter = |rf| {
let ring_iter = dc_rep_data.get_dc_ring().iter().map(|(token, _)| {
let cur_replicas: Replicas = replication_data
.nts_replicas_in_datacenter(*token, dc_name, rf)
.cloned()
.collect();
(*token, cur_replicas)
});
TokenRing::new(ring_iter)
};
let compressed_replica_ring =
compressed_replica_ring_rf.map(|rf| PrecomputedReplicasRing {
replicas_for_token: produce_replica_ring_iter(*rf),
max_rep_factor: *rf,
});
let above_rack_count_replica_rings = replica_ring_rf_above_rack_count
.map(|rf| (*rf, produce_replica_ring_iter(*rf)))
.collect();
let dc_precomputed_replicas = DatacenterPrecomputedReplicas {
compressed_replica_ring,
above_rack_count_replica_rings,
};

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions