Skip to content

Optimise probes filter for small limits #876

Description

@alexey-yarmosh

80% of measurement requests today use limit: 1, we can optimise for that.

On every request today:

  1. Filter all probes.
  2. Shuffle all.
  3. Group by asn+city.
  4. Build a list by picking a probe from every group until all groups exhaust.
  5. Pick the required number of probes from the beginning. Just first probe in most of the cases.

Instead:

On every probes list update:

  1. Build probe groups by asn+city.
  2. Sort groups desc by size.

On every request:

  1. Filter groups by checking its first probe (since probes are the same in one group).
  2. Pick the required number of probes from groups using random round-robin pick.

I'm oversimplifying here, but the idea looks valid for me.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions