feat: measure link latency and prefer low-latency paths for ring hosts - #2254
Open
abendrothj wants to merge 1 commit into
Open
feat: measure link latency and prefer low-latency paths for ring hosts#2254abendrothj wants to merge 1 commit into
abendrothj wants to merge 1 commit into
Conversation
The reachability probe that already runs every 10s now records the round-trip time of its successful request. The measured latency is carried on SocketConnection topology edges (excluded from edge identity so probe jitter does not churn the topology; edges are republished only on a material change) and MLX ring host selection prefers the lowest measured latency, falling back to the interface-type heuristic for unmeasured links. Previously ring hosts were chosen purely by interface-type label, so a degraded or misclassified link (e.g. traffic falling back to a VPN path after a crash/restart while the Thunderbolt bridge was still down) could be selected over a healthy sub-millisecond link. Addresses the low-latency link preference part of exo-explore#1723.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses the link-preference half of #1723 (P2P mesh falling back to a slow path after crash/restart).
Problem
MLX ring host selection (
find_ip_prioritised) chooses between candidate IPs purely by interface-type label, with aTODO: Profile and get actual connection speeds. After the crash/restart scenario in #1723, inference traffic can land on a 40–70 ms path (Tailscale/WAN) even when a sub-millisecond Thunderbolt link is available — the label heuristic has no way to notice, and VPN interfaces classify asunknown.Changes
check_reachabilityalready makes an HTTP request to each peer interface every 10s; it now records the round-trip time of the successful request instead of discarding it.SocketConnectiongainslatency_ms. Identity is explicitly IP+port (matching the existing IP-based__hash__), so measurement jitter does not make otherwise-identical edges unequal. The worker republishes an edge (delete + create) only when latency changes materially — both >2 ms absolute and a factor-of-two — so fast-link jitter never churns the topology.find_ip_prioritised(ring=True)now sorts by measured latency first, using the interface-type priority (thunderbolt first) only as tie-break/fallback for unmeasured links. The RDMA coordinator path (ring=False) is unchanged. This resolves the TODO and fixes the stale docstring (which described the coordinator priority, not the ring one).With this, after a restart where the Thunderbolt bridge comes back late: the periodic probe measures both paths, the topology reflects real latencies within ~10s, and the next placement picks the fast link by measurement rather than by label.
Not in scope
exo_pyo3_bindings; that code was replaced by the zenoh stack in libp2p -> zenoh #2132.Tests
find_ip_prioritised: measured low latency beats a thunderbolt-labelled slow link; latency outranks labels; unmeasured links fall back to label priority; measured links beat unmeasured ones.latency_changed_materially: first measurement, sub-noise-floor jitter, sub-factor-two changes, and genuine degradation (0.8 ms → 45 ms).basedpyright, ruff, and the master/shared/utils/worker-unit test suites pass locally.
Refiled from #2215, which was closed unintentionally.