fix(datalayer): sync cross-replica state from one goroutine, not one per endpoint - #8
Open
Lucas-Fernandes-Martins wants to merge 1 commit into
Open
Conversation
…per endpoint Addresses review feedback on llm-d#2310. The PollingDispatcher is per endpoint because it establishes and polls a connection per endpoint. Cross-replica syncing does neither: it reads local state and hands it to the syncer, so one goroutine iterating the endpoints is sufficient. Doing it per endpoint was also leaking. The goroutine only returned on ctx.Done(), and NewEndpoint receives the process-lifetime parentCtx, so ReleaseEndpoint stopped the collector while the ticker kept publishing state for an endpoint that no longer existed. Iterating the live set each tick removes both the leak and the stale writes, with no teardown to get wrong. Collector already documents itself as running "data collection for a single endpoint" and is handed that endpoint by Start, so it now keeps it. That makes the existing collector registry enumerable without a wrapper type or a second registry to keep in step.
Lucas-Fernandes-Martins
force-pushed
the
lfm/cross-replica-single-goroutine
branch
from
August 10, 2026 16:38
48ab8aa to
cc3bd7c
Compare
Lucas-Fernandes-Martins
marked this pull request as ready for review
August 10, 2026 16:42
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Review feedback on the upstream PR llm-d/llm-d-router#2310, which introduced the independent cross-replica sync ticker already on
lfm/combined-fixes.ahg-gasked why the sync runs a goroutine per endpoint:He's right, and the per-endpoint design was also leaking. The goroutine only returns on
ctx.Done(), butNewEndpointis called with the process-lifetimeparentCtx(datastore.go:438), soReleaseEndpointstopped the collector while the sync ticker kept publishing state for an endpoint that no longer exists — a goroutine leak plus stale cross-replica writes. Copilot flagged the same thing independently.A single loop over the live endpoints fixes both at once: no per-endpoint goroutine to leak, and a released endpoint simply stops being visited.
Summary
runCrossReplicaSync(ctx)goroutine, started from the existingRuntime.Start(ctx, mgr), which already receives a context and runs immediately afterConfigure(wherecrossReplicaPubis built). Each tick it iterates the live endpoints and dispatches.Collectorkeeps the endpoint it was started with. It already documents itself as running "data collection for a single endpoint" andStartis handed that endpoint, so this makes the existing registry enumerable viacollectorManager.RangeEndpoints— no wrapper type, and no second registry that could drift out of step with the collectors.No change to what is published or how often; only how many goroutines do it, and that removed endpoints stop.
Reviewer guide
pkg/epp/datalayer/runtime.goNewEndpointtoStart, andrunCrossReplicaSyncdrops itsepparameter in favour ofRangeEndpoints.pkg/epp/datalayer/collector.goepfield set byStart, plus anEndpoint()accessor under the existing mutex.pkg/epp/datalayer/manager.goRangeEndpointsonly;Register,RemoveandStopAllare untouched.pkg/epp/datalayer/cross_replica_publisher_test.goWorth a second opinion on placing the launch in
Start: it is the onlyRuntimemethod taking a lifetime context, and it runs afterConfigure, socrossReplicaPubis always set by then — but the ordering is load-bearing and the nil check is what keeps it safe if that ever changes.Note that
RangeEndpointsonly sees collectors that have been started. A collector is registered a few lines beforeStartis called, so a brand-new endpoint can miss at most one tick — before which it has nothing worth publishing anyway.Test plan
go build ./...,gofmt -l, andgo vet ./pkg/epp/datalayer/...all clean.go test -race ./pkg/epp/datalayer/...green, including the pre-existing cross-replica publisher and collector tests.TestRunCrossReplicaSync_PublishesAllEndpoints— one loop publishes every registered endpoint.TestRunCrossReplicaSync_StopsAfterRelease— regression guard for the leak: publishing stops once the endpoint is removed from the registry.Note on the third review comment
Copilot also flagged
crossReplicaPublisherTypeas an unused const that would fail to compile. That is stale — the symbol no longer exists anywhere in the tree (grepfinds no references) and the build is clean. No change needed.