What happened:
The hermetic integration tests under test/integration/epp/ flake intermittently on CI with Server failed to bind port 127.0.0.1:<port> errors, causing lint-and-test to fail on unrelated PRs.
Observed in at least:
Failure signature (different sub-tests, same root cause):
Error: Condition never satisfied
Messages: Server failed to bind port 127.0.0.1:<random-port>
What you expected to happen:
Hermetic integration tests should pass deterministically on CI without flaking due to port allocation races.
How to reproduce it (as minimally and precisely as possible):
Flaky — run lint-and-test repeatedly on a PR; intermittently a sub-test of TestFullDuplexStreamed_* fails with the bind-port error within its 10s budget.
Anything else we need to know?:
Likely cause: the random-port allocation in the hermetic test fixture races with another process / parallel sub-test on the runner; the 10s bind retry budget expires before the port frees up.
Suggested follow-up:
- Audit the port-allocation helper in the hermetic harness; prefer a listen-then-pass-fd pattern over allocate-then-bind to avoid the race.
- Or: increase the retry/timeout budget on bind failures.
Environment:
- CI: GitHub Actions,
ubuntu-latest runner (default lint-and-test workflow).
- Not environment-specific to a user setup; reproduces on CI only.
What happened:
The hermetic integration tests under
test/integration/epp/flake intermittently on CI withServer failed to bind port 127.0.0.1:<port>errors, causinglint-and-testto fail on unrelated PRs.Observed in at least:
TestFullDuplexStreamed_KubeInferenceObjectiveRequest/Standalone-NoCRD/no_backend_pods_available#01https://github.com/llm-d/llm-d-inference-scheduler/actions/runs/25718109453/job/75516805509
TestFullDuplexStreamed_GRPC_KubeInferenceObjectiveRequest/do_not_shed_requests_by_defaulthttps://github.com/llm-d/llm-d-inference-scheduler/actions/runs/25633770069/job/75241972222
Failure signature (different sub-tests, same root cause):
What you expected to happen:
Hermetic integration tests should pass deterministically on CI without flaking due to port allocation races.
How to reproduce it (as minimally and precisely as possible):
Flaky — run
lint-and-testrepeatedly on a PR; intermittently a sub-test ofTestFullDuplexStreamed_*fails with the bind-port error within its 10s budget.Anything else we need to know?:
Likely cause: the random-port allocation in the hermetic test fixture races with another process / parallel sub-test on the runner; the 10s bind retry budget expires before the port frees up.
Suggested follow-up:
Environment:
ubuntu-latestrunner (defaultlint-and-testworkflow).