Summary
GUEST_READY_TIMEOUT is a hard-coded 30 s with no environment or config override, so a box
that boots slowly on a contended host fails permanently with no available mitigation:
Box uPKTB5hOHtZf failed to start: timeout after 30s
Evidence at T+30s:
• shim_alive = true
• console_bytes = 0
• ready_socket_exists = true
• likely_cause = guest agent never wrote to console (init or vsock plumbing broken)
src/boxlite/src/litebox/init/tasks/guest_connect.rs:110:
const GUEST_READY_TIMEOUT: Duration = Duration::from_secs(30);
It has exactly two references — the definition and the call site at :85. There is no env var,
no AdvancedOptions field, and no per-box knob, so a caller on a loaded machine has no way to
raise it. The doc comment says it is "exposed as a constant so tests can call
wait_for_guest_ready with a short timeout", i.e. tests can shorten it but production callers
cannot lengthen it.
How it showed up
Running the full Python SDK integration suite on macOS (make test:integration:python, which
the pre-push hook runs), tests/test_tcp_filter.py::TestEdgeCases::test_multiple_allowed_hosts
intermittently fails with the message above. It failed 3 times during full-suite runs.
Reproduction attempts — what does and does not trigger it
I could not reproduce it in isolation. Recording the negative results so the next person does
not repeat them:
| Condition |
Result |
| Standalone, idle machine (3 consecutive runs) |
pass — 15.9 s, 15.6 s, 17.6 s |
| Standalone, 24 CPU spinners (load avg 20.6 / 18 cores) |
pass — 28.4 s |
Standalone, 40 CPU spinners + 4 dd I/O loops (load avg 52.9) |
pass — 69.2 s |
Whole test_tcp_filter.py file, 21 boxes back to back |
pass — 21/21, 179 s |
| During / immediately after the full integration suite |
fails, 3 occurrences |
Two things worth noting:
- Wall time degrades sharply with host load (15.9 s idle → 28.4 s → 69.2 s), so box startup is
clearly contention-sensitive; but pure CPU/IO pressure alone did not push the guest-ready
handshake past 30 s here.
- The failure signature is
console_bytes = 0 — the guest wrote nothing, which does not look
like ordinary slowness. Something about running the whole suite (hundreds of boxes) is the
trigger; I did not isolate what.
Why the constant is worth fixing regardless
Whatever the underlying trigger turns out to be, a user or CI job that hits it today has no
escape hatch: they cannot raise the deadline to confirm "slow" versus "hung", and the same
30 s applies to a warm laptop and a heavily loaded CI box alike.
Suggested: make it overridable (env var and/or an AdvancedOptions field) with the current
30 s as the default, keeping the existing diagnostic block on expiry.
Related
Environment
- macOS 27.0.0, arm64, 18 cores
- boxlite 0.9.7, local (non-REST) runtime,
sdks/python built from source
Summary
GUEST_READY_TIMEOUTis a hard-coded 30 s with no environment or config override, so a boxthat boots slowly on a contended host fails permanently with no available mitigation:
src/boxlite/src/litebox/init/tasks/guest_connect.rs:110:It has exactly two references — the definition and the call site at
:85. There is no env var,no
AdvancedOptionsfield, and no per-box knob, so a caller on a loaded machine has no way toraise it. The doc comment says it is "exposed as a constant so tests can call
wait_for_guest_readywith a short timeout", i.e. tests can shorten it but production callerscannot lengthen it.
How it showed up
Running the full Python SDK integration suite on macOS (
make test:integration:python, whichthe pre-push hook runs),
tests/test_tcp_filter.py::TestEdgeCases::test_multiple_allowed_hostsintermittently fails with the message above. It failed 3 times during full-suite runs.
Reproduction attempts — what does and does not trigger it
I could not reproduce it in isolation. Recording the negative results so the next person does
not repeat them:
ddI/O loops (load avg 52.9)test_tcp_filter.pyfile, 21 boxes back to backTwo things worth noting:
clearly contention-sensitive; but pure CPU/IO pressure alone did not push the guest-ready
handshake past 30 s here.
console_bytes = 0— the guest wrote nothing, which does not looklike ordinary slowness. Something about running the whole suite (hundreds of boxes) is the
trigger; I did not isolate what.
Why the constant is worth fixing regardless
Whatever the underlying trigger turns out to be, a user or CI job that hits it today has no
escape hatch: they cannot raise the deadline to confirm "slow" versus "hung", and the same
30 s applies to a warm laptop and a heavily loaded CI box alike.
Suggested: make it overridable (env var and/or an
AdvancedOptionsfield) with the current30 s as the default, keeping the existing diagnostic block on expiry.
Related
a vitest 120 s hook timeout, i.e. a test-framework limit; this issue is about boxlite's own
internal deadline, which no caller can configure.
Environment
sdks/pythonbuilt from source