Goal
Replace the two hardcoded service-launch timeouts with a single operator-tunable value, so the agent and the kernel runner always derive their budgets from the same number instead of two constants that can silently drift apart.
Background
BA-7258 restored the ordering by raising the agent's reply timeout to a constant that exceeds the kernel runner's launch timeout. That is the minimal, backportable fix, but it leaves two problems open:
- The budget cannot be tuned per deployment. Sites whose images bind slowly under CPU or IO contention have no knob.
- The two constants live in packages that cannot import each other (
ai.backend.kernel runs inside the container), so nothing prevents a future edit to one of them from re-inverting the pair.
The proposed shape, from the BA-7258 report: add a key to the existing [kernel-lifecycles] section of agent.toml, send it in the start-service payload, and have the kernel runner honour it. Older runners ignore the extra key, so this stays backward compatible.
Acceptance criteria
- The launch budget is configurable through
[kernel-lifecycles] in agent.toml and is etcd-overridable like its neighbours.
- The agent and the kernel runner derive their budgets from that one value, with the agent's reply timeout strictly greater than the runner's launch timeout.
- A regression test asserts the ordering so the pair cannot silently invert.
- A start-service payload without the key keeps the pre-existing runner behaviour, so containers created before the upgrade are unaffected.
- The generated
configs/agent/sample.toml documents the new key.
Notes
- The kernel runner terminates the service process when its launch timeout expires, so lowering the runner's budget is a behaviour change, not just a tuning knob. Pick the default with that in mind.
- The kernel runner ships bind-mounted from the agent's own venv, so no image rebuild is needed, but the mount is fixed at container creation: the change only reaches newly created containers.
- The effective ceiling on the agent's side is the client's own request timeout, which is outside this repository.
Related
- BA-7258 — the minimal fix this builds on (backported to 26.8 and 26.4).
- BA-7260 — start-service replies are not correlated with requests; touches
feed_start_service in the same file.
Story points
1
JIRA Issue: BA-7322
Goal
Replace the two hardcoded service-launch timeouts with a single operator-tunable value, so the agent and the kernel runner always derive their budgets from the same number instead of two constants that can silently drift apart.
Background
BA-7258 restored the ordering by raising the agent's reply timeout to a constant that exceeds the kernel runner's launch timeout. That is the minimal, backportable fix, but it leaves two problems open:
ai.backend.kernelruns inside the container), so nothing prevents a future edit to one of them from re-inverting the pair.The proposed shape, from the BA-7258 report: add a key to the existing
[kernel-lifecycles]section ofagent.toml, send it in the start-service payload, and have the kernel runner honour it. Older runners ignore the extra key, so this stays backward compatible.Acceptance criteria
[kernel-lifecycles]inagent.tomland is etcd-overridable like its neighbours.configs/agent/sample.tomldocuments the new key.Notes
Related
feed_start_servicein the same file.Story points
1
JIRA Issue: BA-7322