Skip to content

Commit c4efa33

Browse files
committed
fix(provisioning,server): exit retry loop on Accepted outcome; await runtime in foreground mode
Two live-AWS regressions, both rooted in earlier work on this branch. Provisioning retry loop spawned redundant fleets for async providers. Async providers (EC2Fleet/SpotFleet REQUEST/MAINTAIN) return an Accepted outcome with zero instances on the first call — fulfilment finishes out of band and the status poller reconciles it. The retry loop interpreted "0 fulfilled + not final" as "retry with a new batch", so each accepted request grew a second and third fleet on the side. Polling later saw one healthy fleet and N-1 empty fleets and flipped the request to complete_with_error. Break out of the loop as soon as an Accepted outcome lands so the single accepted fleet drives the rest. No in-tree provider emits RequiresFollowUp today; if one does later, persist_acquiring becomes reachable again. Foreground server start re-entered asyncio.run inside the CLI's own asyncio.run(main()). daemon_mod.start called _spawn_runtime which called asyncio.run, raising RuntimeError "asyncio.run cannot be called from a running event loop". The exception was caught and returned exit_code=1, so the subprocess died silently inside 30s and every rest_api live test hit the "ORB server failed to start" timeout. handle_server_start now drives the pid + token lifecycle directly in foreground mode and awaits the runtime coroutine in the existing event loop. The retry-loop persist_acquiring tests guarded a write that only ran between retry attempts. With Accepted no longer triggering a retry, the persist path is unreachable; the post-call update_request_from_provisioning in request_creation_handlers already stamps IN_PROGRESS (or PARTIAL / FAILED) once the orchestrator returns. Replaced with a test that verifies the Accepted outcome exits the loop after exactly one provider call.
1 parent 4ab16a7 commit c4efa33

5 files changed

Lines changed: 48718 additions & 117 deletions

File tree

0 commit comments

Comments
 (0)