Skip to content

Scheduler worker tasks and remote sandbox requests can block indefinitely on unresponsive HTTP calls #1236

Description

@zhongwen666

Problem

Two HTTP call sites lack timeouts and can hang indefinitely when a worker or remote sandbox server becomes unresponsive:

  1. BaseTask.run_on_worker in rock/admin/scheduler/task_base.py is awaited without any timeout inside the scheduler's bounded-concurrency runner. A single stuck worker holds its semaphore slot forever, starving the scheduler.
  2. RemoteSandboxRuntime._request in rock/sandbox/remote_sandbox.py issues requests.post without a timeout, so a hung server blocks the calling thread indefinitely.

Impact

Stuck workers/servers can wedge the admin scheduler and SDK callers, with no recovery until the process is restarted.

Expected

Both call sites should enforce a timeout and surface the failure instead of blocking forever.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions