Skip to content

feat(worker): bounded queue, drop-with-error stop, off-scheduler join - #132

Merged
ausimian merged 1 commit into
mainfrom
fix/issue-121-122-worker-lifecycle
May 31, 2026
Merged

feat(worker): bounded queue, drop-with-error stop, off-scheduler join#132
ausimian merged 1 commit into
mainfrom
fix/issue-121-122-worker-lifecycle

Conversation

@ausimian

Copy link
Copy Markdown
Owner

Summary

Closes #121 and #122 — reworks the per-stream worker so teardown never blocks a
BEAM scheduler and the async queue can't grow without bound. One PR, since both
rework WorkerThread's internals.

#121 — non-blocking teardown

The resource destructor used to drain the entire queue and join() the OS
thread inline, so collecting a busy stream during GC stalled a scheduler.

  • Worker state moves into a shared_ptr<State>, so the thread safely outlives
    the BEAM resource.
  • On stop the worker cancels its queued tasks (each replies
    {:error, :stopped}) instead of running them, so a join waits for at most the
    one in-flight kernel — never the backlog.
  • A dedicated Reaper joins finished worker threads off-scheduler; it tracks
    live workers so the NIF unload callback can stop+join stragglers (no
    thread is ever abandoned — detaching isn't allowed under the BEAM). The
    destructor just hands the thread to the reaper, non-blocking.
  • Emily.Stream.close/1 releases a stream's worker deterministically off the GC
    path (a plain, non-blocking NIF — the reaper does the join).

#122 — back-pressure, cancellation, timeout

  • Bounded queue (config :emily, worker_queue_limit:, default 8192);
    run_async rejects when full, so a flood can't pin host/GPU memory. Every op
    is awaited, so a process holds ≤1 queued item — the cap only trips under a
    many-process flood.
  • A stopped/dropped worker replies {:error, :stopped} to every queued
    caller instead of leaving it blocked forever.
  • Opt-in await timeout (config :emily, await_timeout:, default :infinity
    no behaviour change) and Emily.Native.worker_queue_depth/1 for observability.
  • Removed the dead run_sync.

De-scoped (as discussed): true in-flight cancellation (can't interrupt a Metal
kernel) and liveness-check-before-reply (enif_send already no-ops on dead pids).

Verification

  • mix precommit green, stable across multiple seeds (543 tests + 39 doctests +
    79 properties, 0 failures, credo --strict clean).
  • New test/emily/worker_lifecycle_test.exs: non-positive limit rejected,
    idempotent stop_worker, drop-cancels-with-:stopped-and-never-hangs,
    back-pressure rejection, Stream.close/1, and a churn test
    (100 create/drop busy streams) asserting the reaper reaps without crashing or
    leaking.
  • Ran a 400-worker churn stress (32 concurrent, fire-and-forget evals, half
    GC'd / half closed): no crash, allocator memory returned to baseline (delta 0).

Reworks the per-stream worker so teardown never blocks a BEAM scheduler
and a flood of operations can't grow the queue without bound.

- Move worker state into a shared_ptr<State> so the thread outlives the
  BEAM resource. On stop the worker cancels its queued tasks (each posts
  {:error, :stopped}) instead of draining them, and a dedicated Reaper
  joins finished worker threads off-scheduler — it tracks live workers
  and is itself joined in the NIF unload callback. The resource
  destructor just hands the thread to the reaper (non-blocking).
- Bound the queue (config :emily, worker_queue_limit:, default 8192);
  run_async rejects when full so memory can't grow without limit.
- Add Emily.Stream.close/1 for deterministic teardown, an opt-in await
  timeout (config :emily, await_timeout:, default :infinity), and
  Emily.Native.worker_queue_depth/1. Remove the dead run_sync.

Closes #121
Closes #122
@ausimian
ausimian merged commit a02734a into main May 31, 2026
3 checks passed
@ausimian
ausimian deleted the fix/issue-121-122-worker-lifecycle branch May 31, 2026 10:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Avoid BEAM scheduler blocking during MLX worker resource destruction

1 participant