Phase 1: async substrate + eval canary - #41
Merged
Conversation
Introduces the async NIF machinery and converts `eval/2` as the
first canary. No behaviour change from the caller's perspective —
`Emily.Native.eval/2` still blocks until the tensor is materialised
and returns `:ok`. Internally the NIF now enqueues the eval onto
the worker thread, returns a fresh ref, and awaits the worker's
reply via `Emily.Native.Async.call/1` on the caller's mailbox. No
BEAM scheduler thread is blocked during MLX work.
The four spikes on the exploration plan (A-D) validated the
plumbing before this landed; see docs/planning/async-worker-exploration.md.
New:
- `c_src/emily/async.hpp` — `async_reply` helper that mints a ref,
captures the caller PID, enqueues onto the worker, and posts
`{ref, {:ok, payload}}` or `{ref, {:error, reason}}` back via
`enif_send`. Exception classification mirrors `fine::nif_impl`'s
catch ladder (`std::invalid_argument` → `:argument`,
`std::runtime_error` → `:runtime`, anything else → `:unknown`).
- `lib/emily/native/async.ex` — `call/1` awaits the reply and
re-raises errors as `ArgumentError` / `RuntimeError` to match the
sync path's semantics.
Changed:
- `c_src/emily/worker.hpp` — added `run_async` (non-blocking
enqueue, task owns error propagation). `run_sync` is unchanged;
other NIFs continue to use it.
- `c_src/emily_nif.cpp` — `eval` renamed to `eval_nif` and
converted to async via `async_reply`. Drops the
`ERL_NIF_DIRTY_JOB_CPU_BOUND` flag (no longer dirty-appropriate;
enqueue is sub-microsecond and the work runs off-scheduler on the
worker thread).
- `lib/emily/native.ex` — `eval/2` wraps `eval_nif/2` with
`Async.call/1`. Public signature and return type unchanged.
Regression tests in `test/emily/async_eval_test.exs` cover:
mailbox hygiene (empty after 1k evals), caller killed mid-flight
(worker remains usable), latency on a resident 4x4 tensor
(averages <20 us locally, budgeted under 1 ms).
Full test suite passes (425 tests, 0 failures).
ausimian
marked this pull request as ready for review
April 18, 2026 23:34
This was referenced Apr 19, 2026
ausimian
added a commit
that referenced
this pull request
Apr 19, 2026
After the Phase 1-3 async-worker migration (#41-#43), NIFs no longer block a scheduler on a sync promise — they enqueue on the worker and the public wrapper awaits a reply via enif_send. Update the README Concurrency model section and the CHANGELOG worker-thread-dispatch bullet so they describe the shipping behaviour.
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First PR of the async-worker migration. See
docs/planning/async-worker-exploration.mdon PR #40 for the full plan and Spike findings.What changes
c_src/emily/async.hpp(new):async_replyhelper that mints a ref, captures the caller PID, enqueues onto the worker thread, and posts{ref, {:ok, payload}}/{ref, {:error, reason}}back viaenif_send. Exception classification matches fine's sync ladder.c_src/emily/worker.hpp: adds non-blockingrun_asyncalongside existingrun_sync. Other NIFs still userun_sync— this PR only converts one canary.c_src/emily_nif.cpp:eval→eval_nif+async_reply. Drops theERL_NIF_DIRTY_JOB_CPU_BOUNDflag.lib/emily/native.ex:eval/2wrapseval_nif/2viaAsync.call/1. Public signature unchanged.lib/emily/native/async.ex(new):call/1awaits the worker's reply and re-raises errors asArgumentError/RuntimeError.What's the user-visible effect
Nothing.
Emily.Native.eval(worker, tensor)still blocks until materialisation and returns:ok. The caller's process now does areceiveinstead of blocking the scheduler on astd::future, so the scheduler is free to run other processes during MLX work.Test coverage
test/emily/async_eval_test.exs:Follow-ups (not in this PR)
c_src/ops/*.cpp(~800 LOC mechanical diff).to_binarywith strategy 3a validated by Spike B.Test plan
mix precommitmix testfull suitemix test --include conformance) before merge to verify no regression