Phase 2: convert all op NIFs to async enif_send - #42
Merged
Conversation
Bulk-converts every worker-bound op NIF across c_src/ops/*.cpp to
the async model introduced in Phase 1. Each NIF now enqueues its
lambda onto the worker thread via emily::async_encoded and returns
a ref immediately; the worker posts {ref, {:ok, result}} or
{ref, {:error, reason}} back to the caller via enif_send. Public
Elixir wrappers in Emily.Native retain their sync return semantics
by awaiting the reply through Emily.Native.Async.call/1.
Files converted (all ops flipped from fine::ResourcePtr<Tensor>
returns under run_sync to fine::Term returns via async_encoded):
- unary.cpp (40 elementwise + round)
- binary.cpp (23 arithmetic/compare/logical/bitwise)
- reduce.cpp (sum/mean/prod/max/min/all/any/logsumexp,
var/std, argmax/argmin, cumulative)
- cast.cpp (astype, bitcast)
- creation.cpp (zeros, ones, full, arange, eye)
- shape.cpp (reshape, transpose, squeeze, expand_dims,
broadcast_to, concatenate, stack, flatten,
tile, swapaxes, pad, repeat)
- sort.cpp (sort, argsort, partition, argpartition, topk)
- misc.cpp (clip, roll, softmax, logcumsumexp, array_equal)
- index.cpp (slice, slice_update, take, where,
take_along_axis, put_along_axis,
scatter_add_axis, gather, scatter, scatter_add)
- linalg.cpp (matmul, tensordot, outer, inner, quantize,
dequantize, quantized_matmul, and CPU-only
decompositions: lu, svd, qr, cholesky, eigh,
solve, solve_triangular)
- fast.cpp (rms_norm, layer_norm, rope, scaled_dot_product_attention)
- random.cpp (split, uniform, normal, randint, bernoulli,
gumbel, categorical; random_key stays sync —
no worker needed)
- fft.cpp (fftn, ifftn, rfftn, irfftn)
- pooling.cpp (window_sum/max/min/product, window_scatter_{max,min})
- conv.cpp (conv_general)
Stays sync (no worker involvement): from_binary, shape, dtype,
to_binary (Phase 3), create_worker, memory introspection NIFs,
random_key. async_encoded itself is added alongside the existing
async_reply helper in emily/async.hpp.
Lambda captures now move their ResourcePtr arguments (std::move)
rather than copying, saving a pair of enif_keep/enif_release per
call. Refcount accounting is unchanged; move is net-neutral.
Tolerance adjustment in test/soak/memory_test.exs: 4 MB -> 16 MB.
The async path has a bounded ~9-10 MB working set (heap-fragment
messages in flight + worker queue lag between enif_send and
lambda destruction) that the sync path did not have. Measured
empirically as a plateau — not a per-iter leak — across 100 to
8000 iters on my local test. A real leak would grow linearly,
still caught by the new ceiling.
Dialyzer ignore entries for invalid_contract warnings in
backend.ex, quantization.ex, and quantized_weight.ex dropped their
hardcoded line numbers: dialyzer reports at different positions
now because the widened inferred typing from Async.call flows
through to these callers. The warnings themselves are unchanged.
Full test suite: 425 tests, 0 failures.
Conformance (tiny-random HuggingFace models): 442 tests, 0 failures.
Dialyzer: 0 new warnings.
ausimian
marked this pull request as ready for review
April 19, 2026 00:07
This was referenced Apr 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second PR of the async-worker migration. Phase 1 merged as #41; this PR converts every worker-bound op NIF in `c_src/ops/*.cpp` to the async pattern.
Scope
15 files touched on the C++ side, one new helper (`emily::async_encoded`), and `lib/emily/native.ex` updated to wrap every NIF with `Async.call/1`. The public Elixir surface is unchanged — callers see the same blocking semantics.
Ops converted (everything that takes a worker):
Stays sync: `from_binary`, `shape`, `dtype`, `to_binary` (Phase 3), `create_worker`, memory introspection NIFs, `random_key`.
What's the user-visible effect
Same as Phase 1 — none at the API level. Every `Emily.Native.*` call still blocks until the result is ready and returns the tensor. Internally, every op now runs without blocking a BEAM scheduler.
Notable detail: memory_test tolerance bump
`test/soak/memory_test.exs` tolerance raised from 4 MB to 16 MB. The async path has a bounded ~9-10 MB working set (heap-fragment messages in flight + worker queue lag) that the sync path didn't. Measured as a plateau — not a leak — across 100 to 8000 iters. A real per-iter leak would grow past 16 MB and still fail.
Dialyzer
The pre-existing `:invalid_contract` warnings in `backend.ex`, `quantization.ex`, and `quantized_weight.ex` now report at different line numbers because the widened inferred return type from `Async.call` (`term()`) propagates through the callers. Hardcoded line numbers in `.dialyzer_ignore.exs` removed. Behaviour unchanged — dialyzer still skips 0 new warnings.
Test plan