M14.5: worker-thread dispatch for vendored MLX - #29
Merged
Conversation
The SIGSEGV from M14's stream-per-process isn't fixed by the safe_eval mutex — the race is in Metal command encoder state outside of eval. Building MLX from source (for native thread- safety) needs to land before M15 linalg, not after.
Replace stream-index NIF convention and global eval mutex with per-stream dedicated OS threads. Each WorkerThread owns an MLX stream and its Metal CommandEncoder; NIFs dispatch via run_sync (promise/future). Eliminates thread-local CommandEncoder mismatch under BEAM process migration and enables concurrent Metal dispatch. MLX vendored as git submodule (vendor/mlx), built from source as libmlx.a with metallib staged into priv/.
USE_SYSTEM_FMT=ON required a system-installed fmt library, which the CI runner doesn't have. Remove the flag so MLX's cmake fetches fmt itself.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
safe_evalmutex with per-stream dedicated OS threads. EachWorkerThread(C++) owns an MLX stream and its MetalCommandEncoder; NIFs dispatch viarun_sync(promise/future). Eliminates thread-localCommandEncodermismatch under BEAM process migration and enables concurrent Metal dispatch across multiple workers.vendor/mlx), buildlibmlx.avia cmake, statically link into the NIF, stagemlx.metallibintopriv/. No prebuilt download step.Emily.MlxStreamGenServer: owns the defaultWorkerThreadresource under the application supervisor.default_worker/0caches the worker ref in the process dictionary. User-created streams (Emily.Stream.new/1) get their own worker withenif_monitor_processcleanup.submodules: recursive, remove stale prebuilt cache step, add.gitmodulesto build cache key.Test plan
mix precommitpasses (compile --warnings-as-errors, format, credo --strict, test)*_fullsuites: training_full, qwen3_full, qwen3_quant_full, vit_full, whisper_full, fast_kernels_full🤖 Generated with Claude Code