[Bug] check_error() throws std::runtime_error from completion handler → uncaught SIGABRT on iPhone 15 Pro / iOS 26.4 (Qwen3 1.7B 4bit)

[kitadesign]

Summary

mlx::core::gpu::check_error() throws std::runtime_error from a MTL::CommandBuffer::addCompletedHandler callback when the command buffer status is MTL::CommandBufferStatusError. The exception is uncaught at the Swift boundary, which immediately calls std::terminate() → abort() — the entire app dies (SIGABRT, bug type 309) with no chance for Swift code to recover.

I am hitting this reliably on iPhone 15 Pro (A17 Pro) / iOS 26.4.2 after roughly 2 minutes of continuous LLM inference (Qwen3 1.7B 4bit). The crash always fires from com.Metal.CompletionQueueDispatch, often while the device is screen-locked.

This appears to be distinct from the existing reports:

• [Bug] MLX 0.25.2 causing the UI rendering to be stuck and occasionally crash. #237 (MLX 0.25.2 causing the UI rendering to be stuck and occasionally crash) — the symptom there is Command queue creation failed from Stream/Device over-creation. My stack trace never enters Stream creation; it dies inside the completion handler itself.
• [Bug] Random Address size fault crashes on iOS devices #121 (Random Address size fault crashes on iOS devices, CLOSED) — that one was scoped to pre-A14 GPUs (iPhone SE2 / iPad 8, etc.). I am on A17 Pro, the most current Apple GPU.

Environment

Item	Value
Device	iPhone 15 Pro (iPhone16,1, A17 Pro)
OS	iPhone OS 26.4.2 (build 23E261)
mlx-swift	0.29.1
mlx-swift-examples	2.29.1 (using MLXLLM)
Model	Qwen3 1.7B 4-bit (via Hugging Face Hub)
Inference setup	MLXLMCommon streaming generation, default Stream / default Device
App profile	Background-capable (UIBackgroundModes = [location, audio]), isIdleTimerDisabled = true, screen often locked while inference runs

Reproduction

Sustained streaming inference of Qwen3 1.7B 4bit (one ~150-token rewrite every 10–30s, kicked off automatically on a 60s GPS-driven scan loop). The crash typically fires within ~2 minutes of repeated generate(...) calls. Crash also reproduces with the screen locked (this is a walking-radio app).

Approximate timing from the crash log:

procLaunch:   2026-05-15 12:30:55
procExit:     2026-05-15 12:32:56   ← ~2 min later
isLocked:     1
wasUnlockedSinceBoot: 1

I do not currently have an isolated minimal repro — happy to put one together if it helps, but the stack trace is unambiguous and the source path is short, so I wanted to get this on your radar first.

Expected behavior

A MTL::CommandBufferStatusError from a Metal completion handler should not be able to terminate the entire app. The mlx-swift Swift layer should catch the C++ exception (or mlx::core::gpu::check_error should not throw across a callback boundary that the host language cannot catch) and surface it as either a throws Swift API or an explicit error state on the array / Stream.

Actual behavior

abort() from com.Metal.CompletionQueueDispatch. The triggered thread (top frames):

# 0 __pthread_kill +8
# 1 pthread_kill +268
# 2 abort +148
# 3 __abort_message +132
# 4 demangling_terminate_handler() +272
# 5 _objc_terminate() +172
# 6 std::__terminate(void (*)()) +16
# 7 __cxxabiv1::failed_throw(__cxxabiv1::__cxa_exception*) +88
# 8 __cxa_throw +92
# 9 mlx::core::gpu::check_error(MTL::CommandBuffer*) +476
#10 mlx::core::gpu::eval(mlx::core::array&)::$_1::operator()(MTL::CommandBuffer*) const +28
#11–#16 std::function plumbing
#17 invocation function for block in MTL::CommandBuffer::addCompletedHandler(...) +52
#18 MTLDispatchListApply +52
#19 -[_MTLCommandBuffer didCompleteWithStartTime:endTime:error:] +608
#20 -[IOGPUMetalCommandBuffer didCompleteWithStartTime:endTime:error:] +220
#21 -[_MTLCommandQueue commandBufferDidComplete:startTime:completionTime:error:] +108
...

Exception Type: EXC_CRASH (SIGABRT), Termination Reason: SIGNAL 6 Abort trap: 6, asi: "abort() called". No Application Specific Information beyond that — the C++ std::runtime_error::what() ("[METAL] Command buffer execution failed: ...") is not surfaced in the crash report, so I unfortunately do not have the underlying Metal error string.

Source pointer

The throwing function is here:

// mlx/backend/metal/eval.cpp
inline void check_error(MTL::CommandBuffer* cbuf) {
  if (cbuf->status() == MTL::CommandBufferStatusError) {
    std::ostringstream msg;
    msg << "[METAL] Command buffer execution failed: "
        << cbuf->error()->localizedDescription()->utf8String();
    throw std::runtime_error(msg.str());
  }
}

…and the call site is the completed-handler in mlx::core::gpu::eval(mlx::core::array&). Throwing from inside an addCompletedHandler block on iOS — where Swift sits above and cannot catch C++ exceptions — guarantees std::terminate.

Suggestions

A few possibilities, in increasing order of effort:

1. Catch + log + flag: Wrap the body of the addCompletedHandler in a try { check_error(cbuf); } catch (const std::exception& e) { /* store on Stream/Device error state, log, … */ }. The next eval() / outputs() call on the consuming side can then re-throw — which Swift can catch.
2. Surface via mlx-swift: Expose the captured error as a Swift-side throws API on the synchronous evaluation entry points so apps can recover (skip token, reset stream, fall back) instead of dying.
3. Document the iOS hazard: Note in the README that running mlx-swift inference under sustained load on iOS can hit Metal-side errors that today are unrecoverable, and recommend a watchdog / restart strategy until the catch path lands.

Happy to test patches on this device — just let me know what you'd like to see.

Underlying Metal error

I did not capture the localized description string, but given the trigger pattern (sustained ~2 min of Qwen3 inference, often with the screen locked, on A17 Pro), I suspect resource pressure on Metal — possibly the same family as #237 even if the surfaced symptom is different. If there is a way to pull the error string (e.g. signposts, a debug build that logs before throwing), I can re-run and capture it.

---

For context, the host app is a SwiftUI walking-radio app that rewrites Wikipedia extracts via an on-device LLM. We have since fully removed the mlx-swift dependency in favor of Apple Foundation Models specifically because of this crash. If a fix lands, I would happily revisit MLX as the secondary LLM.

[davidkoski]

A few thoughts:

• perhaps this one will help: Catch error in CommandBuffer and poison the events mlx#3523
  • it would allow failures in the gpu worker thread to be handled
• I presume withError or setErrorHandler isn't catching this
  • you said the exception is uncaught at the swift boundary -- this would catch it
  • but I suspect it is entirely on the C++ side
• have you tuned the cacheLimit?
  • https://swiftpackageindex.com/ml-explore/mlx-swift/main/documentation/mlx/memory
  • especially on iOS this may be useful -- set the cacheLimit to a few megabytes -- you may be hitting some peak use or just accumulated buffers

Ultimately I think we need that fix from the mlx side.

[kitadesign]

Thank you so much for the quick response, @davidkoski 🙏

The mlx PR #3523 approach (deferred error throwing at safe sync points) looks exactly like what would let us recover from this on the Swift side — really glad to see it already has CI green. Is there an estimated timeline for landing on main and rolling into mlx-swift? Given this crash is a hard SIGABRT from a background thread, recovery is effectively impossible from the host app today, so an early fix would unblock real-device on-iOS use significantly.

On the other suggestions:

• withError / setErrorHandler: agreed, I expect these won't catch it either — the throw happens entirely on the C++ side inside the Metal completion-handler block, before any Swift frame is on the stack. Happy to confirm empirically once #3523 lands so we can compare.

• cacheLimit: I have not tuned it. I'll try setting it to a few MB and report back whether sustained inference survives longer. If this meaningfully delays the crash, that's a useful workaround to document even after the proper fix lands.

If a test build / patch is available before #3523 merges that I can sideload on this exact device (iPhone 15 Pro / iOS 26.4.2), I'm happy to run it under the same 2-min Qwen3 1.7B 4bit reproduction and capture the underlying Metal error string for you. Just point me at a branch.

Thanks again for taking a look 🙏

[davidkoski]

It may be a while before this appears -- that PR isn't merged yet and it has to make it through mlx, mlx-c and then mlx-swift. You might be able to cherrypick those commits into your local mlx (Sources/Cmlx/mlx) and give it a try that way. It doesn't look like it requires new API so that should work.

[tseylerd]

@davidkoski thanks for the help on this one. Another problem I experienced trying to porting the patch to the mlx and the propagating it to mlx-swift is that mlx-swift is compatible only with mlx 0.31.1, but the patch is only applicable to 0.31.2 or later and the code there diverged quite significantly (or I didn't understand something). So to cherry-pick that, one needs to:

• Patch mlx-swift to make it compatible with mlx 0.31.2 (this is where I didn't succeed so far, it builds but doesn't work)
• Update mlx-c submodule to point mlx-c main
• Regenerated and commit headers in mlx-c
• Patch mlx-c a bit as well so that it links successfully against mlx 0.31.2

So it looks to me that it's a lot of work to make it land to the new mlx-swift release. Do you know if there're any plans to migrate mlx-swift on mlx 0.31.2?

[davidkoski]

There are some changes in mlx that tie a Stream to a thread -- this model doesn't work for swift since a Task can run on multiple threads. There is work to relax that restriction (looks like it is completed: ml-explore/mlx#3578)

The next steps are:

• tag mlx
• update mlx-c
• update mlx-swift to pick up mlx/mlx-c

I suspect there will be some work to make the new stream/thread model work properly. You can keep an eye on the mlx-c side and make sure the API that you expect is exposed there.

Anyway, that is the long version. There is no way to make mlx-swift pick it up any faster, but you could potentially surgically do it in your own fork, though it might be tricky as you noted.

[tseylerd]

@davidkoski thank you! I ported this PR to mlx v0.31.1:
tseylerd/mlx#1

Tests pass and it works on my setup. I'd appreciate it if you or somebody could take a look and verify the correctness.
If it's correct, we may offer this branch as a workaround or port it to mlx somehow so that all folks can deliver fixed versions of their software.

Actually, this is quite important. If somebody uses mlx-swift-lm in an iPhone app, the app will crash every time it goes into the background when running inference.