Assertion / Illegal instruction on macOS when calling `HidApi::new` in different threads

[Tiwalun]

I've run into an issue on macOS where calling HidApi::new from different threads leads to an abort of the process, with an exception for an illegal instruction.

Looking at the stacktrace, it seems to be cause by a CFAssertMismatchedTypeID assertion somewhere in Core Foundation.

Are there any special precautions that have to be taken when using hidapi from multiple threads? I couldn't find anything in the documentation.

Stacktrace

* thread #5, name = 'try 1', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
  * frame #0: 0x00007ff80cf83f3e CoreFoundation`_CFAssertMismatchedTypeID + 110
    frame #1: 0x00007ff80ce254bd CoreFoundation`CFRunLoopAddSource + 973
    frame #2: 0x00007ff80f76e305 IOKit`IOHIDDeviceScheduleWithRunLoop + 74
    frame #3: 0x00007ff80f772651 IOKit`__IOHIDManagerDeviceApplier + 527
    frame #4: 0x00007ff80f735537 IOKit`__IOHIDManagerDeviceAdded + 766
    frame #5: 0x00007ff80f73513b IOKit`__IOHIDManagerSetDeviceMatching + 347
    frame #6: 0x0000000100010724 hidapi-repro`hid_enumerate(vendor_id=0, product_id=0) at hid.c:654:2
    frame #7: 0x000000010000ce03 hidapi-repro`hidapi::hidapi::HidApiBackend::get_hid_device_info_vector::hdb756fded0153d2d at hidapi.rs:25:36
    frame #8: 0x000000010000dbde hidapi-repro`hidapi::HidApi::new::h471133b553ec121f at lib.rs:171:27
    frame #9: 0x0000000100006141 hidapi-repro`hidapi_repro::enumerate::h2a623970c0b3f1ae at main.rs:16:13

Repo with code to reproduce: https://github.com/tiwalun/hidapi-repro

[ruabmbua]

Hi! Because of the not very well defined nature of the underlying C library (regarding multithreading), we used to prevent creating a second instance of a Hidapi object, by acquiring a lock.
However people started requesting that I lift this restriction, and testing on the backends that we could test showed, that as long as you open a device not from multiple threads, everything works out fine.

Unfortunately we may have not tested on macos (due to nobody owning a mac). I recommend you look at the hidapi C library if it should be possible to use it from multiple threads in macos.

B.t.w. it could totally be, that the problem only occurs because you are not on the main thread. I have some experience in the far past with macos not allowing certain things on a normal spawned thread (IO related & gpu related).

If it really can not work, I will probably add a check just for macos to not allow creating more than one API instance.

[Tiwalun]

Unfortunately, running everything on the main thread is not possible for my use case.

I've had a look at the library, and found some existing issues which might be related:

• onlykey-agent crashes on mac trustcrypto/onlykey-agent#38
• Crash when calling enumerate from different threads (not simultaneously) trezor/cython-hidapi#148

I've tried to use hid_exit from the underlying library to reset the global variables, and that has helped for the simple case in the reproducer I shared. Unfortunately, it doesn't help in the actual application :(.

The workaround I have for now is to spawn a dedicated thread for hidapi, run HidApi::new() in there, and keep it alive for the whole duration of the process. With that, it seems possible to use hidapi from other threads without issues afterwards 🤷‍♂️.

My current assumption is that this line https://github.com/libusb/hidapi/blob/d0856c05cecbb1522c24fd2f1ed1e144b001f349/mac/hid.c#L443 associates the IOHidManager with the run loop of the thread calling hid_init, and because of this the thread has to stay alive as long as hidapi is used, otherwise the run loop doesn't exist anymore, and then an assertion is triggered.

I'll hopefully find some time to work on a proper fix for this, either directly in hidapi, or just adding a pure Rust backend to hidapi-rs...

[micolous]

That's correct, IOHIDManager needs to be alive and associated with an active CFRunLoop while any IOHIDDevice instance it created is still open or in use.

When IOHIDManager is unscheduled, it also unschedules all its IOHIDDevices, when closed it closes all devices, and when released, it releases all devices. Unlike most Core Foundation objects, there's no reference counting - IOHIDManager just tears everything apart once it's done, and there's no way to control that.

This is much easier to manage from Rust (#30), where you could have an Arc-wrapped IOHIDManager referenced by every Device instance.

[andresv]

Stumbled on the same issue when using hidapi 2.4.1. In my application new thread is created each time when somebody wants to do something with the device.

* thread #12, stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
  * frame #0: 0x00007ff80705540e CoreFoundation`_CFAssertMismatchedTypeID + 110
    frame #1: 0x00007ff806eddba5 CoreFoundation`CFRunLoopAddSource + 973
    frame #2: 0x00007ff809e7c38d IOKit`IOHIDDeviceScheduleWithRunLoop + 89
    frame #3: 0x00007ff809e80b7e IOKit`__IOHIDManagerDeviceApplier + 555
    frame #4: 0x00007ff809e44179 IOKit`__IOHIDManagerDeviceAdded + 810
    frame #5: 0x00007ff809e43d59 IOKit`__IOHIDManagerSetDeviceMatching + 378
    frame #6: 0x00007ff809e43b83 IOKit`IOHIDManagerSetDeviceMatchingMultiple + 255
    frame #7: 0x0000000100ba9e04 um6dbench`hid_enumerate(vendor_id=0, product_id=0) at hid.c:733:2
    frame #8: 0x0000000100ba4e93 um6dbench`hidapi::hidapi::HidApiBackend::get_hid_device_info_vector::h1a0ac8ae93d5a3eb at hidapi.rs:25:36
    frame #9: 0x0000000100ba204e um6dbench`hidapi::HidApi::new::h43ba44533b9d7c18 at lib.rs:171:27
    frame #10: 0x00000001003fe7d3 um6dbench`probe_rs::probe::cmsisdap::tools::list_cmsisdap_devices::h9510f0612ea21ccc at tools.rs:31:22
    frame #11: 0x0000000100512305 um6dbench`probe_rs::probe::list::AllProbesLister::list_all::hc932401596101843 at list.rs:126:24
    frame #12: 0x00000001005122b8 um6dbench`_$LT$probe_rs..probe..list..AllProbesLister$u20$as$u20$probe_rs..probe..list..ProbeLister$GT$::list_all::hb4edfc444d6a8d14(self=0x0000000000000001) at list.rs:71:9
    frame #13: 0x0000000100512250 um6dbench`probe_rs::probe::list::Lister::list_all::h7c0fbc27f9e18c8f(self=0x000070000f3709a0) at list.rs:41:9

So I added another thread to main.rs to just hold a hidapi as @Tiwalun suggested and after that I do not see SIGILL animore.

std::thread::spawn(|| {
    let _hid = hidapi::HidApi::new();
    loop {
        std::thread::sleep(std::time::Duration::from_secs(3600));
    }
});

[White-Rabbit-Scientific]

Just wanted to note that I have reached the same endpoint with hidapi on MacOS Sequia 15.1.

I cannot run this line inside egui without it crashing:

let api = hidapi::HidApi::new().unwrap();

[splix]

I think I have the same issue (macOS, Sequoia, Intel.) In my case it's the same thread, but a few instances created one after another. Ex., a series of unit tests where each test uses a HID, and running them with --test-threads=1. My understanding that there are some resources that are not immediately dropped.

Adding thread::sleep(Duration::from_millis(1000)); before new() helps sometimes. Better if it sleeps 2000ms and more. But that doesn't look like a right way

[davehorner]

I released a crate using hidapi-rs. I am seeing the same crashes when using macos.
There doesnt seem to be a solution, I see above someone wrapped in a thread. I may try that. if that is the solution; maybe that should be something hidapi-rs manages; if its a matter of keeping it in the main thread, lets document that or create a macos specific function that is named/documented as needing to be called from main.

it talks about panic'ing if there was a call to no_enumerate; i don't know if thats related, but I need enumeration.

I guess I disable this feature on macOS as there is no way to protect against this. lmk if I missed something.
macos is broken. it might be worth surfacing this long open fact in the readme. this should be more visible.

[xobs]

As noted in #175 (which is a duplicate of this issue), this can be seen with the following code:

fn main() {
    for i in 1..100 {
        println!("Loop {i}");
        let jh = std::thread::spawn(|| {
            hidapi::HidApi::new().unwrap();
        });
        jh.join().unwrap();
    }
    println!("Done");
}

This crashes on the second loop. That means that if you use hidapi from a library, you must inform your users that you cannot use it from multiple threads.

I did find an atrocious patch that worked, but I'm not happy with it at all:

diff --git a/build.rs b/build.rs
index 5eafcc7..5247bbc 100644
--- a/build.rs
+++ b/build.rs
@@ -195,6 +195,7 @@ fn compile_macos() {
     cc::Build::new()
         .file("etc/hidapi/mac/hid.c")
         .include("etc/hidapi/hidapi")
+        .define("CFRunLoopGetCurrent", Some("CFRunLoopGetMain"))
         .compile("libhidapi.a");
     println!("cargo:rustc-cfg=hidapi");
     println!("cargo:rustc-link-lib=framework=IOKit");

The only real workaround I've found to support using a library that supports hidapi is to also use hidapi and make sure I use the same version, and then manually create a Hidapi object before any sub-threads are created. That way the runloop that's processing events is guaranteed to never be destroyed.