Cannot connect to usb debugger (Atmel ICE) when compiling on macOS Apple Silicon

[second-string]

My goal is to debug an ATXMega128a1u on Windows and macOS with firmware compile with avr-gcc. I would love for this to be using avarice, since our whole dev/build system is set up with cmake standalone on the command line already.

avarice 2.13 is available on homebrew, but support for the usb/edbg transport selection (avarice -4 --jtag usb) was only added in 2.14. I have been really struggling to get avarice compiled with this support, both on windows and macOS. This focuses on macOS as this is more desired.

tl;dr:

I can get this compiling on Apple Silicon, but the call to hid_enumerate to find USB-attached debuggers is always empty.

Details

I can get the ./Bootstrap, ./configure, make chain working on macOS. This compiles a working avarice, however executing avarice -4 --jtag usb gives me the message:

avarice has not been compiled with libusb support

After digging, I found that autoconf was not setting HAVE_LIBUSB due to a failed link check for the usb_get_string_simple function. I verified that I had both libusb and libusb-compat installed through brew (the latter has the usb_get_string_simple in its header file), but was still seeing this output from running the configure script:

checking for bfd_init in -lbfd... no
checking for usb_get_string_simple in -lusb... no
checking for libusb20_dev_open in -lusb... no
checking for library containing hid_init... no

I fixed this by providing the proper -I and -L paths to the configure invocation, since it clearly wasn't finding them in the homebrew (I also started including -ggdb for debug symbols to debug these issues later on):

 ./configure \
CFLAGS="-ggdb -I/opt/homebrew/opt/libusb-compat/include" \
CXXFLAGS="-ggdb -I/opt/homebrew/opt/libusb-compat/include " \
LDFLAGS="-L/opt/homebrew/opt/libusb-compat/lib"

Configure script then properly output checking for usb_get_string_simple in -lusb... yes. When I ran make however, it would fail at the link step, saying it was trying to link to my libusb but it was arm64 and it was looking for x86_64. This was when I realised that avarice is trying to compile as x86_64, not native Apple Silicon. My fix for this is pretty gross but I launched a shell with rosetta (arch -x86_64 zsh), then manually installed brew into the normal Intel mac location /usr/local. When I used the Intel brew to install libusb and libusb-compat into the "old normal" locations of /usr/local/lib and /usr/local/include, I could re-run configure without passing manual args. make also worked properly.

Running the avarice binary with the usb arg now didn't say it had no libusb support, but it exited silently. Using lldb I realised that it also needed hidapi linked in (see the checking for library containing hid_init line in the configure script output near the top). I installed this through Intel brew and verified that that no became a yes on a reconfigure.

Now I am finally stuck. The binary runs fine, but the call to hid_enumerate to look for attached debuggers (jtag2usb.cc:639) always returns an empty list. I've tried with both an Atmel Ice and an ATXmega128a1u Xplained Pro board attached. I've also tried running the built avarice from an x86_64 Rosetta shell and from a native Apple Silicon shell, both have same issue.

So somehow this call is silently failing to properly enumerate the connected devices, but I'm not sure how else to continue debugging this. Alternatively, if someone just gives me a 2.14 binary for macOS (and windows) with libusb support compiled in and verified working, I will be happy and I don't need to do it myself!

[dl8dtl]

I'm afraid this requires much rewrite of the (low-level) code.
I only slightly remember that from AVRDUDE. On MacOS, you cannot access a HID through libusb but only through libhidapi. AVaRICE has never been taught to use libhidapi, however.

[maxgerhardt]

Alternatively, if someone just gives me a 2.14 binary for macOS (and windows) with libusb support compiled in and verified working, I will be happy and I don't need to do it myself!

The Windows binary in the Github Actions CI can definitely connect to a ATMega328P XMINI evaluation board, which has an mEDBG debugger on-board.

C:\Users\Max\Downloads\AVaRICE Windows 64-bit\usr\bin>avarice.exe --edbg  -w :4444
AVaRICE version 2.14svn20200906, Apr 24 2025 18:02:57

JTAG config starting.
Found a device, serial number: ATML2323052700008014
Reported device ID: 0x950F
Configured for device ID: 0x950F atmega328p
JTAG config complete.
Preparing the target device for On Chip Debugging.
Waiting for connection on port 4444.

[maxgerhardt]

@second-string can you download and test https://github.com/maxgerhardt/avarice/actions/runs/14683804004 on your Mac? I've libusb and libhidapi enabled the build, and locally copied the needed .dylib files (together with a install_name_tool fix), so the binary should work even if you didn't execute brew install libusb libusb-compat hidapi. What's the output of the program with ./avarice --edbg -w :4444?

[second-string]

The windows 2.14 binary is weird. I was able to get some messages where it would properly find the connected debugger, but then give me an error around a different timeouts. When running the same commands with the dev board, I was back to the behavior where it just prints the AVaRICE version and exits immediately. Here are some of the arg combinations I tried:

avarice.exe --edbg -w :4444
avarice.exe --edbg :4444
avarice.exe --edbg -w --xmega :4444
avarice.exe --edbg -w --xmega --part atxmega128a1u :4444

Here is an example response command response with the -d debug flag:

avarice_win_log.txt

It hangs after the initJtagBox() line for about 5 seconds and then exits.

Regarding macOS:

On MacOS, you cannot access a HID through libusb but only through libhidapi. AVaRICE has never been taught to use libhidapi, however.

@dl8dtl Unless I'm misunderstanding your comment, I think it is using libhidapi. This is the line that is returning an empty list even with an Atmel PID debugger connected.

@second-string can you download and test https://github.com/maxgerhardt/avarice/actions/runs/14683804004 on your Mac?

@maxgerhardt This is unfortunately the same behavior that I was seeing with my built copy - it prints the AVaRICE version number and then immediately exits. I've tried stepping through your binary with lldb but since there aren't full debug symbols, it's a bit hard to read the assembly (for me). It seem like it actually might be getting a bit further though, as I've verified that it starts this hid_thread pthread here. Is it possible to run the CI with the debug flags enabled? using the -d flag gives zero further logging.

./configure CFLAGS="-ggdb" CXXFLAGS="-ggdb"

One interesting thing is that the binary from the CI run that you linked is compiled fully as a native apple silicon binary:

~/<path to downloaded binaries>/local/bin $ file avarice
avarice: Mach-O 64-bit executable arm64

But the one I compile is always x86_64 (hence all the issues with the Intel brew installations and linking)

file src/avarice
src/avarice: Mach-O 64-bit executable x86_64

[maxgerhardt]

Going back to the XMEGA A1U Xplained Pro (ATxmega128A1U) kit, per schematics it uses an EDBG and a PDI connection to communicate with the target microcontroller. What is the exact output when you run

./avarice --edbg --pdi --debug :4444

?

[second-string]

Ah I had completely forgotten about that arg. Our custom board has PDI and JTAG, and I've been trying to mostly get it with JTAG but also switching back to PDI sometimes.

Good news! I know have a working command from windows to successfully launch the gdb server when connected to both the dev board or our custom board (both JTAG and PDI):

# for PDI with dev board or custom PCB (interesting that this doesn't need the extra --xmega)
./avarice.exe --edbg --pdi :444

# for JTAG with custom PCB connected to JTAG header
./avarice.exe --edbg --xmega --jtag usb :4444

I can also use the first command to debug the dev board when it's connected to macOS, using either the x86 avarice binary that I built or the arm64 one that you linked from CI. The final remaining issue is using an Atmel ICE with macOS and our custom board, ask t seems like there is still an issue with enumerating a USB HID device. Both PDI and JTAG still fail silently, so I'm assuming it's the usb step that's the issue.

[maxgerhardt]

When you build libhid from source (https://github.com/libusb/hidapi), it should also build a hidtest test program. What is the output of that program? Does it make any difference if you run that program as sudo?

[maxgerhardt]

There is a very interesting line in that program that is only included for Mac in hidtest but not included in avarice anywhere: A call to hid_darwin_set_open_exclusive(0);.

https://github.com/libusb/hidapi/blob/8c9cbf6c020974d23e4690497778f3df173d1166/hidtest/test.c#L175-L179

Maybe you can patch in a call to that function, recompile avarice and restest.

[second-string]

Will be away and unavailable to get to these test steps for the next week and a half. I'll test when I'm back.

Would it be possible for you to approve the pipeline run for this PR so I can get the arm64 version built with debug symbols please? #113 . I don't know how to build targeting macOS silicon like this CI is doing

[second-string]

hidtest does not help. Compiled manually and run does not find the attached Atmel ICE. Also adapting hidtest to search specifically for the ICE VID/PID does not find anything. Following a check from this stack overflow post, I did in fact determine that the device can be see from pyusb.

This is the output of a test program with pyusb that prints out a lot of the device/interface/config/endpoint properties:

=== Device Descriptor ===
  idVendor:      0x3eb
  idProduct:     0x2141
  bcdDevice:     0x101
  bDeviceClass:      0xef
  bDeviceSubClass:   0x2
  bDeviceProtocol:   0x1
  Manufacturer:  Atmel Corp.
  Product:       Atmel-ICE CMSIS-DAP
  Serial Number: J41800039321

=== Configuration 1 ===
  bNumInterfaces: 2
  MaxPower:       50 mA

  --- Interface 0, Alt Setting 0 ---
    Interface Class:    0x3
    Interface Subclass: 0x0
    Interface Protocol: 0x0

    >>> Endpoint Address: 0x1
        Attributes:       0x3
        Max Packet Size:  512
        Interval:         1

    >>> Endpoint Address: 0x82
        Attributes:       0x3
        Max Packet Size:  512
        Interval:         1

  --- Interface 1, Alt Setting 0 ---
    Interface Class:    0xff
    Interface Subclass: 0xff
    Interface Protocol: 0xff

    >>> Endpoint Address: 0x84
        Attributes:       0x2
        Max Packet Size:  512
        Interval:         255

    >>> Endpoint Address: 0x3
        Attributes:       0x2
        Max Packet Size:  512
        Interval:         255

The bInterfaceClass value for interface 0 is 0x3, which denotes a HID device properly. But the top-level bDeviceClass is 0xEF, denoting a "Miscellaneous device" (see Base Class EFh heading here). I wonder if that is causing macOS to grab it with a non-HID driver.

I compiled avrdude from source with debug symbols and stepped through this section when attaching to the programmer: https://github.com/avrdudes/avrdude/blob/0ad18e5c2f6c1af4a4f241bd8d34830a4d111a98/src/jtag3.c#L1659-L1693

It turns out that avrdude also fails to find the device in the HIDAPI section, and falls back to the libusb section. So maybe it's just not possible to grab the Atmel ICE as an HID device in the newer version of macOS (or on apple silicon maybe)? Not sure if there is something that can be done to mitigate that, but I think my next step will be to add a similar libusb fallback here to see if I can get that working. No idea if this code is written modularly enough, or if it's going to be dependent on hidapi-specific structures and whatnot.

EDIT: oof, okay, it looks like that's going to be too nasty to make work. I didn't realize that the hidapi stuff had its own thread running with different logic than the libusb connections. So unless there's some way to make hidapi force a connection to what it sees as non-HID devices, I'm not sure that this will work on macOS anymore

[second-string]

Okay I made some progress. It's not that it has an invalid device class that causes macOS to not assign it the normal HID driver, but a separate kext that I must have installed a while ago for the exact opposite reason. According to some threads, it seems there used to be issues connecting to the Atmel ICE on later versions of intel macos using libusb, because the Apple HID driver would claim the device exclusively. I realised that I already had this AtmelICE.kext installed, and that was causing the main issue.

After removing /Library/Extensions/AtmeICE.kext and /Users/<user>/Library/StagedExtensions/....AtmelICE.kext and rebooting, the device is properly assigned the apple HID driver. I can see it in the output of sudo ioreg -irc IOUSBHostDevice, it now appears in the list output by hidutil list, and my adapted hidtest that only looks for its specific VID/PIC also finds it.

Unfortunately, it's not 100% working yet. When running hidtest, the code times out trying to query the device:

hidapi test/example tool. Compiled with hidapi version 0.15.0, runtime version 0.15.0.
Compile-time version matches runtime version of hidapi.

Device Found
  type: 03eb 2141
  path: DevSrvsID:4294970129
  serial_number: J41800039321
  Manufacturer: Atmel Corp.
  Product:      Atmel-ICE CMSIS-DAP
  Release:      101
  Interface:    0
  Usage (page): 0x1 (0xff00)
  Bus type: 1 (USB)

  Report Descriptor: (35 bytes)
0x06, 0x00, 0xff, 0x09, 0x01, 0xa1, 0x01, 0x15, 0x00, 0x26,
0xff, 0x00, 0x75, 0x08, 0x96, 0x00, 0x02, 0x09, 0x01, 0x81,
0x02, 0x96, 0x00, 0x02, 0x09, 0x01, 0x91, 0x02, 0x95, 0x04,
0x09, 0x01, 0xb1, 0x02, 0xc0,
Manufacturer String: Atmel Corp.
Product String: Atmel-ICE CMSIS-DAP
Serial Number String: (74) J41800039321
  Report Descriptor: (35 bytes)
0x06, 0x00, 0xff, 0x09, 0x01, 0xa1, 0x01, 0x15, 0x00, 0x26,
0xff, 0x00, 0x75, 0x08, 0x96, 0x00, 0x02, 0x09, 0x01, 0x81,
0x02, 0x96, 0x00, 0x02, 0x09, 0x01, 0x91, 0x02, 0x95, 0x04,
0x09, 0x01, 0xb1, 0x02, 0xc0,
Device Found
  type: 03eb 2141
  path: DevSrvsID:4294970129
  serial_number: J41800039321
  Manufacturer: Atmel Corp.
  Product:      Atmel-ICE CMSIS-DAP
  Release:      101
  Interface:    0
  Usage (page): 0x1 (0xff00)
  Bus type: 1 (USB)

Unable to read indexed string 1
Indexed String 1:
Unable to send a feature report: IOHIDDeviceSetReport failed: (0xE0005000) unknown error code
Feature Report
   00 00 00 00
waiting...
waiting...
waiting...
waiting...
waiting...
waiting...
waiting...
waiting...
waiting...
waiting...
read() timeout

When running avarice with the debug flag, I can see there is finally a proper connection made to the box, but it's failing to communicate basic starting messages (whether or not the device is plugged into a PCB, so it seems to be an issue of actual USB communication). I get the same output both from the arm64 binary provided from the CI and also my locally compiled x86_64 version. I added in the hid_darwin_set_open_exclusive(0); call to mine and rebuilt but it didn't make a difference unfortunately.

[1088] brianteam@bts-wv-mbp ~/Downloads/avarice-avarice-rel-2_14 $ ./src/avarice -d -4 --jtag usb :4444
AVaRICE version 2.14, May 15 2025 16:46:19

Defaulting JTAG bitrate to 250 kHz.

Found HID PID:VID 0x03eb:0x2141, serno J41800039321
Probing for HID max. packet size
Unexpected DAP_Info response 0x00 0x00
HID thread started
JTAG config starting.

command "sign-on" [0x01, 0x10]
0E 00 00 00 01 10 00
Received 0x00 0x00 0x00 0x00 0x00 0x00
Querying for response: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
Querying for event: unexpected response (0x00)
read() timed out

<this same 'command sign on' block repeats 9 more times>

initJtagBox() failed: doSimpleJtagCommand(): too many failures