Skip to content

Dfuutil missing for arm64 #436

Closed
Closed
@dolfandringa

Description

@dolfandringa

The ststm32 core for platformio defines a dependency on tool-dfuutil, but that package does not seem to be available for linux_aarch64 so installing the platform fails. I am not actually using dfu for uploading code (I am using the stlinkv2 programmer), so for me a workaround is to manually remove the dependency from ~/.platformio/platforms/ststm32/platform.json. For me that works, but it is definitely not an ideal workaround, and others might actually use dfu. So would it be possible to release an arm64 linux package for it?

Activity

thinkyhead

thinkyhead commented on Sep 9, 2021

@thinkyhead

Also, we need a Darwin arm_64 version of tool-dfuutil for the M1 Mac system.
If a set of binaries is needed, maybe I can cobble them together.

jhaand

jhaand commented on May 31, 2022

@jhaand
Contributor

It would be nice if the latest dfu-util would be added to the tool-dfuutil package. At this moment only an old revision is used and the latest 0.11 only seems available for Macos Arm64.
The dfu-util project has released 0.11 and it's not integrated in platformio for other platforms.
As you can see here:
https://registry.platformio.org/tools/platformio/tool-dfuutil/compatibility

See also this issue.
sipeed/platform-gd32v#17
Although this points to a Sipeed archive, this tool is owned by Platformio and in the top 100 of used packages.

jhaand

jhaand commented on Jun 1, 2022

@jhaand
Contributor

Today a new version of dfu-util 0.11 was introduced in the registry. Unfortunately not for linux_aarch64.

thinkyhead

thinkyhead commented on Jun 30, 2022

@thinkyhead

It's not helping me either. The chart on their page says arm64 is supported, but we're still not picking up a universal binary.

> pio --version
PlatformIO Core, version 6.0.2

> pio pkg install -g --tool "platformio/tool-dfuutil@^1.11.0"

> pio run -e Opulo_Lumen_REV3

dyld[91452]: Library not loaded: '/usr/local/opt/libusb/lib/libusb-1.0.0.dylib'
  Referenced from: '/u/t/.platformio/packages/tool-dfuutil/bin/dfu-suffix'
  Reason: tried: '/usr/local/opt/libusb/lib/libusb-1.0.0.dylib' (no such file), '/usr/local/lib/libusb-1.0.0.dylib' (mach-o file, but is an incompatible architecture (have (arm64), need (x86_64))), '/usr/lib/libusb-1.0.0.dylib' (no such file)

> find ~/.platformio -name "dfu-*" -type f
/u/t/.platformio/packages/tool-dfuutil/bin/dfu-util
/u/t/.platformio/packages/tool-dfuutil/bin/dfu-prefix
/u/t/.platformio/packages/tool-dfuutil/bin/dfu-suffix

> file ~/.platformio/packages/tool-dfuutil/bin/*
/u/t/.platformio/packages/tool-dfuutil/bin/dfu-prefix: Mach-O 64-bit executable x86_64
/u/t/.platformio/packages/tool-dfuutil/bin/dfu-suffix: Mach-O 64-bit executable x86_64
/u/t/.platformio/packages/tool-dfuutil/bin/dfu-util:   Mach-O 64-bit executable x86_64
thijstriemstra

thijstriemstra commented on Jul 12, 2022

@thijstriemstra

Same on issue on Raspberry Pi 3b+:

$ pio --version
PlatformIO Core, version 6.1.1

$ pio pkg install -g --tool "platformio/tool-dfuutil@^1.11.0"
Tool Manager: Installing platformio/tool-dfuutil @ ^1.11.0
Error: Could not find the package with 'platformio/tool-dfuutil @ ^1.11.0' requirements for your system 'linux_armv7l'

$ uname -a
Linux mpcnc 5.10.103-v7+ #1529 SMP Tue Mar 8 12:21:37 GMT 2022 armv7l GNU/Linux

Unable to compile Marlin 2.1.x, sigh.

Update: Marlin fixed this literally 2 hrs ago and all seems to work again 🥳 when applying this patch: MarlinFirmware/Marlin@068624a

loozhengyuan

loozhengyuan commented on Oct 20, 2024

@loozhengyuan

I am still encountering this issue for macOS ARM64. Based on the registry, it looks like darwin_arm64 is available the wrong binary is downloaded?

$ pio pkg uninstall -g --tool "platformio/tool-dfuutil@^1.11.0"         
Tool Manager: Removing tool-dfuutil @ 1.11.0
Tool Manager: tool-dfuutil@1.11.0 has been removed!
$ pio pkg install -g --tool "platformio/tool-dfuutil@^1.11.0"           
Tool Manager: Installing platformio/tool-dfuutil @ ^1.11.0
Unpacking  [####################################]  100%
Tool Manager: tool-dfuutil@1.11.0 has been installed!
$ file ~/.platformio/packages/tool-dfuutil/bin/*
~/.platformio/packages/tool-dfuutil/bin/dfu-prefix: Mach-O 64-bit executable x86_64
~/.platformio/packages/tool-dfuutil/bin/dfu-suffix: Mach-O 64-bit executable x86_64
~/.platformio/packages/tool-dfuutil/bin/dfu-util:   Mach-O 64-bit executable x86_64

Is it a case of wrong binary is fetched? Or the binary for darwin_arm64 does not exist? Or the wrong binary was uploaded to the registry?

valeros

valeros commented on Oct 25, 2024

@valeros
Member

Hi @loozhengyuan, could you run pio pkg update -g -t platformio/tool-dfuutil and try again?

loozhengyuan

loozhengyuan commented on Oct 25, 2024

@loozhengyuan

@valeros Thanks for looking into this! 🙇

I was able to confirm that the arm64 binaries are installed now:

$ pio pkg update -g -t platformio/tool-dfuutil
Tool Manager: Updating tool-dfuutil @ 1.11.0
Tool Manager: Removing tool-dfuutil @ 1.11.0
Tool Manager: tool-dfuutil@1.11.0 has been removed!
Tool Manager: Installing platformio/tool-dfuutil @ 1.11.241024
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
$ ~/.platformio/packages/tool-dfuutil/bin/*
~/.platformio/packages/tool-dfuutil/bin/dfu-prefix: Mach-O 64-bit executable arm64
~/.platformio/packages/tool-dfuutil/bin/dfu-suffix: Mach-O 64-bit executable arm64
~/.platformio/packages/tool-dfuutil/bin/dfu-util:   Mach-O 64-bit executable arm64

However, it does not seem to work correctly:

$ pio run --target upload
...
Adding dfu suffix to firmware.bin
dyld[29406]: Library not loaded: /tmp/dfu-util-0.11-Darwin-arm64/lib/libusb-1.0.0.dylib
  Referenced from: <98506EAE-D454-32F5-917E-8BC573B4FB07> ~/packages/tool-dfuutil/bin/dfu-suffix
  Reason: tried: '/tmp/dfu-util-0.11-Darwin-arm64/lib/libusb-1.0.0.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/tmp/dfu-util-0.11-Darwin-arm64/lib/libusb-1.0.0.dylib' (no such file), '/tmp/dfu-util-0.11-Darwin-arm64/lib/libusb-1.0.0.dylib' (no such file)
valeros

valeros commented on Oct 27, 2024

@valeros
Member

@thijstriemstra I've uploaded a new package, please run pio pkg update -g -t platformio/tool-dfuutil again.

thinkyhead

thinkyhead commented on Oct 28, 2024

@thinkyhead

@loozhengyuan , @valeros

At the moment, for many of our Marlin builds PlatformIO is downloading and running tool-dfuutil@1.9.211020. Is this because of the version spec in our environment? We're currently sticking to STSTM32 version 12.1 because a newer version would require us to update all our boards and variants.

[common_stm32]
platform = ststm32@~12.1

I added platform_packages for testing…

[[env:FYSETC_S6]
extends                     = stm32_variant
board                       = marlin_fysetc_s6
platform_packages           = tool-dfuutil@1.11.0

…and I'm getting back a rather sad error message:

FileNotFoundError: [Errno 2] No such file or directory: 
.../.platformio/packages/tool-dfuutil@1.11.0/variants/MARLIN_FYSETC_S6':

Someone please explain why PlatformIO is looking for a folder named tool-dfuutil@1.11.0/variants/MARLIN_FYSETC_S6 – appending the path to the board variants folder to the path of the dfu tool ??

Anyway, since PlatformIO was insisting on being difficult, or using tool-dfuutil@1.9.211020 I tried copying the contents of tool-dfuutil-1.11.0 into tool-dfuutil@1.9.211020. The results were the same, as it was unable to find libusb-1.0.0.dylib in various fixed paths (including /System/Volumes/Preboot/Cryptexes/OS/tmp which I'm sure we all have).

I then tried version 1.11.23411026 and got the same results. However, when I used the dfu-prefix, dfu-suffix, and dfu-util files installed by sudo port install dfu-util the build finally succeeded. I followed up with sudo port uninstall usblib to see if it would cause dfu-util to fail, and it did not cause any problems.

Here are the binaries that finally worked for me:

dfuutil-darwin-arm64.zip

Now, how can we get these reliable and non-weirdly-linked binaries to download when building Marlin on macOS?

thinkyhead

thinkyhead commented on Oct 28, 2024

@thinkyhead

Note that proper macOS binaries will not only need proper linkage, but must also be Universal binaries (not just arm64), presuming that PlatformIO does not distinguish between Darwin on ARM versus Intel for these tools.

Universal binaries from sudo port install dfu-util +universal:

dfuutil-darwin-universal.zip

Edit: These do appear to rely on /opt/local/lib/libusb-1.0.0.dylib being present, after all. So ultimately we need a set of binaries that include /opt/local/lib/libusb-1.0.0.dylib in the dynamic linker search paths. The 1.11.241026 binaries do not seem to look here.

thinkyhead

thinkyhead commented on Oct 28, 2024

@thinkyhead

That rather weird error message still deserves a look. But the download was at least made to work by adding platformio/ in front of the tool name.

platform_packages = platformio/tool-dfuutil@^1.11.241026

However, as previously mentioned, the October 26 binaries still produce the same dynamic linking error, so the updated Universal binaries are definitely needed, along with a new "1.11.241028" version.

dyld[16354]: Library not loaded: /tmp/dfu-util-0.11-Darwin-arm64/lib/libusb-1.0.0.dylib
  Referenced from: <98506EAE-D454-32F5-917E-8BC573B4FB07> /Users/thinkyhead/.platformio/packages/tool-dfuutil/bin/dfu-suffix
  Reason: tried: '/tmp/dfu-util-0.11-Darwin-arm64/lib/libusb-1.0.0.dylib' (no such file),
  '/System/Volumes/Preboot/Cryptexes/OS/tmp/dfu-util-0.11-Darwin-arm64/lib/libusb-1.0.0.dylib' (no such file),
  '/tmp/dfu-util-0.11-Darwin-arm64/lib/libusb-1.0.0.dylib' (no such file)
*** [.pio/build/FYSETC_S6/firmware.bin] Error -6
valeros

valeros commented on Oct 29, 2024

@valeros
Member

Hi @thinkyhead,

At the moment, for many of our Marlin builds PlatformIO is downloading and running tool-dfuutil@1.9.211020. Is this because of the version spec in our environment?

That's right, tool-dfuutil v1.9.211020 is the default choice for the ststm32 dev-platform v12.1.

Someone please explain why PlatformIO is looking for a folder named tool-dfuutil@1.11.0/variants/MARLIN_FYSETC_S6 – appending the path to the board variants folder to the path of the dfu tool ??

Not sure where's this error coming from. Maybe you have a custom build command added via an extra script or something?

However, as previously mentioned, the October 26 binaries still produce the same dynamic linking error, so the updated Universal binaries are definitely needed

I've uploaded a new package for MacOS ARM64, so the missing libusb-1.0.0.dylib shouldn't be a problem now. You can pull the latest version via pio pkg update -g -t platformio/tool-dfuutil.

6 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      Dfuutil missing for arm64 · Issue #436 · platformio/platform-ststm32