Build seems to be broken

[Eeems]

toltec-dev/toltec#748

[bkirwi]

I've looked into this over the holiday. Some rough notes:

• The build works outside of toltec on my machine. (Using the standard RM toolchain etc.) However it's quite straightforward to reproduce the issue with toltecmk.
• Versions of this package that previously built successfully under toltec will now fail. I think -- but am not certain -- that the difference is in the build environment -- toltec pulling in more recent versions of the Debian libclang packages with slightly different libraries.
• Upgrading relevant libraries on the Rust side (bindgen etc.) does not seem to help.
• Generally speaking I've found working with the tflite library in rust to be a pain, and it's barely maintained. ONNX is an alternative ML runtime format that is better supported in Rust and appears to have comparable performance. I'd like to try migrating over, though that may take some time.

[Eeems]

I've looked into this over the holiday. Some rough notes:

• The build works outside of toltec on my machine. (Using the standard RM toolchain etc.) However it's quite straightforward to reproduce the issue with toltecmk.
• Versions of this package that previously built successfully under toltec will now fail. I think -- but am not certain -- that the difference is in the build environment -- toltec pulling in more recent versions of the Debian libclang packages with slightly different libraries.
• Upgrading relevant libraries on the Rust side (bindgen etc.) does not seem to help.
• Generally speaking I've found working with the tflite library in rust to be a pain, and it's barely maintained. ONNX is an alternative ML runtime format that is better supported in Rust and appears to have comparable performance. I'd like to try migrating over, though that may take some time.

The image version used should not have changed since this was initially built and working, but perhaps something did change. Have you tried using an older image tag?

[bkirwi]

The change is not the image itself, but the packages it pulls in. folly requires various build dependencies:

makedepends=(build:librust-clang-sys-dev build:libclang-dev build:libc6 build:libc6-dev build:clang)

When I build via toltecmk, I get, among other things:

Setting up clang (1:16.0-57) ...

And version 16 of clang was not released until march this year. Which makes some sense: I don't think toltec is doing anything to pin all upstream packages or whatever. But it does make the build nonreproducible.

I'm tempted to propose that clang etc. be baked into the rust image, since they're common build-time dependencies. But that seems like a bit of a hack and I'm not sure I like it yet.

[Eeems]

The change is not the image itself, but the packages it pulls in. folly requires various build dependencies:

makedepends=(build:librust-clang-sys-dev build:libclang-dev build:libc6 build:libc6-dev build:clang)

When I build via toltecmk, I get, among other things:

Setting up clang (1:16.0-57) ...

And version 16 of clang was not released until march this year. Which makes some sense: I don't think toltec is doing anything to pin all upstream packages or whatever. But it does make the build nonreproducible.

I'm tempted to propose that clang etc. be baked into the rust image, since they're common build-time dependencies. But that seems like a bit of a hack and I'm not sure I like it yet.

Ah, fair point. Yeah, creating pinned versions of the debian package repos is out of scope for us right now. It's also not often that we expect a compiler to break builds. I do think it would be worth baking clang into the base toolchain image, as it's a fairly standard build tool. That would be done as a v3.2 tag in my opinion.

I've documented how to modify the toolchain images here: toltec-dev/toolchain#20

[palfrey]

I'm now seeing

[   ERROR] toltec.builder: build(): error: failed to run custom build command for `tflite v0.9.4 (https://github.com/bkirwi/tflite-rs?rev=4212a1bf33ebbcbf535b162cc591d46c7d054d92#4212a1bf)`
[   ERROR] toltec.builder: build(): 
[   ERROR] toltec.builder: build(): Caused by:
[   ERROR] toltec.builder: build(): process didn't exit successfully: `/src/target/release/build/tflite-234716c71c48e956/build-script-build` (exit status: 101)
[   ERROR] toltec.builder: build(): --- stderr
[   ERROR] toltec.builder: build(): thread 'main' panicked at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/proc-macro2-1.0.26/src/fallback.rs:696:9:
[   ERROR] toltec.builder: build(): "aligned_storage_type_(unnamed_struct_at_/opt/x-tools/arm-remarkable-linux-gnueabihf/arm-remarkable-linux-gnueabihf/include/c++/10_3_0/type_traits_2074_2)" is not a valid Ident

(from https://github.com/toltec-dev/toltec/actions/runs/17562595625/job/49882373962?pr=951)

This is related to toltec-dev/toltec#951 and included a change of the depends to makedepends=(build:libclang-16-dev build:clang-16 build:llvm-16-dev) as those were the most recent clang/llvm packages available in the newer images.