Is cargo-bitbake useless? (at least for newer Yocto releases)

[paolobarbolini]

We recently upgraded to Yocto styhead. After going through yet another problem with cargo-bitbake (WORKDIR becoming UNPACKDIR), we looked at how Rust recipes were being managed upstream these days, specifically in poky, and found that they have all migrated away from cargo-bitbake ¹.

What basically happened, as far as we could tell, is that cargo-bitbake was replaced with a normally written recipe (no more auto-generated .bb file) and a new bitbake command (bitbake -c update_crates RECIPE_NAME) that converts the Cargo.lock coming from the repository into a list of crates ². This is similar to how cargo-bitbake works but I'd argue the file structure is much better organized, since auto-generated content (the list of crates and hashes) sits in a secondary file instead of the main one.

We immediately converted all of our Rust recipes to this new system and found the whole thing to be much faster and cleaner to manage. This led to the deprecation of our fork of cargo-bitbake given that we don't care about previous Yocto versions ³.

The conversion can be done by:

1 Writing a "standard" .bb file, just like you would do with any other language
2. inheriting cargo and cargo-update-recipe-crates
3. require ${BPN}-crates.inc
4. Create an empty file for ${BPN}-crates.inc
5. Run bitbake -c update_crates RECIPE_NAME
6. Repeat step 5 when updating the package to a new version

I'm not sure if this was obvious to everyone else, but we happened to find this out by pure chance, and after doing a little digging in the documentation we found both the new method and cargo-bitbake suggested as possible solutions for Rust packaging ⁴, while thinking about our own experience we would never touch cargo-bitbake again now that a better solution is available. Is it time to let cargo-bitbake die?

Footnotes

1. https://git.yoctoproject.org/poky/tree/meta-selftest/recipes-extended/hello-rs/hello-rs_0.1.0.bb?h=styhead ↩

2. https://git.yoctoproject.org/poky/tree/meta-selftest/recipes-extended/hello-rs/hello-rs-crates.inc?h=styhead ↩

3. https://github.com/m4ss-Code/cargo-bitbake ↩

4. https://docs.yoctoproject.org/5.1/singleindex.html#cargo-update-recipe-crates ↩

[peterdelevoryas]

This repository is like a honey-pot for wasted effort up-rev'ing dependencies:

• "Update cargo and git2 to latest version" Update cargo and git2 to latest version #70
• "WIP: update dependencies and emit checksums" WIP: update dependencies and emit checksums #75
• "update to recent cargo api" update to recent cargo api #76
• "Update to newer cargo, fix issue" Update to newer cargo, fix issue #79 #81
• "cargo: Update dependencies" cargo: Update dependencies #84

[paolobarbolini]

Yeah and even the repository is a landfill of PRs and issues. Some, like #82, are even getting approved. For what?

[BKDaugherty]

My company is sadly still stuck on dunfell (lol don't ask), and in many ways we really just don't have the yocto knowledge to not use things like cargo bitbake. It's that one arcane thing we reached for forever ago, and now we are deeply dependent on it and a host of other hacks. We also maintain our own fork, but are excited to get off the tool as soon as we can upgrade to some yocto distribution that has something better (or ideally separating our Rust toolchain management from yocto so we also can stop using Rust 1.71! As soon as we find the time lol)

I even have my own cursed PR where I just read from the Cargo.lock file in the landfill (#71), because when I looked into cargo bitbake I was real confused and didn't understand why we needed to do package resolution.

I'm not sure if there's a "Guide to using Rust with Yocto: 10 things you should avoid, and patterns you should be using" but I would love to see it, and couldn't help but +1 this issue

[jaskij]

Good to know this is an option. I'm guessing what you describe is only available since Scarthgarp?

Like I said in #75 , I've been meaning to fork cargo-bitbake (I absolutely hate that it updates the lockfile when generating the recipe), but never got around to it. I could write that fork, that is not an issue, but due to my mental health can not in good faith take up maintaining it. Would someone be willing to team up and take up maintaining such a fork once it's made?

At least until we can all limp by and get to Scarthgarp.

[cardoe]

You don’t need to fork. You can use this repo. Nobody has offered to maintain it.

[peterdelevoryas]

Another comment on this thread: I really appreciate having the option to use cargo bitbake as a backup plan, but I think I'm going to use meta-rust-bin instead, if I can get away with it. It kinda "just worked" for me. I have some crates I want to use that require recent Rust versions (zbus 5).

[paolobarbolini]

but I think I'm going to use meta-rust-bin instead

We used it too at first, but it felt like a giant workaround, so we ended up patching poky with the latest Rust release. It's not perfect, because we don't spend time on reproducibility patches, but at least we retain the ability to customize the build instead of relying on the official release contents.

[peterdelevoryas]

but I think I'm going to use meta-rust-bin instead

We used it too at first, but it felt like a giant workaround, so we ended up patching poky with the latest Rust release. It's not perfect, because we don't spend time on reproducibility patches, but at least we retain the ability to customize the build instead of relying on the official release contents.

You know that's also a good point, I might consider doing that too. I was having a great time with meta-rust-bin until I realized it doesn't integrate with the typical bitbake network controls, so I'd have to either open up network access for crates.io or vendor all the sources for each dependency manually. I also didn't realize rustup makes it very easy to switch to an older toolchain for local, non-yocto development, so that I can make sure regular cargo usage matches the older upstream OpenEmbedded version.

[cmeissl]

but I think I'm going to use meta-rust-bin instead

We used it too at first, but it felt like a giant workaround, so we ended up patching poky with the latest Rust release. It's not perfect, because we don't spend time on reproducibility patches, but at least we retain the ability to customize the build instead of relying on the official release contents.

Depending on the actual version you need, it might also be an option to use meta-lts-mixin. At the time of writing this includes 1.81.0 for kirstone and scarthgap