Erlang Linux Builds

[tsloughter]

Background

While working on binary package support in erlup I installed https://github.com/wojtekmach/otp_releases/releases/download/OTP-24.0-rc1/otp-24.0-rc1-linux-x86_64.tar.gz on my local machine, Fedora 33, and discovered it fails to run because of a ncurses dependency libtinfo. The version on Fedora 33 is 6 but the Erlang package (built on Ubuntu 16.04) requires 5.

Objective(s)

The objective is to have pre-built Erlang binary packages that work on recent Linux distros, as well as older ones up to some point, like LTS releases.

The simplest solution that may be good enough is to add the Ubuntu version to the package name and add builds for more recent Ubuntu versions. Then tools like erlup can have a manual mapping of Ubuntu version to library versions that it uses to determine the package to download.

[starbelly]

@tsloughter This is interesting as there has been discussion that I've caught here and there around doing away with bob (at least partially) in favor of GH actions. Are you thinking something like that? Or do you think this should be done in bob?

[wojtekmach]

Yes, as Hex team we indeed want to move away from Bob to GH actions as much as possible. We want to move the following things over:

• Elixir builds (https://github.com/hexpm/bob#elixir-builds)
• Elixir docs (https://github.com/hexpm/bob#elixir-docs)
• Erlang builds (https://github.com/hexpm/bob#erlang-builds)

Bob would still notably handle Docker builds (https://github.com/hexpm/bob#docker-images)

Ideally https://github.com/elixir-lang/elixir would have a workflow for Elixir builds & docs, and https://github.com/erlang/otp for Erlang builds (erlang/otp#2936)

cc @ericmj @supersimple

[wojtekmach]

@tsloughter you can browse builds for more recent Ubuntu releases here:

• https://repo.hex.pm/builds/otp/ubuntu-18.04/builds.txt

• https://repo.hex.pm/builds/otp/ubuntu-18.04/OTP-24.0-rc2.tar.gz

• https://repo.hex.pm/builds/otp/ubuntu-20.04/builds.txt

• https://repo.hex.pm/builds/otp/ubuntu-20.04/OTP-24.0-rc2.tar.gz

We have https://repo.hex.pm/builds/otp/alpine-3.10/builds.txt too.

[wojtekmach]

@tsloughter Since you mentioned https://github.com/wojtekmach/otp_releases, a friendly remainder that it, along with https://github.com/wojtekmach/beamup, is a proof-of-concept! I'm happy to do some maintenance in the meantime if it helps you but I'd rather focus on erlang/otp#2936 down the road if the OTP team decides to move forward with that.

[ericmj]

Yes, as Hex team we indeed want to move away from Bob to GH actions as much as possible. We want to move the following things over:

To clarify the goal isn't to move to GH actions specifically but rather to move management of builds away from Hex/Bob. I have some concerns with using GH actions because it doesn't give the control that is needed in my experience with Bob. As an example today we needed to rebuild all OTP versions 17-19 for Ubuntu 18.04 due to an issue with libssl versioning. This meant rebuilding ~100 versions which I don't can or should be done with actions. But this is of course something the people that end up managing this will have to explore and decide on.

[wojtekmach]

Yup, thanks for clarifying that.

Fwiw, I believe we could schedule ~100 action runs by triggering that many workflow_dispatch events (each with different arguments) but yeah, unclear if we’d run into any rate limiting and such.

[starbelly]

Yeah, I can see a situation where by you move to GH actions, then you run into resource problems, so you end up making self-hosted runners, and then you might as well kept on using bob :)

Edit:

To Hex team, can you clarify the desire for wanting to move away from Bob for at least some things?

[josevalim]

@starbelly the reason we want to move away is because sometimes the instructions for Erlang/Elixir change, so Bob needs to track a huge matrix of versions, and if that was within each repository (Erlang, Elixir, etc), life would be much easier. :)

[ferd]

I figure nothing here has changed? Unless we want to discuss this in the next meeting, we should change the tag on this or turn it into something else than an agenda item.

[tsloughter]

No changes but an important change to OTP was merged. I can't find the PR at the moment... But basically there is no need to run install after unpacking a pre-built OTP release, so it makes it easier to install. Something I've wanted for adding binary support to erlup.

[wojtekmach]

@tsloughter maybe this one erlang/otp#5427?

[tsloughter]

Yup, that's it