Skip to content

Conversation

@pietroalbini
Copy link
Member

@pietroalbini pietroalbini commented Jan 6, 2026

This PR adds a set of CI jobs to try and prevent nondeterminism in the build process. While they are not a guarantee that Hubris will be reproducible, it will hopefully catch most cases where we accidentally introduce nondeterminism.

There are two new jobs, reproducible-a and reproducible-b, building the same board (I arbitrarily picked cosmo-b, it can be anything). The first job uses the standard Ubuntu runner with little to no modification, while the second job tries to mutate the build environment as much as possible to introduce sources of nondeterminism. The code of the second job is heavily commented to describe each source and how we introduce it.

Inspiration for which variations to introduce was inspired by both past personal experience and the documentation provided by the reproducible-builds project. There is a source I wanted to include (different UIDs) but didn't, and that's because running things as different users inside of GitHub Actions was breaking too many things to be worth it.

There is then a final new job, reproducible-check, that gets the artifacts from both builders and compares them with diffoscope, another tool by the reproducible-builds project. In case there are differences it will upload the diffoscope report as an artifact for analysis. This is a run where I intentionally induced nondeterminism, if you want to look at how things would be in case of a failure (there is a link to the diffoscope report in there).

If there are other sources of nondeterminism we want to try and inject let me know and I'll see how feasible it is to add them.

Fixes #2299

@pietroalbini pietroalbini force-pushed the ea-repro branch 19 times, most recently from b8e9290 to f2b3afd Compare January 7, 2026 19:21
@pietroalbini pietroalbini changed the title WIP: Add job checking for reproducibility issues Add job checking for reproducibility issues Jan 7, 2026
@pietroalbini pietroalbini marked this pull request as ready for review January 7, 2026 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add CI job(s) testing for reproducibility

2 participants