BB2 metadata

[mbauman]

Based on a slack discussion, I figured it'd be worth recording some things I've found in looking at the existing BB1 JLLs for GeneralMetadata.jl and how it compares with the BB2 descriptive toml. It's a huge improvement! But I think there are three places where I have thoughts with respect to my recent security work:

• The easy one: I'd like to have a record of what build dependency versions were actually used when building a particular release, not just the compat bounds that constrained it. E.g., a recent Clang_jll build is missing this.
• I'd also like a record of the buildscript and build system. This is a bit more meta-meta, but it's helpful to know more details about how the artifacts were generated. E.g., a link to the originating Yggdrasil build_tarballs and some information about the build stack itself.
• The hard one: what does src_version actually mean? Or more pointedly, what's the "src_name" that it's implicitly versioning? How can I distinguish Swift vs Swift or Atlas vs Atlas?
  • There exist BB1 Yggdrasil recipes that combine multiple projects at multiple distinct version numbers. Two examples I've found include MPItrampoline and Perl. It's more common (but still quite rare) to have have different sources/versions per architecture (e.g., Git), and that works nicely with the [[builds]] array.
  • What I think I actually want is to have all the [[build.source]]s themselves identified and versioned if at all possible — and this includes patches! It's hard to describe what "identification" and "versioning" actually mean, though.
    • For identification, often the project repository or homepage URLs can be used to anchor the namespace. There's also the ontologies defined by https://repology.org. The latter can then be used to connect against other identifiers like CPEs in the NVD and/or the vendor/product in the EUVD.
    • For versioning, you can sometimes extract a version number from a download URL or tag on a git commit, but these are both highly heuristic. You really want the version number exactly as the upstream project would use it to report a vulnerability. It's also sometimes possible for projects to report git ranges for their vulns (e.g., in the OSV spec), but that's quite rare in practice.

[staticfloat]

For some of this information it may be outright impossible to automatically collect it (thinking of upstream source versions, for instance). Is this something that can be provided on a best-effort basis, or is it critical that this is present (e.g. for all sources we require that the Yggdrasil recipe list extra metadata for each source)

[staticfloat]

I'd also like a record of the buildscript and build system.

For BB2, our toolchains are literally assembled from JLLs, so toolchain objects contain within them lists of JLLs that are assembled together in the build environment. As an example, see this docs page that just happens to have a toolchain printed out. Is that the kind of information you're looking for?

[mbauman]

Yes, versioning information is definitely best effort, and I definitely don't want to incentivize inventing phony data because that's worse than no data at all. I do think downstream consumers (possibly even the registries themselves) may end up implementing package policies based upon the existence of this data. For me, the key thing I'm looking for is automatability in the context of security advisories. It's a challenge in the context of the messy messy unstructured world of CVEs, but with this data we can do automatic due diligence for the upstream packages that have done theirs.

In the best case, if we can connect all the redistributed components to a Repology project and version, we can use their known CPE bindings to automatically flag CVEs and generate JLSEC advisories as they are published.

In the next best case, if we can connect a project to its GitHub repo, we can look to see if the upstream has itself published security advisories. I believe other git forges may have some similar functionality, too. (edit: nope, GitHub's repository GHSA is unique here)

[mbauman]

For the build script and build system, I'm thinking about the nuts and bolts of "what system built this?" and "how do I release a new version of this thing?"

I may just not know enough about BB2 yet, but I want to ensure we have the equivalent of knowing the Yggdrasil buildscript and buildkite infrastructure.

[mbauman]

Oh, it'd probably also be helpful to have some sort of versioning of the schema of the descriptive TOML itself.

[fingolfin]

From the "build recipe author" side, I'd also be happy to provide meta data that is currently missing, if only I had a means to, such as

• homepage URL
• repository URL (can be guessed for e.g. GitHub releases, but not in general)
• SPDX license identifier (though in many case this could perhaps be auto-generated based on whatever license files are installed for the JLL)
• upstream version
• repology link / ID

[mbauman]

@fingolfin wishes do come true! This can now be retrospectively populated with GeneralMetadata.jl, and I'm finally relatively content with the (documented and tested!) schema. As an example, here's the metadata for all versions of Flint_jll. Even better, I've retrospectively re-created the (BinaryBuilder1) --meta-json information for each version and lifted out the sources used, allowing you to see what the artifacts contained at each version.

And better yet, I've automatedly matched download URLs and looked up commit tag information to add an upstream project identifier and version numbers where I could. Flint is an example where we mostly got everything populated (and I think it all looks right!), except for some old versions that used some untagged commits. And, largely, remarkably, it works!

There are cases where upstream version numbers (and tagging conventions) are harder to get right, though. For example, I totally get Singular's tags wrong (they're things like Release-4-4-1p5 for version 4.4.1p5): Singular_jll.toml. Since I pulled it up and looked at it, I fixed it (JuliaRegistries/GeneralMetadata.jl#55).

---

So, @staticfloat, the place where I think we'd want to allow folks to specify this directly (so I can stop guessing wrongly) is in each AbstractSource. So these would be optional keywords, a la:

GitSource("https://github.com/Singular/Singular.git", "abc73b69185a2f369e11458d80ef222307adf869";
    upstream_project="repology.org/project/singular", upstream_version="4.4.1p5")