aqua-registry-g2

The second generation of aqua-registry.

A registry aqua reads as JSON rather than as YAML it has to evaluate. Why that is, and what aqua-registry couldn't do well, is in docs/why.md.

Status

This is still work in progress. We don't accept any issues or pull requests from contributors now.

Contributing

The definitions are generated by ar2 and arrive as pull requests, which merge when their checks pass.

• CONTRIBUTING.md: For users and outside contributors.
• MAINTAINING.md: For maintainers.
• CONSUMING.md: For developers of other tools that read this registry.

Branches and Directory Structure

One branch per package, plus main. A package branch is an orphan: it shares no history with main or with any other package, which is what keeps one package's commits out of another's.

Most of these files are generated and maintained by GitHub Actions, so they don't need to be updated by hand.

Package branches

A package branch is named pkg_<id>.

.github/workflows/test.yaml
registry.yaml
versions.json
versions/
  <escaped version>/
    registry-1.json

• registry.yaml: the definition of the package. It holds what a release can't be read for, and registry-1.json is generated from it. It also names the package, which is how the branch says which one it holds -- its own name doesn't.
• versions/<escaped version>/registry-1.json: the static registry file, which is what aqua reads. One per version, written once. We call it registry.json for short.
• versions.json: the versions above as one list, with the date each release was published and the digest of the file the registry serves for it. It is derived from versions/, which stays the thing that decides what the registry holds.

Schema version

The 1 in registry-1.json is the major version of the registry schema, not a counter. There is only one schema so far.

• A change a reader of the current schema can still read, such as a field it can ignore, keeps the file name. A reader should ignore fields it doesn't know.
• A change it can't read arrives as registry-2.json, written beside registry-1.json rather than replacing it, so a reader that knows only the first keeps working.
• For a while after a new schema arrives, new versions get both files. When that stops, new versions get only the new one, and a reader that knows only the old schema doesn't get them.
• A published file is never deleted.

So a reader asks for the newest registry-<major>.json it knows, and steps down to older ones if that isn't there.

The branch name

A branch is named after the package's id: a number this registry hands out once, and never changes. The package's name is not in it.

package	id	branch
cli/cli	1790772769	pkg_1790772769
kubernetes-sigs/kustomize	1790772903	pkg_1790772903

Because a name changes. A repository is renamed or transferred, or a package is split out of one, and a branch named after the package then has to be moved -- while everything that wrote the old name down is left pointing at nothing. An id doesn't move, so what a rename changes is one line in a table.

names.json is that table. The definition on a branch names its own package, so what the table says can be checked against what the branches say rather than believed.

Escaping a version

A version is a directory under versions/, and its name is escaped: every character outside [A-Za-z0-9.-] becomes an underscore and two lowercase hex digits. The underscore is escaped too, which is what makes the mapping reversible where turning a slash into two underscores would not, since versions contain underscores.

version	directory
v1.2.3	v1.2.3
kustomize/v5.8.1	kustomize_2fv5.8.1
@yarnpkg/cli/4.16.0	_40yarnpkg_2fcli_2f4.16.0
apps_v1.80.0	apps_5fv1.80.0

Nearly every version needs no escaping. What it is for is the ones that hold a slash: written as they are, one version's directory would be another's parent, and the versions a package has couldn't be told from the first segments of their tags -- every kustomize release would be a directory called kustomize. The result holds no slash, and stays within [A-Za-z0-9._-], so a raw URL needs no further escaping.

main

main holds no package.

.github/workflows/
ar2.yaml
index.json
names.json
template/ # template of package branches

• index.json: the package list. This is what aqua g searches.
• names.json: what resolves a name. It maps every package's name to the id of the branch holding it, and every name a package used to have to the name it has now. It is rendered from index.json in the same commit, so the two can't describe different registries.
• ar2.yaml: this registry's configuration for ar2.
• template/: the files a package branch starts with.