Use GH API to collect versions from remote test harnesses #1916
Conversation
OptimumCode
force-pushed
the
collect-remote-versions-for-trends
branch
from
c52fa54 to
97fe15c
Compare
| gh api \ | ||
| -H "Accept: application/vnd.github+json" \ | ||
| -H "X-GitHub-Api-Version: 2022-11-28" \ | ||
| /repos/$GH_ORGANIZATION/$impl/git/matching-refs/tags/v | jq -c '[ .[].ref | ltrimstr("refs/tags/v")] + ["latest"]' > $MATRIX_VERSIONS_FILE |
There was a problem hiding this comment.
question: the last existing version will be displayed as latest. Could it cause any issues with creating a graph, @Julian? Also, does the order matter? Should the latest version be the first on the list of versions?
There was a problem hiding this comment.
I think latest hopefully shouldn't be an issue -- we'll have the latest version tagged also with its version number right? so it appears twice? If not (if it only appears once named latest) then I think we should fix that, we do want the version number in the UI.
Also just to be sure, it seems like maybe there's an issue here if someone adds a tag starting with a v -- maybe we should make a note to add validation to harness repos which fails if there's such a tag pushed.
There was a problem hiding this comment.
Also clearly all the shell code would be nicer if it were built-in to bowtie :( rather than here in CI -- I wanted to do that at some point as well
There was a problem hiding this comment.
If not (if it only appears once named latest) then I think we should fix that, we do want the version number in the UI.
Yes, you are right, it will only appear once as latest (because we don't keep a tag with version for the latest one - we would have to move it on every push to the main while the implementation's version is the same).
What do you think, @Julian, if we extract the version using bowtie info command right here in this case?
maybe we should make a note to add validation to harness repos which fails if there's such a tag pushed
There is one case, when we actually want to push a tag with a version manually - it is when we move out a test harness which already has matrix-versions.json file. We would need to create a tag for each version manually (manually here means - not from GH workflow). But we could create a workflow, for example, which leaves an annotation on the commit to which the tag is attached (if this tag was pushed by someone other than GH actor). This should highlight that it might break something.
Also clearly all the shell code would be nicer if it were built-in to bowtie
I agree. Do you think it should be something similar to bowtie filter-implementations? A command to show all available versions for a particular implementation. We could create an issue for that (if it is okay to implement later)
There was a problem hiding this comment.
What do you think, @Julian, if we extract the version using bowtie info command right here in this case?
I think yeah that using bowtie info sounds like it might solve quite a few problems rather than calculating it externally!
There is one case, when we actually want to push a tag with a version manually
The kind of case I was talking about was someone creating a tag called very-important-thing -- which starts with a v but clearly isn't a version at all, but I think would cause issues on the linked line!
Do you think it should be something similar to bowtie filter-implementations? [...] We could create an issue for that (if it is okay to implement later)
Yeah that seems reasonable -- an issue for later seems sufficient for now!
There was a problem hiding this comment.
The kind of case I was talking about was someone creating a tag called very-important-thing
The problem is that we don't have a unified version format across the implementations (e.g. blaze impl has a commit hash as a version and justinrainbow-json-schema had a dev-master as a version until recently). So, the version can be almost anything... If we move the version collection inside the bowtie we maybe could try to check if the image with such a version exists at all and not consider a tag as a version if the image for it does not exist
(And I just checked - the Golang versions have v at the beginning... We will have tag vv* in this case - we probably need to check for that and remove one v)
There was a problem hiding this comment.
Re tags starting with v that are not versions:
When we move version collection inside the bowtie we could use GH releases API to collect versions (test harness will not have any releases that do not correspond to a version). We can't use it right now because the GH api for releases uses pagination and bash code to handle it will look pretty ugly...
There was a problem hiding this comment.
That sounds perfectly fine with me!
The PR adds a collection of version information from remote test harnesses. It collects all available version tags (like
v*), plus it uses thelatesttag to get the latest available test harness version.Related to #1849
📚 Documentation preview 📚: https://bowtie-json-schema--1916.org.readthedocs.build/en/1916/