Add cooldown period for new tools update

[luboszima]

Hi, for security reasons I would like to have a cooldown period for all the newest updates for my tools.

Is it possible to add this functionality?

Similar functionality is implemented in Dependabot.

[ericcornelissen]

I'm 100% in favor of this suggestion. And I think this makes sense for all variants of the action (i.e. ericcornelissen/tool-versions-update-action, ericcornelissen/tool-versions-update-action/commit, and ericcornelissen/tool-versions-update-action/pr).

Unfortunately, I don't think asdf supports this feature and I'm not sure how we would implement it manually for any arbitrary tool 🤔

[luboszima]

@ericcornelissen Good point. After analyzing the plugin architecture, I've identified that each plugin implements bin/list-all differently, which creates challenges for automation.

Current plugin discovery:

$ asdf plugin list --urls
gum            https://github.com/lwiechec/asdf-gum
helm           https://github.com/Antiarchitect/asdf-helm.git
kubectl       https://github.com/asdf-community/asdf-kubectl.git

We can match these plugins with what's in .tool-versions to see which tools we're actually using.

The problem is getting version info automatically from all these different plugins. Each plugin checks for versions differently. For example, postgresql doesn't use GitHub to get the latest releases; instead, it goes directly to https://ftp.postgresql.org/pub/source/ to check the latest versions. Other plugins use standard GitHub.

If all plugins used GitHub, we could use the standard GitHub API to get release info from all these tools/plugins, but with these different implementations for each plugin, it's more complex.

Two ways to handle this:

• Create a simple list matching each plugin to where it gets versions (but we'd have to update it manually)
• Write a script that figures out where each plugin gets its versions automatically (better but harder to build)

The second option is probably better - letting the script automatically find version sources - though it would be harder to implement.

[ericcornelissen]

I don't think it makes sense for this project to implement logic that performs this check for all kinds of origins, whether it can find the source automatically too or not - unless it's significantly simpler and more robust than I'm imagining.

If you want to write such a script for the plugins you're using and require changing from this action to support that workflow, I'm open to accepting the necessary adjustments.

One thing you could do, and I think would be beneficial more generally, is to suggest to the asdf project to add support for this. Of course, that would still require plugin authors to provide the corresponding command, but to me that seems like both a more appropriate solution and a solution with benefits beyond this action.

[ericcornelissen]

Here's an idea: what if instead of going by the release date of the version of the tool, we go by the number of times this action has seen that version of the tool. If you run daily, that should be a good proxy for a number of cooldown days (similarly for weekly, etc.).

As an example: consider that you run this action every night and helm gets an update (say v2) on November 1st. Your action runs that night, that's the first time it saw helm@v2 so it does not update. The next night it runs again, that's the second time it saw helm@v2. Now, if you cooldown policy is 2 it would update because this is the second time it saw this version.¹

One shortcoming of this approach (at least the simplest implementation I can think of, as described in the next paragraph) would be that if helm@v3 was released on November 2nd we'd no longer consider helm@v2 and instead go straight to a helm@v3 update on November 4th - which I think is a fine trade-off.

The way I would probably try to implement this is as follows: we add a cache entry for this action that's just a text file of all the latest versions we've seen, updated every time the action is run. If we are considering updating to a new version we check if this this text file contains that tool version X times and if it does we update. The default could just be 0 which would preserve the default behavior.

Thoughts @luboszima?

Footnotes

1. of course we could also require seeing it three times when the cooldown policy is 2, that's just an implementation detail. ↩

[luboszima]

@ericcornelissen That's also a good solution, and I agree with it - simple and effective.

I'm also considering an edge case, though I don't think it needs to be a major concern: if someone wants more precise control over the cooldown period for individual tools from .tool-versions to align with their specific release cycles, they wouldn't be able to configure this, as the discussed solution applies globally to all tools.

---

Alternatively, I'm thinking about implementing version constraints similar to Terraform providers.

The implementation could involve adding a new input parameter, such as versions, where tools and their version constraints would be defined. This would allow us to specify update restrictions, such as limiting updates to patch versions only - for example, yamllint ~>1.31.0.

Example configuration:

    versions: |
      shfmt 3.6.0
      yamllint ~>1.31.0

I like both approaches for handling versions, either through your proposed solution or via version constraints.

[ericcornelissen]

I'm also considering an edge case, though I don't think it needs to be a major concern: if someone wants more precise control over the cooldown period for individual tools from .tool-versions to align with their specific release cycles, they wouldn't be able to configure this, as the discussed solution applies globally to all tools.

I see, that's an interesting problem. I think, definitely for a first version, it's fine if we have a global cooldown - afaik Dependabot's cooldown is also "global" (tho I guess you can use patterns to get cooldowns for specific groups of dependencies).

In the future, I think it would be feasible to use a .tool-versions-like syntax for expression per-tool cooldowns. So for your example it would (in an ideal state) look something like:

cooldown: |
  gum      3
  helm     1
  kubectl  4

Alternatively, I'm thinking about implementing version constraints similar to Terraform providers.

The implementation could involve adding a new input parameter, such as versions, where tools and their version constraints would be defined. This would allow us to specify update restrictions, such as limiting updates to patch versions only - for example, yamllint ~>1.31.0.

Example configuration:

    versions: |
      shfmt 3.6.0
      yamllint ~>1.31.0

I like both approaches for handling versions, either through your proposed solution or via version constraints.

Again, this seems like a reasonable proposal. However, it seems to me to be independent of the present issue. If you would use this feature, please open a separate issue for it.

Regardless, I foresee two (related) problems with this. Again, I don't think asdf has support for this. As a result, I'm not sure which version constraint syntax we'd use for this. You're using Terraform's, I'm used to SemVer ranges, tools possibly use either or something else (yamllint probably follows whatever they use in the Python ecosystem). If asdf had a preferred syntax, we'd just use that, but now we'd have to choose something, which complicates things...