Enforcing semver

[giacomocavalieri]

Hello! I'd like to start this conversation to understand if this is a wanted feature and what it would look like in Gleam. I think it would be really nice if the build tool could enforce package authors to follow server rules when publishing new versions of a package, like Elm or cargo semver checks do.

How would this work

When publishing a package on hex the compiler also adds a package interface file with the type information of all the public modules and functions of a package. So when publishing a new package we could fetch the old interface and check the new one against the old one and deduce the minimum version bump that should be taking place. Here's all the things I could think of:

• Major bump:
  • removing a public module
  • removing a function/type/constructor/const from a module
  • changing the type of a function/constructor/const in a non backwards compatible way. There's type changes that are non breaking though: for example going from fn(Int) -> Int to fn(a) -> Int is not breaking anyone's code. We'd have to be careful about those.
  • removing a label from a function/constructor's argument
  • adding/removing a type parameter from a type (even if it's a phantom type!): going from Wibble(a) to Wibble(a, b) will break the code wherever the type is annotated
• Minor bump:
  • adding a new module
  • adding a function/type/constructor/const to a module
  • adding a label to a function/constructor's argument
  • changing the type of a function/constructor/const in a non backwards compatible way. For example turning fn(List(Int)) -> a into fn(List(a)) -> Int would not break anyone's code
• Patch bump: everything else that was not listed above

On internals

Internals are a bit tricky: making a public function/const/type/constructor/module internal will effectively hide it from the public API but it won't break anyone's code, so it would technically be just a minor change. However, I'd tend to consider that a breaking change to a library's public API so I would err on the side of being overly cautious and forcing a major version bump when turning something public to internal. What do you think?

Also, I think any change to an internal function/const/type/constructor/module should only result in a patch (or maybe minor?) bump as we say that internals are not to be relied on and could change at any time with no major version bumps.

What should this look like in practice?

I have no idea! But if this is a wanted feature we could start sketching what the error message should look like when someone's not following semver. Also I think it would be quite nice if the error highlighted why the change is required, e.g. "function wibble was removed from ...", "function wibble was added to ..." and so on.

Should this always be enforced? I think it should, if it can be avoided people will, and it would degrade the dx of others relying on their published code not to break their API unexpectedly.

[lpil]

Internals are a bit tricky: making a public function/const/type/constructor/module internal will effectively hide it from the public API but it won't break anyone's code, so it would technically be just a minor change. However, I'd tend to consider that a breaking change to a library's public API so I would err on the side of being overly cautious and forcing a major version bump when turning something public to internal. What do you think?

It would be a breaking change as the item has left the public API.

Also, I think any change to an internal function/const/type/constructor/module should only result in a patch (or maybe minor?) bump as we say that internals are not to be relied on and could change at any time with no major version bumps.

They're not part of the public API so they would not be taken into account at all.

Should this always be enforced? I think it should, if it can be avoided people will, and it would degrade the dx of others relying on their published code not to break their API unexpectedly.

I think a loud warning would be good as there's times which a correctness bug would require a breaking change to be made, but that change is required to fix the critical bug. In that case you would want all the users to get that change ASAP

[giacomocavalieri]

I think a loud warning would be good as there's times which a correctness bug would require a breaking change to be made, but that change is required to fix the critical bug. In that case you would want all the users to get that change ASAP

You're totally right, I didn't think about this possibility!

And thanks for clarifying on the internal stuff, that makes sense. So I assume this is a wanted feature, should I open an issue?

[lpil]

Sounds good, though I think we've some little details to figure out still.

I think we would also want to have a command so folks can determine what version they should use, or even to do the bump automatically. Would gleam publish bump for you?

How would Gleam know which version is the previous one? It's normal to be updating multiple major versions at the same time, so we can't safely pick the most recently published version.

[sbergen]

Minor bump:

• adding a function/type/constructor/const to a module

Shouldn't adding a constructor to a non-opaque type be a major bump, as it breaks exhaustiveness of pattern matches?

[lpil]

Adding a variant is a breaking change, aye.

[GearsDatapacks]

Here's a proposal for how the feature would work based on the previous discussion here:

• Since semantic version is well ordered, if the user already specifies which version they want to publish, we can figure out which version precedes it, and use that to compare.
• If the user wants the compiler to figure out the changes for them, we can by default use the latest version to compare, and add a way for the user to specify a different base version instead.

I'm not sure exactly how this would look, but for example:

# Uses the version specified in `gleam.toml` to find the previous version to compare with
$ gleam publish
# Compares with the latest published version
$ gleam publish --choose-version
# Compares with v1.2.1
$ gleam publish --choose-version --after v1.2.1

Alternatively, we could use the current version in gleam.toml as the most recent one. If you're publishing an update to 1.2.1 and you haven't specified a new version yet, there's likely 1.2.1 in your gleam.toml.

[lpil]

Why would we want to let the programmer pick a release other than the previous one?

[GearsDatapacks]

The picking another release would only be in the case where the programmer is asking the build tool which version to use next, and they don't already know which version they're trying to publish.
Maybe that's not useful enough to warrant inclusion though

[lpil]

Let's keep the API as small as possible for now, and we can add stuff as we learn more about their use.

[GearsDatapacks]

Okay, sounds good. I can start implementing this then