Go OTEL version 1.39.0 -> 1.40.0 includes non backward compatible change

[bushnell-deshaw]

Description

Go version 1.40.0 of the otel module included this change: 61f0e48.

This is a non-backward-compatible change. Correct code which used to work breaks after this change. Suppose you have a module top which imports otel at 1.39.0 and uses module 3party which directly imports the semconv/v1.37.0 package. top works fine because otel@1.39.0 also imports semconv/v1.37.0.

Now you upgrade top to otel@1.40.0, which imports semconv/v1.39.0. But now top stops working because when you attempt to register resources, otel detects that some of them are using schema "1.37.0" and some are using schema "1.39.0", and the merge can be attempted but is not guaranteed to succeed.

This violates the semantic versioning rules for Go and therefore requires a major version flip for otel.

I understand this is a difficult problem to solve given the way semconv is handling schemas, but it is very harmful to address it by pushing breaking changes under minor releases. If it is not possible to functionally use different semconv/vn.n.n modules in the same program without risk of an error arising, then any change of an import to a different semconv module may be a backwards-incompatible change.

[dmathieu]

Duplicate of #7297 (which is deemed invalid).

[bushnell-deshaw]

In what way is #7297 "invalid"?

[dmathieu]

See the label on the issue, as well as the actual solution provided in the comments.

[bushnell-deshaw]

#7297 is complaining that otel/semconv breaks backward compatibility guarantees. That's not strictly correct, and is why I opened this issue.

This issue is not saying that semconv breaks any guarantees. It is that otel itself, by including a change to import a different semconv package, creates a backward incompatibility with previous versions of the otel module.

#7297 is asking "please support more than one semconv version" and if the answer is "no, we aren't going to", that's totally fine. That addresses #7297.

But having made that decision, it now needs to be regarded as a breaking (and non-backward incompatible change) anytime otel changes its dependency. This issue is about that quite different concern.

[dmathieu]

As you mention in your PR description, the merging of semantic conventions is the process meant to help with those upgrades.
We know it can be a pain when it fails, but instrumentations stability are also decoupled from semantic conventions.
See open-telemetry/opentelemetry-specification#4813

[bushnell-deshaw]

As you mention in your PR description, the merging of semantic conventions is the process meant to help with those upgrades. We know it can be a pain when it fails, but instrumentations stability are also decoupled from semantic conventions. See open-telemetry/opentelemetry-specification#4813

I think I understand the intention. Unfortunately the Go module system's backward compatibility guarantees are very strict in their intention, and are hard to mesh with what otel is trying to do (unless you can guarantee that the merge always proceeds reasonably without an error). The workaround in the other issue sounds like "ignore the error", which is a fine answer except that it's too late to create that requirement. :) Having released v1 without an "ignore the error" requirement, it can't be added now without (you guessed it) breaking backward compatibility.

[bushnell-deshaw]

In case it's not clear, the problem here (from an operational point of view) is that Go tools assume that any minor or patch release can be somewhat freely applied in a variety of circumstances, which is in turn key to the way Go arranges to deal with the "diamond" problem where you depend on A and B, which each depend on C but at different versions. Because of the compatibility promise, Go will pick the highest version and use it, and then because C has guaranteed backward compatibility, A and B will reliably both work.

The result of this means that the importer even if they declare a specific version, does not necessarily get that version, so if there is any action they should take on an upgrade, they get broken.

[dmathieu]

Reopening this issue so you can get more opinions.

[pellared]

Closing as duplicate of #2341 (and other issues like #4476)