Use of an experimental gRPC API in pdata

[mx-psi]

Please be aware that this change adds as a dependency experimental gRPC APIs that are explicitly subject to change. If we were to move, remove, or change our library, your users would no longer be able to compile your library. You should not depend on our experimental APIs.

https://github.com/grpc/grpc-go/blob/master/Documentation/versioning.md

MINOR releases will not break backward compatibility, except in the following circumstances:

• An API was marked as EXPERIMENTAL upon its introduction.

If these features are essential to you, we should have a meeting and discuss a path toward stabilization or find another way you can access them via another stable API.

Originally posted by @dfawley in #13719 (comment)

[mx-psi]

@dfawley It would be useful to understand why these packages cannot be published as separate modules. This should solve most of the concerns from your side about compilation, wouldn't it? (at least, it would just make it 'as bad' as depending on any specific 0.x library)

[dfawley]

I'm not sure how that solves any problems. These packages are direct dependencies of grpc itself. So if we (gRPC) want to update them, we will also update our dependency of the experimental package. So when a user updates grpc before otel, the version of the experimental library used by otel will be updated, too, and then fail to compile.

(Edit: clarified "the version of the experimental library used by otel")

[dfawley]

Just to keep the discussion going:

Perhaps you can separate your new grpcencoding package into another module that is only released at v0 and is opt-in for your users until we can stabilize these APIs?

[dfawley]

Also, cc @bogdandrutu (the original PR author that added this dependency)

[mx-psi]

Apologies for the delay. I had not realized that "These packages are direct dependencies of grpc itself." which complicates things a bit. Nonetheless I still think we can avoid this kind of situation with a v0 module: we have no way to programatically tell which APIs are experimental right now from gRPC.

Perhaps you can separate your new grpcencoding package into another module that is only released at v0 and is opt-in for your users until we can stabilize these APIs?

The grpcencoding package is internal, we control all of its usages

[dfawley]

we have no way to programatically tell which APIs are experimental right now from gRPC.

Unfortunately, Go itself provides no way to programatically signal this, either: golang/go#34409

We do document it pretty clearly, however:

// # Experimental
//
// Notice: All APIs in this package are EXPERIMENTAL and may be changed or
// removed in a later release.

I accept that this is an imperfect system. To this end, we do have a directory (semi-recently created) called "experimental" that we try to put all our new experimental APIs inside. Unfortunately, it's hard to move everything that already exists into it, especially because we know there are other popular packages out there using our experimental APIs in their stable releases (see e.g. etcd-io/etcd#15145), because we would break them with the move.

Nonetheless I still think we can avoid this kind of situation with a v0 module

I think you are right in this particular instance. There are other cases, though, where we have experimental APIs that are inside a stable package (example), so we can't easily use a v0 module for existing cases like that. But also, maintaining a multi-module repo that has inter-dependencies can be pretty painful to coordinate changes across boundaries, so I'd rather avoid the separate module approach if at all possible.

The grpcencoding package is internal, we control all of its usages

Right - what I'm suggesting is that instead of turning it on by default by importing it internally, you provide an optional module and/or package, and then users can import it if they want that feature and agree to accept a risk of breakage due to it being v0.

Regardless of what grpc does, it seems you would need to do something like this in order to resolve the use of an unstable gRPC API (without just deleting its functionality entirely).

[dfawley]

I would like to make sure this issue isn't forgotten. If we were to make changes here, which we have clearly documented that we might do, all of your users will be impacted.

If this isn't optional functionality that you can move to your own v0 module or a separate, non-default, experimental package, and if you can't or don't want to vendor gRPC, then I would probably recommend rolling back the change that added these dependencies and then we can discuss options for having a stable way of doing what you need.

[mx-psi]

@bogdandrutu we need your input here