Add support for OTEL_SEMCONV_STABILITY_OPT_IN (#492)

[zeitlinger]

Adds declarative configuration for semantic convention version selection, aligned with semantic-conventions#3424.

Configuration

Per-domain semconv config (semconv object)

Under .instrumentation/development.general.<domain>.semconv, supports:

• version (integer) — target semconv version (e.g., 1)
• experimental (boolean) — use experimental conventions
• dual_emit (boolean) — also emit the previous stable major version

Domains: code, db, gen_ai, http, k8s, messaging, rpc

Env var compatibility (stability_opt_in_list)

A comma-separated string at .instrumentation/development.general.stability_opt_in_list follows the format of OTEL_SEMCONV_STABILITY_OPT_IN. Domain-specific semconv properties take precedence.

Fixes #492

[jack-berg]

Couple of minor comments, but I like it!

cc @open-telemetry/specs-semconv-approvers please take a look and confirm this makes sense.

[trask]

could also make sense to scope somehow under

instrumentation/development:
  general:
    http:
    database:
    rpc:

[jack-berg]

could also make sense to scope somehow under

Yeah, adopt a convention to nest a standard stability level property under each instrumentation domain type, like:

instrumetnation/development:
  general:
    http:
      stability_opt_in:
    database:
      stability_opt_in:
    rpc:
      stability_opt_in:

[jack-berg]

Actually, neither of these approaches is compatible with env var substitution of the equivalent OTEL_SEMCONV_STABILITY_OPT_IN:

instrumentation/development:
  general:
    # this doesn't work because on primitive types are candidates for substitution and this property is an array
    semconv_stability_opt_in: ${OTEL_SEMCONV_STABILITY_OPT_IN} 
---
instrumentation/development:
  general:
    http:
      # this doesn't work because the property would expect a single value while the env var is a comma separated list
      stability_opt_in: ${OTEL_SEMCONV_STABILITY_OPT_IN} 

We have schema modeling guidance that allows us to have alternative versions to accommodate env var compatibility:

In instances where there is a type mismatch between the JSON schema and equivalent standard env var, an alternative version of the property may be provided to resolve the mismatch.

Applied here, I think that would look something like:

instrumentation/development:
  general:
    # comma separated list string following format and semantics of OTEL_SEMCONV_STABILITY_OPT_IN
    stability_opt_in_list: ${OTEL_SEMCONV_STABILITY_OPT_IN}
    http:
      stability_opt_in: stable

With some well defined merge mechanic between .instrumentation/development.general.stability_opt_in_list and .instrumentation/development.general.http.stability_opt_in.

[zeitlinger]

@jack-berg please check again

[lmolkova]

JFYI similar config in collector - https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/rfcs/semconv-feature-gates.md

• receiver.hostmetrics.EmitV1ProcessConventions
• receiver.hostmetrics.DontEmitV0ProcessConventions

/cc @mx-psi

[jack-berg]

Mostly nits at this point.

This is exciting - I think we're accidentally (or maybe not) making progress towards stable by default since according to this schema, by default only stable instrumentation is produced.

[lmolkova]

Looks great to me. I left a comment about GenAI - I think there is some ambiguity there.

[zeitlinger]

all addressed now

[zeitlinger]

All review comments addressed.

[trask]

re-approving, thanks @zeitlinger!