Should opentelemetry-configuration be merged into the spec #4935
Description
Continuing discussion from the 3/10 spec SIG.
Should opentelemetry-configuration be merged in the spec repository?
Some considerations:
Split brain
With the current setup, parts of declarative configuration spec live in opentelemetry-configuration and parts live in opentelemetry-specification, with different maintainers. In #4916 I propose adjusting the spec contribution process to require corresponding proposals to modify the opentelemetry-configuration schema (if applicable). This illustrates the split brain - it would be better if spec changes and corresponding config schema changes could be made in lock step in a single PR, rather than requiring changes to two repos with different groups of maintainers.
Versioning
Right now opentelemetry-configuration is on a different version number and cadence than the spec, with opentelemetry-configuration on 1.0. and the spec on 1.55.0. The spec has a monthly release cadence. opentelemetry-configuration releases as needed (similar to opentelemetry-proto).
Many spec changes will not impact opentelemetry-configuration, and so aligning the versions would be noisy with lots of noop releases for opentelemetry-configuration. Lots of versions of config is undesirable because we encode the version into the file_format: property. More versions means higher likelihood of misalignment between a given config files file_format and the version the SDK implementation uses. This will lead to FUD. Additionally, we publish each version to schemastore for better IDE integration. Anecdotally, I haven't seen examples of other JSON schemas in schemastore with the large number of versions that a monthly release cadence implies.
This noise might be reduced with discussed tighter integration between opentelemetry-configuration and semantic-conventions - semantic-convention standard instrumentation options are encoded into the config schema at .instrumentation/development. With more frequent changes reflecting semantic-conventions, the number of noop releases would be reduced.
It's also possible to merge opentelemetry-configuration into the spec and maintain different release cadences. Though this messy and would likely result in increased confusion, so I would personally rule this out.
Different maintenance expectations
Maintaining opentelemetry-configuration requires knowledge of JSON schema, the modeling rules to help ensure we model things consistently, and the tooling we've developed to compile and validate the schema. All things the spec maintainers are capable of learning. But as it stands, the @open-telemetry/configuration-maintainers are more experienced / in the loop.
We could mitigate this issue using codeowners to give current config maintainers more authority over the config schema and requiring their approval before merging PRs that touch those bits.
That's all the points I remember from the SIG. If I forgot any, please add comments.
cc @open-telemetry/configuration-approvers