Boundary and interaction with OTel SemConv and Weaver

[danielgblanco]

OpenTelemetry has existing (and mature) definitions for Semantic Conventions. These already define most of what we consider "good" telemetry from the perspective of someone writing instrumentation code. Weaver provides the necessary tooling to manage semconv registries and telemetry schemas. Most crucially, it provides tooling to evaluate compliance of telemetry data against a given telemetry schema, including support for advisors.

On the other hand, some of the tooling that Weaver provides, like custom advisors, can provide a consolidated approach to evaluate some of the rules codified in this Instrumentation Score. A consolidated approach for both SemConv and instrumentation score would benefit end-users, so steps must be taken to ensure that these two projects remain aligned.

I believe Instrumentation Score is complementary to SemConv and Weaver, and leaning towards data at rest and over long periods of time, and also focusing on the operational aspect with opinionated best practices on top of OTel data. I believe SemConv/Weaver to be more focused on the producer of telemetry, and Instrumentation Score more focused on the end usage of that data (e.g. I don't think Weaver intends to measure aspects like quality of context propagation).

In any case, although these aspects may, with time, be intrinsic to nature of each project, I think we should state it clearly to avoid confusion for users and contributors.

[danielgblanco]

One aspect to consider, mentioned in the issue linked above, is if we want the Instrumentation Score to focus on telemetry quality for the purpose of observability. One can capture telemetry for multiple purposes, e.g. auditability, security, observability. Those are qualities of a system (i.e. is it auditable? is it secure? is it observable?).

I think that would be a good segmentation with semconv. Semconv cares about good telemetry from the perspective of data correctness, while Instrumentation Score can operate above and care about good telemetry applied to a domain. I was thinking in a similar way to what ISO/IEC 25000 SQuaRE differentiates between "product quality" and "quality in use".