Provide guidance/examples on how to define custom conventions

[lmolkova]

OTel weaver supports defining semantic conventions with dependencies

This allows to define convention registries that are based on otel, but not part of this repo, which is useful for:

• unique instrumentations that exist in collector or language contrib repositories. Such conventions are still necessary, but as a documentation / formal schema rather than a 'standard'
• native instrumentations where component authors should own their schema
• company / organization / team-specific conventions

While tooling supports it today, it goes through major improvement round (open-telemetry/opentelemetry-specification#4815), is not fully documented.

We should have

• example(s) of how it can be done in weaver-examples. PoC - [WIP] Add example of instrumentation-specific conventions opentelemetry-weaver-examples#33
• policies and default templates to help people with validation, doc- and codegen - weaver-packages and https://github.com/jsuereth/weaver/blob/main/docs/specs/default-templates/default_templates.md
• pilot project with interested SIG
• guidance and links in this repo

Most of this work needs to happen outside of this repo, but adding issue here to track the e2e and advertise the work.

[KalleOlaviNiemitalo]

Some sort of diff feature might also be useful. Scenario: I maintain an instrumentation that advertises compliance with semantic conventions 1.37.0, and I want to upgrade it to 1.39.0. If I have documented the telemetry in a format understood by OTel weaver, it could give me a list of the semantic conventions that the instrumentation claims to implement and that were changed between those versions. Then if the list is empty, I could just bump the referenced version number.

[lmolkova]

@KalleOlaviNiemitalo check out weaver registry diff, we also use it in this repo to generate current schema files