Add Go instrumentation watcher for the ecosystem registry by mikeblum · Pull Request #665 · open-telemetry/opentelemetry-ecosystem-explorer

mikeblum · 2026-06-07T23:28:04Z

Summary

A Go instrumentation watcher for opentelemetry-go-contrib: clones the repo,
extracts metadata + telemetry via Go AST, and emits a versioned, content-addressed
inventory at ecosystem-registry/go/contrib/v{version}/instrumentation.yaml.

Scope is the components that instrument a developer's code — the 14
instrumentation wrappers (gin, grpc, http…) and 5 bridges (zap, logrus…), 19
libraries total. Pipeline-config components (exporters, propagators, samplers,
detectors, processors) are out of scope: they configure the SDK, not a target library.

Core responsibilities

Maps to all five in watchers-registry-consumers.md — see the watcher README's "Core Responsibilities" section for per-function detail:

#	Responsibility	Status
1	Version management	✅ latest release tag + `main` SNAPSHOT, per-module versions, `VersionExists` idempotency
2	Schema evolution	✅ explicit `file_format: 0.1`, metadata from each module's own `go.mod`
3	Change detection	⚠️ versioned + content-addressed for diffing; auto GitHub-issue creation deferred
4	Data regeneration	✅ `make sync` per tag, idempotent releases, SNAPSHOT refreshed
5	Deterministic output	✅ stable-sorted arrays, no timestamps, SHA-256/12 hash — verified byte-stable across runs

Test plan

go test ./... (unit + integration, incl. determinism + uniqueness tests)
e2e make sync → v1.44.0 + SNAPSHOT, 19 libraries, byte-identical across two runs
Reviewer validation

Notes

#3 auto-issue creation is the one deferred responsibility (README roadmap).
ecosystem-registry/go/ files are produced by nightly CI, so not in this diff.

…ging, and build tooling

…contrib repos

…data

… into Library records

…ed inventory

…gle watcher entrypoint

Walk the go-contrib subtrees that instrument a developer's code: the instrumentation wrappers (gin, grpc, http…) and bridges (zap, logrus…). The other components (exporters, propagators, samplers, detectors, processors) configure the SDK pipeline rather than instrument a target library, so they are out of scope for the instrumentation inventory.

…osystem-registry

…egistry

…e module path

…ntory and registry

netlify · 2026-06-07T23:28:09Z

❌ Deploy Preview for otel-ecosystem-explorer failed.

Name	Link
🔨 Latest commit	`0e6e697`
🔍 Latest deploy log	https://app.netlify.com/projects/otel-ecosystem-explorer/deploys/6a2ecaad991be1000862c81c

…ate inventory YAML Spans with no attributes (e.g. kind: internal in packages with no matching semconv) were emitted alongside spans that had attributes, producing inconsistent inventory entries. Filter them at extraction time. Adds TestExtractSpansFiltersEmptyAttributes (analyzer) and TestInventoryYAMLValidation (inventory) to catch regressions.

…pan filter fix

mikeblum · 2026-06-08T03:00:02Z

📦 Rendered output preview

The inventory files themselves are not in this PR's diff — they're produced by the nightly CI automation (ecosystem-registry/go/ is generated, like the other watchers). This comment shows what make sync actually emits so reviewers can evaluate the output shape without running it.

Generated from a local make sync against opentelemetry-go-contrib:

Release: v1.44.0 → ecosystem-registry/go/contrib/v1.44.0/instrumentation.yaml (sha256/12 77c946b0d191)
Snapshot: v1.44.1-SNAPSHOT from main (sha256/12 e2da7f6fd844)
19 libraries, 948 lines each. Byte-identical across repeated runs (determinism verified).

Libraries (19)

Scope is components that instrument a developer's code: instrumentation wrappers + log bridges. Pipeline-config components (exporters, propagators, samplers, detectors, processors) are intentionally excluded — see PR description.

display_name	type	module version
logr	bridge	v0.19.0
logrus	bridge	v0.19.0
prometheus	bridge	v0.69.0
slog	bridge	v0.19.0
zap	bridge	v0.19.0
AWS	wrapper	v0.69.0
Echo	wrapper	v0.69.0
Gin	wrapper	v0.69.0
gRPC	wrapper	v0.69.0
Host	wrapper	v0.69.0
HTTP	wrapper	v0.69.0
HTTP Trace	wrapper	v0.69.0
Lambda	wrapper	v0.69.0
MongoDB (v1)	wrapper	v0.69.0
MongoDB (v2)	wrapper	(no release tag yet)
Mux	wrapper	v0.69.0
RESTful	wrapper	v0.69.0
Runtime	wrapper	v0.69.0
X-Ray Config	wrapper	v0.69.0

Note for reviewers: some libraries share a display name (e.g. MongoDB v1 and v2 drivers). The name field is derived from the full module path to keep records unique. A driver showing an empty version has no per-module release tag yet — it picks one up once go-contrib tags it.

Example record — `otelecho` (spans + metrics fused)

- name: instrumentation-github.com-labstack-echo-otelecho
  display_name: Echo
  description: ""
  source_path: instrumentation/github.com/labstack/echo/otelecho
  scope:
    name: go.opentelemetry.io/contrib/instrumentation/github.com/labstack/echo/otelecho
  module:
    path: go.opentelemetry.io/contrib/instrumentation/github.com/labstack/echo/otelecho
    version: v0.69.0
  target_module: github.com/labstack/echo
  go_min_version: 1.25.0
  library_link: https://pkg.go.dev/go.opentelemetry.io/contrib/instrumentation/github.com/labstack/echo/otelecho
  instrumentation_type: wrapper
  installation:
    type: wrapper
    description: ""
  semantic_conventions:
    - HTTP_SERVER_SPANS
    - HTTP_SERVER_METRICS
  stability: experimental
  telemetry:
    - when: default
      spans:
        - kind: server
          attributes:
            - name: client.address
              type: string
            - name: echo.error
              type: string
            - name: http.request.method
              type: string
            - name: http.response.status_code
              type: int
            - name: http.route
              type: string
            - name: network.peer.address
              type: string
            - name: network.protocol.name
              type: string
            - name: network.protocol.version
              type: string
            - name: server.address
              type: string
            - name: server.port
              type: int
            - name: url.path
              type: string
            - name: url.scheme
              type: string
            - name: user_agent.original
              type: string
      metrics:
        - name: http.server.request.body.size
          type: histogram
          unit: By
          attributes:
            - name: http.request.method
              type: string
            - name: http.response.status_code
              type: int
            - name: http.route
              type: string
            - name: network.protocol.version
              type: string
            - name: url.scheme
              type: string
        - name: http.server.request.duration
          type: histogram
          unit: s
          attributes:
            - name: http.request.method
              type: string
            - name: http.response.status_code
              type: int
            - name: http.route
              type: string
            - name: network.protocol.version
              type: string
            - name: url.scheme
              type: string
        - name: http.server.response.body.size
          type: histogram
          unit: By
          attributes:
            - name: http.request.method
              type: string
            - name: http.response.status_code
              type: int
            - name: http.route
              type: string
            - name: network.protocol.version
              type: string
            - name: url.scheme
              type: string

Example record — `otellogr` (log bridge)

- name: bridges-otellogr
  display_name: logr
  description: ""
  source_path: bridges/otellogr
  scope:
    name: go.opentelemetry.io/contrib/bridges/otellogr
  module:
    path: go.opentelemetry.io/contrib/bridges/otellogr
    version: v0.19.0
  target_module: github.com/go-logr/logr
  go_min_version: 1.25.0
  library_link: https://pkg.go.dev/go.opentelemetry.io/contrib/bridges/otellogr
  instrumentation_type: bridge
  installation:
    type: import
    description: ""
  stability: experimental

Generated by make snapshot; the nightly workflow integration is tracked separately.

…eneration Adds a repeatable pipeline for posting watcher output previews to PRs: - cmd/snapshot/ reads the generated inventory, formats a sorted library table (bridges then wrappers, each alphabetical by display_name) and selects the richest telemetry example + first bridge as featured records - scripts/upsert-pr-comment.sh finds and patches the sentinel comment () or creates a new one if absent - make snapshot: go run ./cmd/snapshot/ | ./scripts/upsert-pr-comment.sh

mikeblum · 2026-06-08T03:09:18Z

To address the uv expectation from CI ex. uv sync --locked --all-extras --dev I'm thinking one approach is a default Makefile in watcher-common that all the other watchers use for ci and we can have watchers with ecosystem-specific Makefiles. There are probably other things we need to tweak to make watchers polyglot writ large.

jaydeluca

i started reviewing this, but wanted to pause and ask if it's possible to start even smaller? it's a bit difficult to review 48 files and 8k lines of code

jaydeluca · 2026-06-09T10:39:13Z

+## Commits
+
+This watcher lives in a shared monorepo with a shared Git log. Follow
+[Conventional Commits](https://www.conventionalcommits.org/) and the scope
+convention used across this repo: the scope is the component path, abbreviated
+to the ecosystem and language.
+
+- **Scope:** `ecosystem-automation/golang` — e.g.
+  `feat(ecosystem-automation/golang): resolve go-contrib release tags`.
+- Use the standard types (`feat`, `fix`, `refactor`, `chore`, `docs`, `test`,
+  `style`, `perf`).
+- No AI attribution, `Co-Authored-By`, or tool links in commit messages.
+- Keep the log reviewable: one logical change per commit, not a play-by-play of
+  the editing session.


personally I prefer AI to not make commits at all, so let's remove this. People who use ai to make commits can put this type of thing in a personal agents file

struck it. I saw AGENTS.md and CLAUDE.md in the repo and thought it was needed.

jaydeluca · 2026-06-09T10:39:56Z

+## Footguns
+
+- The `insturmentation` typo in the module path and directory name is fixed —
+  the module is `.../golang-instrumentation-watcher`. Don't reintroduce it.
+- Snapshot writes must clean up the prior snapshot and write a replacement in
+  the same run, or the frontend sees a missing snapshot until the next sync.
+- Don't hand-edit files under `ecosystem-registry/` — they are pipeline output
+  and the immutable historical record.


im not sure we need this either

Suggested change

## Footguns

- The `insturmentation` typo in the module path and directory name is fixed —

the module is `.../golang-instrumentation-watcher`. Don't reintroduce it.

- Snapshot writes must clean up the prior snapshot and write a replacement in

the same run, or the frontend sees a missing snapshot until the next sync.

- Don't hand-edit files under `ecosystem-registry/` — they are pipeline output

and the immutable historical record.

jaydeluca · 2026-06-09T10:41:02Z

+	rm -f coverage.out
+
+.PHONY: test-perf
+test-perf: ## ⚡ Run benchmark tests


there's a ton of stuff in this file that I don't think are necessary, like benchmarks and security scans. Can you just scope this to only what's needed?

stripped this down to the basics

jaydeluca · 2026-06-09T10:42:29Z

+A **Weaver registry** (`registry/signals.yaml` + `attributes.yaml`) is also
+generated as an optional dev/validation artifact (`make dev` →
+`weaver registry check`), not as the canonical consumer output.


i think we should start small and build up to this. can we strip this down to just whats needed/included in this first step?

kicked the weaver registry work to another follow up branch

…on into feat/weaver-registry Remove all Weaver-specific code (generator, Group/AttributeRef/AttributeDef types, CalculateStats, convertTelemetryToGroups) so this branch is solely about walking go-contrib and emitting the versioned inventory. Weaver registry output lives on feat/weaver-registry branched from this HEAD.

mikeblum · 2026-06-14T15:35:17Z

i started reviewing this, but wanted to pause and ask if it's possible to start even smaller? it's a bit difficult to review 48 files and 8k lines of code

struck all the weaver registry work and pushed to another branch. What's left is the go-contrib metadata.ymls.

mikeblum · 2026-06-14T15:42:26Z

I was unsure about needing both go/contrib/vX.Y.Z and go/contrib/vX.Y.Z-SNAPSHOT. It get why as its what maven and gradle expect but it feels a bit awkward and redundant here. Is the idea we symlink instead to appease the crawler?

mikeblum added 11 commits June 7, 2026 18:27

chore(ecosystem-automation/golang): bootstrap module with config, log…

86d3dac

…ging, and build tooling

feat(ecosystem-automation/golang): check out and version upstream go-…

590b0b2

…contrib repos

feat(ecosystem-automation/golang): model instrumentation library meta…

37f3a38

…data

feat(ecosystem-automation/golang): extract telemetry via AST and fuse…

d46f66e

… into Library records

feat(ecosystem-automation/golang): write a versioned, content-address…

c9ae152

…ed inventory

feat(ecosystem-automation/golang): drive the sync pipeline from a sin…

71ff42d

…gle watcher entrypoint

fix(ecosystem-automation/golang): write inventory to the repo-root ec…

24626f3

…osystem-registry

chore(ecosystem-automation/golang): ignore the generated weaver dev r…

99e14c1

…egistry

fix(ecosystem-automation/golang): derive unique library names from th…

2124d65

…e module path

feat(ecosystem-automation/golang): emit deterministically sorted inve…

38010e1

…ntory and registry

mikeblum added 3 commits June 7, 2026 21:13

chore(ecosystem-automation/golang): fix markdownlint errors in READMEs

a4387cf

chore(ecosystem-registry/go): regenerate go contrib inventory after s…

2bcff0f

…pan filter fix

jaydeluca reviewed Jun 9, 2026

View reviewed changes

mikeblum added 3 commits June 14, 2026 10:15

rm AGENTS.md

382b0d2

cicd: strip down Makefile

f6579b5

make: remove vuln scan

0e6e697

mikeblum force-pushed the feat/golang-watcher branch from 95684ed to 0e6e697 Compare June 14, 2026 15:37

mikeblum commented Jun 14, 2026

View reviewed changes

Conversation

mikeblum commented Jun 7, 2026

Summary

Core responsibilities

Test plan

Notes

Uh oh!

netlify Bot commented Jun 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ Deploy Preview for otel-ecosystem-explorer failed.

Uh oh!

mikeblum commented Jun 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📦 Rendered output preview

Libraries (19)

Example record — otelecho (spans + metrics fused)

Example record — otellogr (log bridge)

Uh oh!

mikeblum commented Jun 8, 2026

Uh oh!

jaydeluca left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mikeblum commented Jun 14, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

netlify Bot commented Jun 7, 2026 •

edited

Loading

mikeblum commented Jun 8, 2026 •

edited

Loading

Example record — `otelecho` (spans + metrics fused)

Example record — `otellogr` (log bridge)