SIG meeting notes

[pellared]

OpenTelemetry Go contributors meets periodically (roughly weekly) to discuss ongoing development of OpenTelemetry Go and related topic. You can learn about the meetings here as well as join these meetings as they are open.

These meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them (create an issue with a change proposal).

This meta-issue records minutes of OpenTelemetry Go SIG meetings as issue comments, so that they can be cross-linked easily with the relevant issues. You can subscribe this to this issue to get notifications after each meeting minutes is published. This meta-issue is for minutes only; comments that are not meeting minutes will be deleted.

Meeting notes:

• 2025-04-10
• 2025-04-17

[pellared]

Date: 2025-04-10

Attendees:

• Tyler Yahn
• Robert Pająk
• Sam Xie

Transcript

Optimizing the Logs SDK Batch Processor

• Robert discussed optimizing the batching processor on high load. Damien already merged the PR.
• The current implementation copies the whole buffer indefinitely when it cannot export.
• The optimization avoids copying if it knows it won't be able to export.
• Benchmarks do not show any overhead during normal operation.
• Robert checked concurrency and found no race conditions.
• Tyler questioned concurrency issues and the behavior when the channel returns true but immediately becomes false.
• Robert explained the existing behavior would handle this by copying and retrying.
• The optimization reduces CPU load and memory copying on high load.
• Tyler questioned the lock acquisition and release, suggesting potential optimization.
• Robert agreed the locking could be optimized further.
• Robert feels it's a hack, but a good enough step for now.
• Robert plans to explore different implementations of the batching processor later.
• Robert suggested dropping the latest log record when the queue is full, which matches the span matching processor's behavior and could be more performant.
• Tyler argued for keeping the oldest record because it has had the most attempts to be exported and may be too large.
• Sam suggested the load could be coming from the latest record.
• Robert suggested asking other languages how they approach the queue-full scenario for consistent behavior.
• Tyler stated that the current behavior is the right one, the log record that's been in the queue the longest has had the most attempts to be exported, and has either failed or hasn't been attempted.

Related PR:

• Optimize Logs SDK BatchProcessor #6569

Refactoring of the Log Tests

• Robert is seeking a second approval for the refactoring of the log tests.
• The PR aims to minimize changes, primarily modifying the recorder to use a map to avoid duplicate entries for the same scope.
• Tyler has started a review but hasn't completed it.
• Robert provided a link to a PR showing how it will be reused in hotels app.

Related PR:

• log/logtest: Change Recorder.Result #6507

Audit Compliance - Logger API

• Sam discussed the observed time in the spec, noting that if the user doesn't provide an observed timestamp, the implementation should default to the current time.
• Tyler identified this as a bug in the spec, stating the API should not specify implementation details.
• Tyler requested any similar issues be identified and cleaned up.
• The consensus is that this is a bug in the spec that needs fixing.
• Tyler raised concerns about going stable on the API due to potential changes in the complex attribute approach.
• Robert suggested updating the translation function for attributes if needed.
• Tyler's concern is user experience if different packages import attributes from different places.
• Tyler suggested the need for better leadership at the spec level.
• Robert mentioned raising this issue to the TC with Pablo.
• Tyler agreed they should wait on the TC's decision.
• Robert noted the community was cautious and feared the explosion of data types, which influenced the current design.
• Tyler thinks Robert should add prioritization to the issue raised for the TC.
• Robert noted that the PR for stabilizing event name has already been merged on the open telemetry proto.

Related issues:

• Audit Compliance: Logs API Logger #5679
• Review the language around standard attributes (and breaking changes) opentelemetry-specification#4468

Transcribing Meetings and Posting in Issues

• Tyler raised the idea of transcribing meetings and posting them in issues to improve asynchronous collaboration across time zones.
• Robert showed Tyler a transcribing app.
• The goal is to make it easier for those not present to understand discussions and decisions.
• Robert and Tyler support the idea.
• Tyler showed an example from the Go.
• The issues would be locked down so only certain people can modify them.
• Robert suggested adding a note that people may subscribe to the issue.
• Robert offered to work on the transcripts.
• Robert will share docs to the other maintainers on how he's doing this it'd be great.

Contribution Fest PR

• Tyler noted that Damien looks like he's missing a review on the easily Updated PR.
• Sam will review it.

Related PR:

• feat: add example for otelrestful opentelemetry-go-contrib#7118

Release Discussion

• Robert would like to add his PR to the milestone.
• There are still many PRs and issues from Contribfest that need to be addressed.
• A release may be premature, especially with Robert's PR and other things.

Related PR:

• [DoNotMerge] sdk: implementations to embed noop instead of embedded #6638

Triaging New Bugs and Issues

• The team triaged new bugs and issues.
• Several issues were identified and assigned low priority.

KubeCon CFP

• Tyler mentioned that the KubeCon North America CFP is open.
• The Observability Days are going to open in a little bit as well.

Action Items

• Sam: Create an issue/PR to address the API specification bug related to observed timestamps. Remove the implementation details of the Observed Timestamp in the Log API opentelemetry-specification#4481
• Robert: Explore different implementations of the batching processor later.
• Robert: Add prioritization of Review the language around standard attributes (and breaking changes) opentelemetry-specification#4468 raised for the TC.
• Robert: Share docs to the other maintainers on how he's doing transcriptions.
• Sam: Review feat: add example for otelrestful opentelemetry-go-contrib#7118

[pellared]

Date: 2025-04-17

Attendees:

• Tyler Yahn
• Damien Mathieu
• Sam Xie
• Alex Kats

Transcript

Delaying HTTP Semantic Convention 1.20

• Damien Mathieu proposed delaying the HTTP semantic convention 1.20 for one release due to several bug fixes and missed instrumentations.
• Multiple fixes make Damien uneasy about switching right now.
• A pull request was merged by Sam that added semantic conventions that were not handled before.
• Tyler Yahn agreed with the delay, suggesting it would allow more time for people to switch.
• They discussed using Weaver for the otelhttp changes, which could generate code that avoids breaking changes.
• Damien mentioned discussions with Tyler and Robert during KubeCon about using Weaver for otelhttp changes.
• Tyler supported the idea of translating semantic conventions and mentioned another talk about Prometheus translation using Weaver.

Related issues:

• https://github.com/open-telemetry/opentelemetry-go-contrib/issues?q=is%3Aissue%20state%3Aopen%20label%3A%22instrumentation%3A%20otelhttp%22%20label%3Abug

Related project:

• Go: HTTP Semconv Migration

Code Owner Additions for otelgin and otelmongo

• Damien proposed adding code owners for otelgin and otelmongo.
• One request has already been approved by two folks.
• Preston Vasquez has been the de facto code owner for MongoDB instrumentations.
• Damien mentioned a PR adding MongoDB v2, which is waiting for merge.
• Damien suggested that if Preston's request is not approved, the MongoDB instrumentation should be removed due to lack of a code owner.
• Tyler agreed with the code owner additions.
• Tyler asked Alex to approve the first request in the PR as well.
• Tyler and Damien clarified that code owners still require maintainer approvals and do not have complete ownership.
• Damien trusts Preston to review PRs on the MongoDB side but will be more careful on the convention side.

Related PRs:

• Add @prestonvasquez as codeowner of mongo opentelemetry-go-contrib#7242
• otelgin: add flc1125 as a code owner opentelemetry-go-contrib#7238

Log Stabilization

• Robert is actively working on log stabilization.
• The idea of complex attributes is still being discussed at the specification level.
• The decision on how to structure the logs API is on hold until clarification from the specification regarding complex attributes.
• The events name got merged or stabilized.

Related PR:

• OTEP: Extending attributes to support complex values opentelemetry-specification#4485

Related projects:

• Go: Logs (GA) (view)
• Logs SIG

Prometheus Legacy Name Validation Scheme

• Support for the duplicated Prometheus legacy name validation scheme was kept.
• Bart from Prometheus Core indicated that the scheme is deprecated but not entirely supported everywhere, and removing it would break systems like Cortex.
• There was a request from someone in the collector for a configuration option to configure the validation scheme in the exporter.
• Damien doesn't think they will have hundreds of validation schemes in the like middle term future.
• Tyler agreed with Damien and said that he'd be hesitant to try to release their implementation and stabilize it without removing this.
• Damien is kind of waiting for David on this one to be fair.
• Tyler created a draft to track this so they don't just launch it.
• Damien doesn't think the Prometheus exporter should be released until the Prometheus Common Library is stable.
• Tyler agreed that no module should be stabilized as long as it has dependencies on unstable modules.

Related issue:

• [Prometheus] Exporter should support 'NoTranslation' mode #6668

Related project:

• Go: Metric Prometheus Exporter (GA)

CFP for KubeCon North America

• Tyler mentioned that the CFP for KubeCon North America is open, as well as the community day also in the US.
• He encouraged attendees to submit talks to promote the project.
• He mentioned the OTel CFP Channel for assistance.

Action Items

• David to look into the Prometheus Legacy Name Validation Scheme when he comes back.

[pellared]

Date: 2025-04-24

Attendees:

• Tyler Yahn
• Sam Xie
• Alex Kats
• Alex Arnell

Transcript

Release Progress

• The group is overdue for a release (approximately 2-3 months).
• A review of items blocking the release was conducted.
• Several tasks, mainly from the contrib fest and internal/testing related, were identified as candidates to be moved to the next milestone.
• Specific issues discussed:
  • CodeCov reports failing: Tyler to spend time investigating.
  • Conversion: Needs more reviews.
  • Damien's feedback is needed.
  • "Flatten the bridge" to be moved to the subsequent milestone.
• Alex Kats inquired about delaying the some cop migration by one release, which Tyler confirmed.
• A links check was identified as a disaster and will be added to the next milestone.
• Contrib repository items were reviewed, including contrib fest issues (to be moved), and items needing review or updates.
• Alex Kats has approved the AWS X-ray extract.
• otelhttp handle request pattern needs another review.
• A control flag issue was identified.
• The last release was March 5th
• Some things in the trace API, our SDK, for the auto stuff needs to actually get fixed.

Weaver Proof of Concept for Semantic Conventions

• Tyler presented a proof of concept using Weaver to generate Go client libraries for semantic conventions, primarily for metrics.
• The goal is to create client libraries with static Go types to facilitate accurate semantic convention generation.
• The generated libraries include types, instruments, and methods for adding metrics with required and optional attributes.
• The system handles enums and provides documentation.
• The current implementation doesn't support observable types.
• Tyler demonstrated an implementation using the RPC conventions to rewrite the gRPC instrumentation in otelgrpc.
• The new approach simplifies setup and handles naming, description, and units automatically.
• Sam asked about generating development-level semantic conventions, and Tyler clarified that both stable and development conventions are supported (excluding deprecated ones).
• Discussion on instrument type selection and potential overrides.
• Sam suggested users could use constant functions for descriptions/units, but Tyler suggested these might be removed in favor of the new approach.
• Tyler requested feedback on the approach and the repository.

otelhttp: Cannot Record Error for Spans

• Sam raised a bug in otelhttp where there's no opportunity to record errors to spans.
• Sam questioned whether exceptions should be recorded on spans, even with an error type.
• Tyler clarified that exceptions are generally treated as panics.
• Sam suggested that having stack traces would be helpful for debugging.
• Tyler asked about semantic conventions related to this.
• The group discussed the error.type semantic convention and whether the current implementation complies with it.
• Sam to look through this, and if you if you see that that's the case it may make sense to do.

Review Requests

• Robert requested reviews on the http.Client and AssertEqual PRs.

Additional Instrumentation Ports

• Alex Kats asked about the long-term plan for additional instrumentation ports, specifically regarding a Chi implementation request.
• Tyler explained the preference for independent packages with clear ownership, rather than including everything in the contrib repo.
• The OpenTelemetry registry is intended to help users discover different packages.
• Discussion about the benefits of clear ownership and the potential for a "spurious call to authority" if everything is in the contrib repo.
• Alex Kats raised concerns about semantic convention compliance in external libraries.
• Tyler suggested the need for a validation tool or audit tool to check compliance.
• Sam mentioned that otelsql is special because it instruments the building library.

Automated CI/CD Tooling for Telemetry Validation

• Alex Arnell expressed interest in automated CI/CD tooling to validate telemetry against specific schemas.
• He wants to ensure that telemetry data added to the Heroku platform aligns with semantic conventions.
• Tyler agreed to create an issue to track this request and encourage collaboration.

Action Items:

• Tyler: Spend time investigating the CodeCov reports failing issue.
• Tyler: Create an issue to track the request for automated CI/CD tooling for telemetry validation.
• Sam: Look through semantic conventions and see that that's the case it may make sense to do.
• Reviewers: Provide reviews on Robert's "Add WithHTTPClient" and "Add AssertEqual" PRs.