[service/internal/graph] Measure telemetry as it is passed between pipeline components

[djaglowski]

Depends on #12856

Resolves #12676

This is a reboot of #11311, incorporating metrics defined in the component telemetry RFC and attributes added in #12617.

The basic pattern is:

• When building any pipeline component which produces data, wrap the "next consumer" with instrumentation to measure the number of items being passed. This wrapped consumer is then passed into the constructor of the component.
• When building any pipeline component which consumes data, wrap the component itself. This wrapped consumer is saved onto the graph node so that it can be retrieved during graph assembly.

[codecov]

Codecov Report

Attention: Patch coverage is 82.98507% with 57 lines in your changes missing coverage. Please review.

Project coverage is 91.54%. Comparing base (ac520a5) to head (7c557ad).

Files with missing lines	Patch %	Lines
service/internal/graph/connector.go	65.95%	32 Missing and 16 partials ⚠️
service/internal/graph/exporter.go	85.71%	2 Missing and 1 partial ⚠️
service/internal/graph/processor.go	89.65%	2 Missing and 1 partial ⚠️
service/internal/graph/receiver.go	75.00%	2 Missing and 1 partial ⚠️

❌ Your patch check has failed because the patch coverage (82.98%) is below the target coverage (95.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #12812      +/-   ##
==========================================
- Coverage   91.67%   91.54%   -0.13%     
==========================================
  Files         503      503              
  Lines       27819    28094     +275     
==========================================
+ Hits        25502    25720     +218     
- Misses       1828     1866      +38     
- Partials      489      508      +19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jaronoff97]

some initial thoughts, overall implementation seems very sane to me though. Thank you!

[codeboten]

Thanks for moving this work forward @djaglowski, just a few questions

[djaglowski]

Thanks for the reviews.

#12817 implements a subset of this PR. If we can get that merged in first, I'll rebase to reduce the scope of this one substantially.

[MikeGoldsmith]

I think this looks really good 👍🏻

As shared in the other PR, I'd really like to see bytes counters too but this is a great start.

[codeboten]

testing this further, this appears to be the current behaviour in main, it looks like something broke in the last 20 commits

[codeboten]

Looks like things have been unhappy since this change: dc8e2dd

[dmitryax]

The PR LGTM. However, I'd like to get an agreement on #12916 before releasing this

[songy23]

I built the collector using the branch and when running it, this is what I see at localhost:8888/metrics:

I just tested and saw the same, internal metrics are broken at mainline head.

[songy23]

Looks like things have been unhappy since this change: dc8e2dd

Yes, reverting that commit fixes it http://github.com/open-telemetry/opentelemetry-collector/pull/12917

[djaglowski]

With #12933 merged, the metrics created by this PR are once again useless without the telemetry.newPipelineTelemetry feature gate being enabled, so I am moving this back to draft.

[mx-psi]

@dmitryax @codeboten I will merge this by Friday EOD Monday morning EU time unless you block by then

[codeboten]

Thanks @djaglowski, just one comment about the update of the tpm package, otherwise this looks good