Skip to content

WIP: add telemetry - #512

Open
spencerdcarlson wants to merge 3 commits into
kafka4beam:dev/telemetryfrom
spencerdcarlson:scd/telemetry
Open

WIP: add telemetry#512
spencerdcarlson wants to merge 3 commits into
kafka4beam:dev/telemetryfrom
spencerdcarlson:scd/telemetry

Conversation

@spencerdcarlson

Copy link
Copy Markdown
Contributor

No description provided.

Comment thread rebar.config
, {telemetry, "~> 1.0"}
]},
{erl_opts, [warnings_as_errors, {d, build_brod_cli}]}
{erl_opts, [warnings_as_errors, {d, build_brod_cli}, {d, brod_use_telemetry}]}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm convinced this is not necessary.
Please remove it.

@zmstone

zmstone commented Jul 8, 2022

Copy link
Copy Markdown
Contributor

Thanks for the PR.

BTW. I originally wanted a solution without {telemetry, "~> 1.0"} in the dependency (and the brod_use_telemetry to be undefined by default).
So to push the responsibility to the wrapping project.

@zmstone
zmstone changed the base branch from master to dev/telemetry July 8, 2022 19:44
@zmstone

zmstone commented Jul 8, 2022

Copy link
Copy Markdown
Contributor

I have changed the base to dev/telemetry
let's start instrumenting the code, preferably increment with small PRs.

@maxno-kivra

Copy link
Copy Markdown

This would be very nice to have. What is needed to get this in?

@akosasante

Copy link
Copy Markdown

👋  Also looking to help push this forward, @zmstone , how can I help?

@mikpe

mikpe commented Nov 25, 2023

Copy link
Copy Markdown
Contributor

Use dependency injection.
Have measurement points invoke a local, "virtual", telemetry module.
That module checks if a telemetry callback module has been registered in the environment: if it has, pass the telemetry on to it, if not, do nothing.
The end result is that this project has no hard-coded dependencies in any telemetry framework, but the user can hook one up via the callback module they control.

zachallaun added a commit to zachallaun/brod that referenced this pull request Aug 14, 2026
Add a dependency on the telemetry library and a brod_metrics module
confining all telemetry calls, based on the pattern established in
[kafka4beam/wolff](https://github.com/kafka4beam/wolff/blob/4feffa6534a960377507fb2dc840db07548b2533/src/wolff_metrics.erl).

The first event, `[brod, product_request_sent]`, is emitted by
brod_producer for every produce request successfully sent on the wire.
This makes the actual wire-level batching observable, which is
otherwise invisible to callers

More events can be added to brod_metrics incrementally.

Related: kafka4beam#503, kafka4beam#512
zachallaun added a commit to zachallaun/brod that referenced this pull request Aug 14, 2026
Add a dependency on the telemetry library and a brod_metrics module
confining all telemetry calls, based on the pattern established in
[kafka4beam/wolff](https://github.com/kafka4beam/wolff/blob/4feffa6534a960377507fb2dc840db07548b2533/src/wolff_metrics.erl).

The first event, `[brod, product_request_sent]`, is emitted by
brod_producer for every produce request successfully sent on the wire.
This makes the actual wire-level batching observable, which is
otherwise invisible to callers

More events can be added to brod_metrics incrementally.

Related: kafka4beam#503, kafka4beam#512
zachallaun added a commit to zachallaun/brod that referenced this pull request Aug 18, 2026
Add a dependency on the telemetry library and a brod_metrics module
confining all telemetry calls, based on the pattern established in
[kafka4beam/wolff](https://github.com/kafka4beam/wolff/blob/4feffa6534a960377507fb2dc840db07548b2533/src/wolff_metrics.erl).

The first event, `[brod, product_request_sent]`, is emitted by
brod_producer for every produce request successfully sent on the wire.
This makes the actual wire-level batching observable, which is
otherwise invisible to callers

More events can be added to brod_metrics incrementally.

Related: kafka4beam#503, kafka4beam#512
zachallaun added a commit to zachallaun/brod that referenced this pull request Aug 18, 2026
Add a dependency on the telemetry library and a brod_metrics module
confining all telemetry calls, based on the pattern established in
[kafka4beam/wolff](https://github.com/kafka4beam/wolff/blob/4feffa6534a960377507fb2dc840db07548b2533/src/wolff_metrics.erl).

The first event, `[brod, product_request_sent]`, is emitted by
brod_producer for every produce request successfully sent on the wire.
This makes the actual wire-level batching observable, which is
otherwise invisible to callers

More events can be added to brod_metrics incrementally.

Related: kafka4beam#503, kafka4beam#512
zmstone pushed a commit that referenced this pull request Aug 18, 2026
* Depend on telemetry and emit produce request event

Add a dependency on the telemetry library and a brod_metrics module
confining all telemetry calls, based on the pattern established in
[kafka4beam/wolff](https://github.com/kafka4beam/wolff/blob/4feffa6534a960377507fb2dc840db07548b2533/src/wolff_metrics.erl).

The first event, `[brod, product_request_sent]`, is emitted by
brod_producer for every produce request successfully sent on the wire.
This makes the actual wire-level batching observable, which is
otherwise invisible to callers

More events can be added to brod_metrics incrementally.

Related: #503, #512

* Address code review feedback
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants