-
Notifications
You must be signed in to change notification settings - Fork 127
Description
Apologies if this exists and I missed the appropriate configuration, but after enabling the otlp endpoint on the DataDog Agent I noticed that the service generated for the agent does not define the appProtocol field for the port.
Some applications, in my case kgateway, seem to use this field to figure out the protocol to use to communicate with the grpc server.
Without it I get an error such as:
[external/envoy/source/extensions/tracers/opentelemetry/grpc_trace_exporter.cc:39] OTLP trace export failed with status: Unavailable, message: upstream connect error or disconnect/reset before headers. reset reason: protocol error{upstream_reset_before_response_started{protocol_error}}
Because the backend cluster configuration generated did not specify http2 as the protocol.
Current workaround
I ended up creating a new service that mimicked the service created by this operator and added this field myself and everything worked as expected and traces from my otel implemented service made it into DataDog.
I did initially try to edit the service created for the agent, but (I think) the operator simply undid those changes.
Relevant code
I briefly looked to see if this was something that was already supported.
The relevant code seems to be around here and I didn't see anywhere that the appProtocol would be added.