OTLP encoding not working with OpenTelemetry sink

[ozanichkovsky]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

While trying out use_otlp_encoding configuration option for OpenTelemetry introduced in version 0.50.0 I stumbled into a problem with sending its output directly into OpenTelemetry sink.

I generated desc file by running the following command in /lib/opentelemetry-proto/src/proto/opentelemetry-proto using 32.1 version of protoc:

protoc -I . --include_imports --include_source_info \
  -o /path/to/otlp_logs.desc \
  opentelemetry/proto/collector/logs/v1/logs_service.proto

The output of vector tap opentelemetry.logs is using camel case for field names:

{"resourceLogs":[{"resource":{"attributes":[{"key":"environment","value":{"stringValue":"development"}},{"key":"service.name","value":{"stringValue":"log-generator-service"}},{"key":"service.version","value":{"stringValue":"1.0.0"}}]},"scopeLogs":[{"logRecords":[{"attributes":[{"key":"service.name","value":{"stringValue":"sdk-client"}},{"key":"id","value":{"stringValue":"a45ea3b7-c256-411c-a194-e02a3f3bcc55"}},{"key":"map","value":{"kvlistValue":{"values":[{"key":"user","value":{"kvlistValue":{"values":[{"key":"id","value":{"stringValue":"user-373"}}]}}}]}}},{"key":"slice","value":{"arrayValue":{"values":[{"stringValue":"test1"},{"stringValue":"test2"}]}}}],"body":{"kvlistValue":{"values":[{"key":"body","value":{"stringValue":"Payment processed successfully."}}]}},"observedTimeUnixNano":1759996703464707864,"severityNumber":"SEVERITY_NUMBER_INFO","severityText":"INFO"}],"scope":{"name":"main-logger"}}]}],"timestamp":"2025-10-09T07:58:39.467618239Z"}

And there is no output, because encoded body is empty:

2025-10-09T08:11:17.088619Z DEBUG sink{component_kind="sink" component_id=otel_output component_type=opentelemetry}:request{request_id=1}:http: vector::internal_events::http_client: Sending HTTP request. uri=http://127.0.0.1:4319/v1/logs method=POST version=HTTP/1.1 headers={"content-type": "application/x-protobuf", "accept-encoding": "zstd,gzip,deflate,br", "user-agent": "Vector/0.50.0 (aarch64-unknown-linux-musl 9053198 2025-09-23 14:18:50.944442940)"} body=[empty]    

On the other hand, if I forget about attributes and create a protobuf structure manually in a remap transform and use snake case, it works as expected:

transforms:
  prepare:
    type: remap
    inputs:
      - "opentelemetry.logs"
    source: |
      .resource_logs = [
          {
            "resource": {
            },
            "scope_logs": [{
              "scope": {
                "name": {
                  if exists(.scope.name)
                  {
                    .scope.name
                  } else {
                    null
                  }
                },
                "version": {
                  if exists(.scope.version)
                  {
                    .scope.version
                  } else {
                    null
                  }
                }
              },
              "log_records": [{
                "time_unix_nano": {
                  if is_timestamp(.timestamp) {
                    to_unix_timestamp(timestamp!(.timestamp), unit: "nanoseconds")
                  } else {
                    0
                  }
                },
                "observed_time_unix_nano": {
                  if is_timestamp(.observed_timestamp) {
                    to_unix_timestamp(timestamp!(.observed_timestamp), unit: "nanoseconds")
                  } else {
                    0
                  }
                },
                "severity_number": {
                  if exists(.severity_number)
                  {
                    .severity_number
                  } else {
                    null
                  }
                },
                "severity_text": {
                  if exists(.severity_text)
                  {
                    .severity_text
                  } else {
                    null
                  }
                },
                "body": {
                  "string_value": {
                      if is_string(.message) {
                        .message
                      } else {
                        encode_json(.message)
                      }
                  }
                }
              }]
            }]
          }
        ]

For me it looks like my desc file is probably the culprit. Is there a correct way of generating it? I tried buf as well and got the same result.

I also tried setting this value to false in

vector/src/sources/opentelemetry/config.rs

Line 176 in 68f0b4c

use_json_names: true,

and it then worked after recompiling Vector.

Configuration

api:
  enabled: true
  address: "127.0.0.1:8686"

sources:
  opentelemetry:
    type: "opentelemetry"
    grpc:
      address: 0.0.0.0:4317
    http:
      address: 0.0.0.0:4318
      headers: []
    use_otlp_decoding: true

sinks:
  otel_output:
    type: opentelemetry
    inputs:
      - "opentelemetry.logs"
    protocol:
      type: http
      uri: http://127.0.0.1:4319/v1/logs
      method: post
      encoding:
        codec: "protobuf"
        protobuf:
          desc_file: "/etc/descriptor/otlp_descriptor.desc"
          message_type: "opentelemetry.proto.collector.logs.v1.ExportLogsServiceRequest"
      request:
        headers:
          content-type: application/x-protobuf
      framing:
        method: bytes

Version

0.50.0

Debug Output

Example Data

No response

Additional Context

No response

References

No response

[ozanichkovsky]

Hello @pront. You asked me about starting this discussion in #23850. So here it is.

[pront]

Hi @ozanichkovsky, thank you for providing all these details. I suspect the issue asymmetrical decoding/encoding.

Option 1

Try this alternative sink configuration:
https://github.com/vectordotdev/vector/blob/fad8439c7051a8a3968b9184f257239e0bc173b7/tests/data/e2e/opentelemetry/logs/vector_otlp.yaml#L17C1-L34C41

This is worth a try because it should work with the released Vector version. If not, I would love to hear about it.

Option 2 (recommended)

We plan to release an otlp encoder, which is already available in all the latest nightlies:

• https://github.com/vectordotdev/vector/pkgs/container/vector

If I am not mistaken, you already tried it and it worked. The new sink config is here:

https://github.com/vectordotdev/vector/blob/master/tests/data/e2e/opentelemetry/logs/vector_otlp.yaml#L18-L26

The above is much simpler and doesn't require using tools like protoc.

Summary

If the alternative config from option 1 works, you can use that as an immediate workaround. We can also consider exposing the use_json_names as a proto encoding option.

Going forward, we recommend people to use otlp codecs because the configuration is minimal and less prone to errors.

P.S. We want to simplify even further by making this the default (might be done before the next release) and by introducing an OTLP encoder. The latter might take some time.

[ozanichkovsky]

Thanks @pront. Option1 won't probably work if another Vector instance is the destination since Vector rejects application/json OTLP requests.

[pront]

Thanks @pront. Option1 won't probably work if another Vector instance is the destination since Vector rejects application/json OTLP requests.

Good point, we would have to make that configurable for that option to be viable.

Is option 2 resolving all your issues? If not, please let me know and we can discuss further.

[ozanichkovsky]

Option 2 with nightly build works as expected. Thanks @pront.