Skip to content

partial write of line protocol occurred with no detail on which line is rejected #27497

Description

@Akdeniz7

Summary:

We are running a monitoring pipeline where customer-side Telegraf agents send line protocol data to a central Telegraf bastion server, which forwards to InfluxDB 2. This setup has been working without any issues.
We recently set up a parallel InfluxDB 3 Core pipeline: customer-side Telegraf agents now send data to both the existing bastion (for InfluxDB 2) and a new OCI load balancer (weighted round robin) in front of two new Telegraf bastion servers, which forward to InfluxDB 3 Core via the influxdb_v3 output plugin.
The client-side Telegraf agents show no errors. Data is visible and queryable in Grafana on both pipelines. However, the new InfluxDB 3 Core bastions consistently produce partial write of line protocol occurred errors every minute.
The core issue is not the error itself but the lack of detail in the response — InfluxDB 2 logged the offending line directly, making it straightforward to diagnose. InfluxDB 3 Core returns only a generic message with no indication of which line, field, or measurement caused the rejection.

Steps to reproduce:

  1. Set up Telegraf 1.38.4 with influxdb_v2_listener input and influxdb_v3 output
  2. Send line protocol data from customer-side Telegraf agents via a load balancer to the bastion Telegraf instances
  3. Bastion Telegraf forwards data to InfluxDB 3 Core using influxdb_v3 output plugin
  4. Observe 400 Bad Request errors in Telegraf logs every minute

Expected behaviour:

Write succeeds, or if rejected, the error response contains the specific line(s) that failed parsing so the client can identify and fix the issue.
Actual behaviour:
InfluxDB 3 Core returns 400 Bad Request with only partial write of line protocol occurred — no line number, no field name, no further detail. Some lines in the batch are written successfully, others are silently dropped. The Telegraf log shows:

[outputs.influxdb_v3] Writing to "http://<IP_ADDRESS>:8181" failed: failed to write metric to <database_name>(will be dropped: 400 Bad Request): partial write of line protocol occurred

The InfluxDB 3 Core server log shows:
ERROR influxdb3_server::http: Error while handling request error=partial write of line protocol occurred method=POST path=/api/v3/write_lp content_length=None database=<database_name> client_ip=<TELEGRAF_PROXY_SERVER_IP>

No additional detail about which line caused the rejection.

Environment info:

  • InfluxDB 3 Core 3.9.3, revision e8cd4f7
  • OS: Linux 6.12.0-202.76.4.4.el9uek.x86_64 x86_64 (Oracle Linux 9)
  • Telegraf 1.38.4 (git: HEAD@c79b06d5)
  • Data flow: Customer Telegraf → OCI Load Balancer → 2x Telegraf bastion (influxdb_v2_listener) → InfluxDB 3 Core (influxdb_v3 output)

Config:

telegraf.conf
`[agent]
interval = "1m"
metric_batch_size = 300000
metric_buffer_limit = 1500000
flush_interval = "1m"
flush_jitter = "0s"

[[inputs.influxdb_v2_listener]]
service_address = ":8086"`

Telegraf bastion output:

[[outputs.influxdb_v3]] urls = ["http://<IP_ADDRESS>:8181"] token = "<CUSTOMER_TOKEN>" database_tag = "customer" [outputs.influxdb_v3.tagpass] customer = ["customer1", "demo", "customer2","telegraf","customer3"]

Strings processor applied to strip newlines from all fields and tags:
`[[processors.strings]]
[[processors.strings.trim]]
tag = "database_name"

[[processors.strings]]
[[processors.strings.replace]]
field = "*"
old = "\n"
new = " "

[[processors.strings]]
[[processors.strings.replace]]
field = "*"
old = "\r"
new = " "

[[processors.strings]]
[[processors.strings.replace]]
tag = "*"
old = "\n"
new = " "

[[processors.strings]]
[[processors.strings.replace]]
tag = "*"
old = "\r"
new = " "`

Logs:

InfluxDB 3 Core server log:
ERROR influxdb3_server::http: Error while handling request error=partial write of line protocol occurred method=POST path=/api/v3/write_lp content_length=None <database_name> client_ip=<TELEGRAF_PROXY_SERVER_IP>

Additional context:

Data is successfully written and visible in Grafana. The error occurs consistently every minute. There is no way to identify which line is being rejected (In influxdb2, erroneous lines were directly visible in the logs.) from either the Telegraf or InfluxDB 3 Core logs. A more descriptive error response (e.g. line number, field name, or the offending line protocol string) would allow users to diagnose and fix the root cause.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions