feat(mqtt source, mqtt sink): add MQTT v5 protocol support by vitalvas · Pull Request #25587 · vectordotdev/vector

vitalvas · 2026-06-05T19:42:58Z

Summary

Add MQTT v5 protocol support to the mqtt source and mqtt sink, selectable via a new protocol_version option (defaults to v311 for backward compatibility).
Expose v5 publish properties (payload format indicator, message expiry, topic alias, response topic, correlation data, content type, user properties) on the sink under a new publish_properties config section, a
nd surface incoming v5 properties as source event metadata.
Add end-to-end acknowledgements to the mqtt source: when enabled, the source switches to MQTT manual ack mode and only sends PubAck to the broker once Vector has delivered the event downstream, providing at-l
east-once semantics for both v3.1.1 and v5.
Derive the sink's accepted input types from the configured encoder so logs, metrics, and traces are all transportable over MQTT (default json encodes all three; use native_json for lossless Vector-to-Vector p
ipelines). Mis-routed event types are now rejected at config build time instead of silently dropped by the codec.
Emit open_connections and connection_shutdown_total telemetry from both source and sink, plus richer component_errors_total coverage for connection, subscribe, and acknowledgement failures.
Consolidate the v3 and v5 connection-building code into a shared common::mqtt::build_connector to keep the source and sink in sync.

Vector configuration

sources:
  my_mqtt_source:
    type: mqtt
    host: "mqtt.example.com"
    port: 1883
    topic: "events/#"
    protocol_version: v5
    acknowledgements: true
    connect_properties:
      session_expiry_interval: 300

sinks:
  my_mqtt_sink:
    type: mqtt
    inputs: ["my_mqtt_source"]
    host: "mqtt.example.com"
    port: 1883
    topic: "events/{{ .namespace }}"
    protocol_version: v5                                                                                                                                                                                                  publish_properties:
      content_type: "application/json"
      message_expiry_interval: 3600
      user_properties:
        source: vector
    encoding:
      codec: json

How did you test this PR?

New unit tests for connector building (v3/v5, manual_acks propagation, credential and client_id validation), publish-property conversion (all fields, boundary values), QoS conversions, source metadata application
(v3 and v5, all 7 v5 fields, no-op on non-log events), and direction tagging on connection errors. 25 mqtt unit tests pass.
Existing v3.1.1 integration tests continue to pass.
Manual verification that the default protocol_version remains v311 and existing configurations behave unchanged.

Change Type

Is this a breaking change?

Yes
No

Does this PR include user facing changes?

Yes. Please add a changelog fragment based on our guidelines.
No. A maintainer will apply the no-changelog label to this PR.

References

Notes

Please read our Vector contributor resources.
Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
Some CI checks run only after we manually approve them.
- We recommend adding a pre-push hook, please see this template.
- Alternatively, we recommend running the following locally before pushing to the remote branch:
  - make fmt
  - make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
  - make test
After a review is requested, please avoid force pushes to help us review incrementally.
- Feel free to push as many commits as you want. They will be squashed into one before merging.
- For example, you can run git merge origin master and git push.
If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
run make build-licenses to regenerate the license inventory and commit the changes (if any). More details on the dd-rust-license-tool.

github-actions · 2026-06-05T19:43:11Z

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9cd665493e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…expiry for acknowledgements

…ocol # Conflicts: # src/sinks/mqtt/sink.rs

vitalvas · 2026-06-08T17:23:29Z

I have read the CLA Document and I hereby sign the CLA

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3fcb871c61

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

vitalvas · 2026-06-08T17:43:29Z

@codex review

chatgpt-codex-connector · 2026-06-08T17:47:34Z

Codex Review: Didn't find any major issues. 👍

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

rtrieu

some minor suggestions to comply with our style guide

rtrieu · 2026-06-09T17:39:19Z

 		type: string: {}
 	}
+	connect_properties: {
+		description: "MQTT v5 connection properties. Only used when protocol_version is v5."


Suggested change

description: "MQTT v5 connection properties. Only used when protocol_version is v5."

description: "MQTT v5 connection properties. Only applicable when `protocol_version` is `v5`."

rtrieu · 2026-06-09T17:39:51Z

+				All subscriptions are made at MQTT QoS 1 (AtLeastOnce), and MQTT wildcards
+				are supported:
+
+				- `+` matches exactly one topic level (for example `sensors/+/temperature`)


Suggested change

- `+` matches exactly one topic level (for example `sensors/+/temperature`)

- `+` matches exactly one topic level (for example ,`sensors/+/temperature`)

rtrieu · 2026-06-09T17:39:57Z

+				are supported:
+
+				- `+` matches exactly one topic level (for example `sensors/+/temperature`)
+				- `#` matches zero or more trailing levels (for example `sensors/#`)


Suggested change

- `#` matches zero or more trailing levels (for example `sensors/#`)

- `#` matches zero or more trailing levels (for example ,`sensors/#`)

rtrieu · 2026-06-09T17:40:36Z

+					}
+				}
+				payload_format_indicator: {
+					description: "MQTT v5 payload format indicator. 0 indicates unspecified bytes and 1 indicates UTF-8 encoded data. Absent for v3.1.1 messages or v5 messages without the property set."


Suggested change

description: "MQTT v5 payload format indicator. 0 indicates unspecified bytes and 1 indicates UTF-8 encoded data. Absent for v3.1.1 messages or v5 messages without the property set."

description: "MQTT v5 payload format indicator. 0 indicates unspecified bytes, and 1 indicates UTF-8 encoded data. Absent for v3.1.1 messages or v5 messages without the property set."

rtrieu · 2026-06-09T17:41:30Z

+		protocol_versions: {
+			title: "MQTT 3.1.1 and 5.0 support"
+			body: """
+				The sink supports both MQTT 3.1.1 (default) and MQTT 5.0, selected via the


Suggested change

The sink supports both MQTT 3.1.1 (default) and MQTT 5.0, selected via the

The sink supports both MQTT 3.1.1 (default) and MQTT 5.0, selected using the

rtrieu · 2026-06-09T17:41:43Z

+			title: "MQTT v5 publish properties"
+			body: """
+				When `protocol_version` is set to `v5`, the sink can attach MQTT 5.0 publish
+				properties to outgoing messages via the `publish_properties` section. Supported


Suggested change

properties to outgoing messages via the `publish_properties` section. Supported

properties to outgoing messages through the `publish_properties` section. Supported

vitalvas added 6 commits May 30, 2026 00:38

feat(mqtt source, mqtt sink): add MQTT v5 protocol support

bcead35

chore(changelog): add author to mqtt v5 fragment

33936e9

feat(mqtt source): add end-to-end acknowledgements

a5e3c72

feat(mqtt source, mqtt sink): expand transport, telemetry, and tests

79fb8b1

chore(mqtt source, mqtt sink): regenerate component config docs

e4cbd4b

chore(mqtt source, mqtt sink): restore upstream issue references

9cd6654

vitalvas requested review from a team as code owners June 5, 2026 19:42

chatgpt-codex-connector Bot reviewed Jun 5, 2026

View reviewed changes

Comment thread src/common/mqtt.rs

Comment thread src/common/mqtt.rs

Comment thread website/cue/reference/components/sinks/mqtt.cue Outdated

vitalvas added 2 commits June 8, 2026 13:17

fix(mqtt source, mqtt sink): enforce stable client_id and v5 session …

317d914

…expiry for acknowledgements

Merge remote-tracking branch 'upstream/master' into feat/mqtt-v5-prot…

3fcb871

…ocol # Conflicts: # src/sinks/mqtt/sink.rs

vitalvas changed the title ~~feat(mqtt): add support mqtt-v5 for source and sink~~ feat(mqtt source, mqtt sink): add MQTT v5 protocol support Jun 8, 2026

chatgpt-codex-connector Bot reviewed Jun 8, 2026

View reviewed changes

Comment thread website/cue/reference/components/sinks/mqtt.cue Outdated

fix(mqtt sink): mark healthcheck disabled and fix spelling check

bacaa92

github-actions Bot added the domain: ci Anything related to Vector's CI environment label Jun 8, 2026

rtrieu approved these changes Jun 9, 2026

View reviewed changes

	description: "MQTT v5 connection properties. Only used when protocol_version is v5."
	description: "MQTT v5 connection properties. Only applicable when `protocol_version` is `v5`."

	- `+` matches exactly one topic level (for example `sensors/+/temperature`)
	- `+` matches exactly one topic level (for example ,`sensors/+/temperature`)

	- `#` matches zero or more trailing levels (for example `sensors/#`)
	- `#` matches zero or more trailing levels (for example ,`sensors/#`)

	description: "MQTT v5 payload format indicator. 0 indicates unspecified bytes and 1 indicates UTF-8 encoded data. Absent for v3.1.1 messages or v5 messages without the property set."
	description: "MQTT v5 payload format indicator. 0 indicates unspecified bytes, and 1 indicates UTF-8 encoded data. Absent for v3.1.1 messages or v5 messages without the property set."

	The sink supports both MQTT 3.1.1 (default) and MQTT 5.0, selected via the
	The sink supports both MQTT 3.1.1 (default) and MQTT 5.0, selected using the

	properties to outgoing messages via the `publish_properties` section. Supported
	properties to outgoing messages through the `publish_properties` section. Supported

Conversation

vitalvas commented Jun 5, 2026

Summary

Vector configuration

How did you test this PR?

Change Type

Is this a breaking change?

Does this PR include user facing changes?

References

Notes

Uh oh!

github-actions Bot commented Jun 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vitalvas commented Jun 8, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

vitalvas commented Jun 8, 2026

Uh oh!

chatgpt-codex-connector Bot commented Jun 8, 2026

Uh oh!

rtrieu left a comment

Choose a reason for hiding this comment

Uh oh!

rtrieu Jun 9, 2026

Choose a reason for hiding this comment

Uh oh!

rtrieu Jun 9, 2026

Choose a reason for hiding this comment

Uh oh!

rtrieu Jun 9, 2026

Choose a reason for hiding this comment

Uh oh!

rtrieu Jun 9, 2026

Choose a reason for hiding this comment

Uh oh!

rtrieu Jun 9, 2026

Choose a reason for hiding this comment

Uh oh!

rtrieu Jun 9, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Jun 5, 2026 •

edited

Loading