Description
Describe the bug
out_rdkafka2.rb is using the username and password to set values for rdkafka, sasl.username and sasl.password as we can see set rdkafka's username and password but there is no config_param to pass the same from fluent configuration. Which also makes it difficult to substitute these params with the environment variables.
A workaround for this is something like rdkafka_options "#{'{ "security.protocol": "sasl_ssl", "sasl.mechanisms": "PLAIN", "sasl.username": "$ConnectionString", "sasl.password": "%s" }' % [ENV['FLUENTD_SECRET']]}"
A proper solution would be to include username and password as config_param which allows to easily pass these values separately.
To Reproduce
<match **>
@type rdkafka2
#general settings
brokers "#{ENV['EVENT_HUB_BROKERS']}"
default_topic "#{ENV['EVENT_HUB_TOPIC']}"
# timeout settings check again for eventhub
required_acks -1
ack_timeout 30
# sasl settings
username "#{ENV['username']}" -- value not being used
password "#{ENV['pass']}" -- value not bein used
<format>
@type 'json'
</format>
..........
Expected behavior
username and password values specified should be used, so that no need to do
rdkafka_options "#{'{ "security.protocol": "sasl_ssl", "sasl.mechanisms": "PLAIN", "sasl.username": "$ConnectionString", "sasl.password": "%s" }' % [ENV['FLUENTD_SECRET']]}"
Your Environment
- Fluentd version: 1.17.1
- fluent-plugin-kafka version: 0.19.3
- Operating system: Linux
Your Configuration
<match **>
@type rdkafka2
#general settings
brokers "#{ENV['EVENT_HUB_BROKERS']}"
default_topic "#{ENV['EVENT_HUB_TOPIC']}"
# timeout settings check again for eventhub
required_acks -1
ack_timeout 30
# sasl settings
username "#{ENV['username']}" -- value not being used
password "#{ENV['pass']}" -- value not bein used
<format>
@type 'json'
</format>
..........
Your Error Log
Config params not getting used
Additional context
No response
Metadata
Metadata
Assignees
Labels
Type
Projects
Status