-
Notifications
You must be signed in to change notification settings - Fork 22
Description
(check apply)
- read the contribution guideline
- (optional) already reported 3rd party upstream repository or mailing list if you use k8s addon or helm charts.
Steps to replicate
Just use the latest plugin 1.1.5 with opensearch 3.2.0
Expected Behavior or What you need to ask
Hello,
I'm using the plugin since 1 year and since a few weeks it seems that it is displaying a lot of opensearch informations insides our fluentd logs.
Here is a part of my dockerfile, creating a new image from a fluentd basis for my deployment
FROM ${REGISTRY}fluent/fluentd:v1.19.0-1.0
....
gem install --verbose --version 1.1.5 fluent-plugin-opensearch --no-document
And here is the part of the configuration dealing with opensearch :
@type opensearch
hosts xxxxxxx.svc.cluster.local
port 9200
user "xxxxxx"
password "xxxxxxxx"
scheme https
ssl_verify false
reload_on_failure true
reload_connections true
reload_after 100
# Don't check opensearch version at startup
# It means fluentd can start even if opensearch is down
verify_os_version_at_startup false
logstash_format true
logstash_dateformat %Y.%m
template_name toto-prod
template_file /index_templates/generated/toto_default_template.json
template_overwrite true
utc_index true
include_timestamp true
include_tag_key true
use_legacy_template false
suppress_type_name true
request_timeout 2147483648
bulk_message_request_threshold 8m
<buffer>
@type file
total_limit_size 128MB
flush_interval 60s
</buffer>
Finally when I run the container, a lot of log lines like this are constantly displayed :
{id: "zMy-zil4adDReLs35z", name: "toto-open-search-cluster-data-2", version: "3.2.0", host: "192.168.1.34", port: "9200", roles: ["data", "ingest"], attributes: {"shard_indexing_pressure_enabled" => "true"}}
If I change the fluentd loglevel, there is no impact, this messages are always spawning frequently on container logs.
It seems to be related to a 'print' on stdout related to the result of an opensearch request to '/_nodes'.
It is displayed very frequently (maybe each time I get 100 msg in fluentd ?)
Do you have any idea about this new behaviour and can I disable it ?
Somewhere else in the plugin, another 'opensearch api call' is done, printing the password we use for connection ! (like the opensearch client object used by fluentd config)
Which (dependency?) source code is raising this informations among the entire logs produced by fluentd ?
I expect to see nothing about my opensearch cluster informations in fluentd logs
Using Fluentd and OpenSearch plugin versions
-
OS version
openshift 4 -
Bare Metal or within Docker or Kubernetes or others?
docker -
Fluentd v1.0 or later
- paste result of
fluentd --versionortd-agent --version
fluentd:v1.19.0-1.0 in container
- paste result of
-
OpenSearch plugin version
- paste boot log of fluentd or td-agent
Generate index patterns
Generating and pushing index deletion policies
Launch Fluentd
2025-09-23 09:55:38 +0000 [info]: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2025-09-23 09:55:38 +0000 [info]: parsing config file is succeeded path="/fluentd/etc/fluent.conf"
2025-09-23 09:55:38 +0000 [info]: inaccessible include directory was specified path="/etc/fluent/conf.d"
2025-09-23 09:55:38 +0000 [info]: gem 'fluent-plugin-http-healthcheck' version '0.1.0'
2025-09-23 09:55:38 +0000 [info]: gem 'fluent-plugin-kafka' version '0.19.5'
2025-09-23 09:55:38 +0000 [info]: gem 'fluent-plugin-opensearch' version '1.1.5'
2025-09-23 09:55:38 +0000 [info]: gem 'fluent-plugin-prometheus' version '2.2.1'
2025-09-23 09:55:38 +0000 [info]: gem 'fluent-plugin-remote_syslog' version '1.1.0'
2025-09-23 09:55:38 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.4.0'
2025-09-23 09:55:38 +0000 [info]: gem 'fluentd' version '1.19.0'
- paste result of
fluent-gem list,td-agent-gem listor your Gemfile.lock - OpenSearch version (optional)
3.2.0 container - OpenSearch template(s) (optional)