Description
Logstash information:
Please include the following information:
- Logstash version (e.g.
bin/logstash --version):logstash 8.17.0 - Logstash installation source (e.g. built from source, with a package manager: DEB/RPM, expanded from tar or zip archive, docker): k8s operator
Plugins installed: not sure if need in this case
JVM (e.g. java -version):
openjdk version "21.0.5" 2024-10-15 LTS
OpenJDK Runtime Environment Temurin-21.0.5+11 (build 21.0.5+11-LTS)
OpenJDK 64-Bit Server VM Temurin-21.0.5+11 (build 21.0.5+11-LTS, mixed mode, sharing)
OS version (uname -a if on a Unix-like system):
Linux logstash-stage-ls-1 5.15.0-1079-azure 88-Ubuntu SMP Thu Jan 16 19:18:54 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Description of the problem including expected versus actual behavior:
I have tool that generate .conf files according to our state in db. I'm removing conf file and adding new conf file in shared folder. Folder shared between two services - my tool and logstash.
When tool removing file and if in this moment logstash trying to read it, then it will throw exception Errno::ENOENT like this and all pipelines will be broken until restarting pod.
SIGHUP also doesn't helps, all pipelines broken.
Am I using it wrong or it is bug? How I can avoid it?
When I'm trying to do the same with docker-compose on my local, I'm not getting such exception and in logs I will see No config files found in path and it will work if I will add file again. Looks like OS depends here, my local Win 11 + docker-compose + docker.elastic.co/logstash/logstash:8.16.1 (for local 8.16.1 version vs 8.17 in k8s)
Steps to reproduce:
Please include a minimal but complete recreation of the problem,
including (e.g.) pipeline definition(s), settings, locale, etc. The easier
you make for us to reproduce it, the more likely that somebody will take the
time to look at it.
1.Run logstash service in k8s with shared folder for .conf files.
2. Remove conf file.
3. Wait for auto reload
Provide logs (if relevant):
[2025-04-02T13:40:00,941][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/usr/share/logstash/configs_auto/dev/logstash-stage-ls-1/caselist.conf"}
[2025-04-02T13:40:00,973][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/usr/share/logstash/configs_auto/preproduction/logstash-stage-ls-1/caselist.conf"}
[2025-04-02T13:40:03,938][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/usr/share/logstash/configs_auto/dev/logstash-stage-ls-1/caselist.conf"}
[2025-04-02T13:40:03,957][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/usr/share/logstash/configs_auto/preproduction/logstash-stage-ls-1/caselist.conf"}
[2025-04-02T13:40:06,939][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/usr/share/logstash/configs_auto/dev/logstash-stage-ls-1/caselist.conf"}
[2025-04-02T13:40:06,953][ERROR][logstash.config.sourceloader] Could not fetch all the sources {:exception=>Errno::ENOENT, :message=>"No such file or directory - /usr/share/logstash/configs_auto/preuat/logstash-stage-ls-1/caselist.conf", :backtrace=>["org/jruby/RubyIO.java:1278:in `sysopen'", "org/jruby/RubyIO.java:4264:in `read'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/local.rb:87:in `block in read'", "org/jruby/RubyArray.java:1981:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/local.rb:77:in `read'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/local.rb:110:in `read'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/local.rb:206:in `local_pipeline_configs'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/multi_local.rb:44:in `block in pipeline_configs'", "org/jruby/RubyArray.java:2803:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/multi_local.rb:39:in `pipeline_configs'", "/usr/share/logstash/logstash-core/lib/logstash/config/source_loader.rb:76:in `block in fetch'", "org/jruby/RubyArray.java:2798:in `collect'", "/usr/share/logstash/logstash-core/lib/logstash/config/source_loader.rb:75:in `fetch'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:211:in `converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:139:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/stud-0.0.23/lib/stud/interval.rb:18:in `interval'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:128:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:455:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}
[2025-04-02T13:40:09,932][ERROR][logstash.config.sourceloader] No source loaders matched! This shouldn't happen ...
[2025-04-02T13:40:09,946][ERROR][logstash.agent ] An exception happened when converging configuration