Need help about the fluentd memory leak issue

[sunnystone85]

I am a new about this.

Upgraded the fluentd from td-agent v4 to fluent-package v5, after upgrade to the new version around 2 weeks, the memory reached > 90%.
We need to restart the services.

There are two ruby plugin, in the match configuartion.

I have no clue about the issue, how I can deal with this issue.

[daipom]

@sunnystone85 Sorry for the trouble.
Could you share the config?

[sunnystone85]

So you mean, it should be the plugin script cause the problem right?
If it is caused by the script
command "ruby ./scripts/xxxxx.rb"

If so how we can fix the issue? the new ruby version not compatible with the old code?
Could you provide some suggestion?

[daipom]

So you mean, it should be the plugin script cause the problem right?

I'm not sure, but it would be good to narrow down the possibilities.
It could also be the problem of out_exec_filter plugin itself.
Or, it could also be the problem of another plugin, such as in_tail, out_file, or buffer.

If we can reproduce the problem simply, we can investigate the cause.

If so how we can fix the issue? the new ruby version not compatible with the old code?

If we conclude that the script is the cause, maybe someone, including me, can figure out something if you could share the script to the extent possible.

[sunnystone85]

Thank you, will try to reproduce the issue, then let you know.
BTW, there is a while section in the plugin script, the script will keep running and causing memory leak?

while str = STDIN.gets
  xxxxxxxxxxxxxxxxxx
  $log.info(r)
  next if r.empty?
  print MessagePack.pack(r)
  STDOUT.flush
end

[daipom]

The script looks fine.
It should keep running with Fluentd.
If there is more than one script running, something is wrong.

[daipom]

Are the logs outputted correctly?
If not, some data may have accumulated without being outputted.