FluentD needs internet connection to start

[imhongw]

Hello Mr. Miyai,

I am using FluentD to buffer some files in case of a connection loss, and will flush the buffer once the connection is re-established.

However, I noticed that this plugin will require an internet connection to do an initial start. If no internet connection is available, it will continue to try establishing a TCP connection with pubsub. I work in an environment where the internet is very unstable and is down most hours of the day. When the internet is down for some time, I will do a reboot but after that, fluentd is not able to start.

Is there any way to continue buffering the files without establishing a connection with pubsub?

Thanks!

[mia-0032]

Hi,
I'm sorry for my late reply. I was busy recently.

That is difficult problem because I think authentication should be done at starting FluentD.
I want to prevent FluentD from starting if the key is incorrect.

It will be an ad hoc change, but if you move https://github.com/mia-0032/fluent-plugin-gcloud-pubsub-custom/blob/master/lib/fluent/plugin/gcloud_pubsub/client.rb#L28 to https://github.com/mia-0032/fluent-plugin-gcloud-pubsub-custom/blob/master/lib/fluent/plugin/gcloud_pubsub/client.rb#L49, I think that you can make it authenticate at publishing messages time.

The change make authenticate every time flushing buffer, so I think a better way is needed.