[Q&A] How to handle gem version conflicts in fluentd?

[RamHaridas]

What is a problem?

Scenario: Let's say that I have installed plugins at two separate paths /path/to/default and /path/to/custom.

• I have installed pluginA at /path/to/default that is dependent on gemA version 1.5 (less than 2.0)
• Then, I installed pluginB at /path/to/custom that is dependent on gemA version 2.3 (greater than 2.0)
• Now, I update the GEM_PATH environment variable GEM_PATH=/path/to/custom:/path/to/default
• This encourages fluentd to load both pluginA and pluginB at runtime.

But the default behavior of fluentd to load gems, would default it to go with the max version of gemA which is 2.3 (as confirmed here #5158 by @Watson1978).
Now, this may work well for pluginB but not for pluginA

I wanted to understand that what steps should be taken to overcome this dependency conflict and how we can overcome it?

Describe the configuration of Fluentd

No response

Describe the logs of Fluentd

No response

Environment

- Fluentd version: 1.19.0
- Fluent Package version: 6.0.0
- Operating system: Rocky 8
- Kernel version: 5.14

[Watson1978]

Basically, due to Ruby's language specification, there is no built-in mechanism to isolate namespaces if they conflict within the same process. All loaded gems share the same runtime environment.
(Since Ruby 4.0, it has the Namespace feature, it might be able to avoid the conflict)

Therefore, it is the responsibility of the plugin (or library) developers to implement their code within proper, unique namespaces to avoid such collisions.