Describe the question / issue
Hi,
I am using FireLens as the logDriver for ECS tasks running on Fargate. I need to configure Mem_Buf_Limit on the forward input in Fluent Bit in order to properly handle backpressure.
From my understanding, FireLens auto-generates the base fluent-bit.conf, and there are no supported options to customize the parameters of the generated inputs (such as Mem_Buf_Limit) directly from outside that configuration.
I discovered that setting aws_fluent_bit_init_ignore_firelens_config allows me to supply a fully custom Fluent Bit configuration without clashing with the FireLens-generated one. This approach works for configuring the forward input as needed.
However, I still need the ECS metadata–related environment variables (e.g. ECS_METADATA_*) that are normally generated by the FireLens init process, as they are required by downstream filters and outputs.
Could you please advise on the recommended way to:
- Configure
Mem_Buf_Limit on the forward input, and
- Preserve access to the ECS metadata environment variables normally provided by the FireLens init process?
Thanks in advance.
References:
https://aws.amazon.com/blogs/containers/how-to-set-fluentd-and-fluent-bit-input-parameters-in-firelens/
#798