Skip to content

Allow control the order of Output Plugin #1115

Open
@sugaf1204

Description

@sugaf1204

Is your feature request related to a problem? Please describe.

In the Fluentd, I can't completely control the order of output plugin.

The order of plugins cannot be controlled between them that have different values for the tag field of output plugin.
For example, when using rewrite_tag_filter, the problem arises because we cannot control the order of customPlugin that matches the tag before rewritten and s3 plugin that matches it after rewritten.

apiVersion: fluentd.fluent.io/v1alpha1
kind: ClusterOutput
metadata:
  name: example
spec: 
  outputs: 
  - tag: "k8s.log.*"
     customPlugin:
       config: |
         <match k8s.log.*>
           @type rewrite_tag_filter
           <rule>
             key $.type
             pattern ^(.+)$
             tag $1.log
           </rule>
         </match>
  - tag: "A.log"
     s3:
     ......
  - tag: "B.log" 
     forward:
     ......
  - tag: "*.log" 
     elasticsearch:
     ......
  - tag: "k8s.**" # catch all
     stdout:
     ......

The order in which these are written out to config is undifined.

Describe the solution you'd like

In the current implementation, within a group of plugins with the same tag field value, the order of them can probably be controlled by metadata.name of CR.
(grouping by tag field value: https://github.com/fluent/fluent-operator/blob/v2.7.0/apis/fluentd/v1alpha1/helper.go#L296)
(sorted by metadata.name: https://github.com/fluent/fluent-operator/blob/v2.7.0/apis/fluentd/v1alpha1/helper.go#L124)

I can solve this problem by writing everything in single customPlugin, but it is not good because it raises the possibility of managing a very huge single config in the future.
I want to control the order of all plugins according to the order of the outputs array defined in one CR, not just the metadata.name.

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions