Skip to content

[SeriesDFilter] Redundant execution of filter_pipe for each fields_group causes slow data loading #2236

Description

@CanvaChen

Description

During data loading, I noticed that filter_pipe is executed repeatedly for each fields_group instead of being applied once or cached. This causes significant slowdowns in data preparation, especially when multiple field groups are configured.

Steps to Reproduce / Observation

  1. Added logging/print statements inside the _getFilterSeries method.
  2. Ran the data loading pipeline with multiple fields_groups (e.g., 3 groups in my setup).
  3. Observed that the filter pipeline was called exactly 3 times, once per fields_group, rather than sharing the filtered result across groups.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions