Would it be reasonable to split up *Get Filenames* into separate input and transform steps?

[alexmller]

I would like to share my thoughts for your opinions on this.

It seems easy to misunderstand the behavior of Get Filenames (#4843):

• if "Is filename defined in a field?" is checked, it processes data from the stream and transforms it
• if not, it acts as an input, discarding incoming fields and generating its own rows

The same is for Execute SQL script and it's "Execute for each row" toggle. Fixing it by updating the docs feels right to me (#5734).

For Get Filenames however, the user dialog could be simplified and source code readability improved by separating both concerns into separate Steps. Of course introducing a breaking change.

Is there a preferred approach when it comes to breaking changes (avoid them at any cost or deal with them in a certain way)?

[hansva]

Hi Alex,

In situations like these, we usually deprecate the old transform and create the new transform(s) next to it.
We have a separate category that will visually mark the transform as deprecated (example is Text file input (deprecated)).

Though I am not sure we should split these concerns. I agree that we need to improve the dialogs and make them more consistent.
But almost all our input transforms have the same options in the same transform.

• Text file input
• Json input
• properties input
• ...

[bamaer]

At the code level, we try to avoid breaking changes within the same major release at all cost. Within actions/transforms, we should even be more careful. I agree with @hansva: deprecating the old transforms and creating new ones is an option, but there probably already is a lot we can do by making the file selector dialogs more consistent and more user friendly.

[alexmller]

That makes sense. I'll see what I can do and create a draft for that.