Compatibility with neural network: replacing with constant value instead of dropping the feature

[stephanecollot]

Hi,

For neural network if you change the number of features, you need to change the input dimension and therefore the number of neurons.
So, we could have an option like:

• leaving='drop' for current behaviour
• leaving='replace' for NN

What do you think?

[aerdem4]

Makes sense. Would this be a flag to set?

I have only one concern. Most BN implementations etc make safe normalization but someone may have a custom model which assumes all features to have non-zero std. So replacing with a small random noise could also be an option.

[stephanecollot]

yes, could be different "leaving strategies", I think a string field would give more flexibility. Good point for the batch normalization and the risk of dividing by zero.

Side remark: with NN you will most probably use FLOFO instead of LOFO anyway, at least that fit my current use case, because I have too many features and training takes too long.