Replacing the StandardScaler with the SquashingScaler for the tabular_pipeline

[rcap107]

At the moment, the tabular_pipeline adds a StandardScaler to the pipeline for linear models. Should we use the SquashingScaler instead?

Paging @dholzmueller as I don't know if this can lead to issues

[dholzmueller]

I guess it depends on why you would use a linear model in the first place:

• because you want to fit a linear function: then you would get surprised that the fitted function is not linear
• because you want to interpret the coefficients: then you might also get surprised, and the interpretation may be better or worse. If you have a feature that is mostly zero, the StandardScaler will rescale the remaining non-zero values to large values, and the linear coefficient will be smaller than it would have been with the SquashingScaler.
• because you just want a fast method that doesn't overfit: sure, using the StandardScaler could be better (could be checked using the code from the SquashingScaler example but with linear models instead of neural nets).

I guess if you're out for interpretation you might not be using tabular_learner anyway, but I'm not sure.

[rcap107]

My reasoning was that the tabular_pipeline (aka tabular_learner) is supposed to be a "robust and strong baseline", so the intention was replacing the StandardScaler for something better, and the SquashingScaler seemed like a good option.

Honestly I have no idea what's the use case for the tabular_pipeline with linear models, so I don't know if this would break workflows or not.

It might be better to keep it "simpler" and just use the StandardScaler after all.

[rcap107]

Discussed in the skrub meeting:

Normally people use linear models for interpretability, which is broken when there are a lot of outliers. We might want to use the SquashingScaler to help with this, but we should increase the value of max_absolute_value to something larger than what we have. We also need to mention the fact we use the SquashingScaler very clearly in the documentation.

So what I'm taking away is a "tentative yes low priority" from @GaelVaroquaux

[dierickxsimon]

I will work on this!