BUG - ColumnTransformer fails with `min-optional-reqs` when using a Polars dataframe

[rcap107]

Follow up of #1540 and #1543

min-optional-reqs is a configuration that includes both early scikit-learn versions and early polars versions, and in scikit-learn 1.2 there was no proper support for polars. The result is that using polars dataframes with the ColumnTransformer leads to incompatibilities.

Fixing this would involve patching the ColumnTransformer on skrub's side, which is complicated and can lead to other issues. Also, given that the fix would involve converting the dataframe to pandas and then back, it's just easier for the users to do that on their side if they encounter the issue.

This should not be fixed on skrub's side.

[GaelVaroquaux]

If we updated our min-optional-reqs to avoid the problem, would we need a big bump? rcap107 created an issue (skrub-data/skrub#1557) Follow up of #1540 and #1543 min-optional-reqs is a configuration that includes both early scikit-learn versions and early polars versions, and in scikit-learn 1.2 there was no proper support for polars. The result is that using polars dataframes with the ColumnTransformer leads to incompatibilities. Fixing this would involve patching the ColumnTransformer on skrub's side, which is complicated and can lead to other issues. Also, given that the fix would involve converting the dataframe to pandas and then back, it's just easier for the users to do that on their side if they encounter the issue. This should not be fixed on skrub's side. — Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you are subscribed to this thread. Message ID: <skrub-data/skrub/issues/1557 @ github . com>

[rcap107]

If we updated our min-optional-reqs to avoid the problem, would we need a big bump?

At least up to 1.4

[glemaitre]

With 1.7 out, 1.4 is a 2 years support.

[rcap107]

With Python 3.9 hitting end of life in October, we should also change the minimum requirement to 3.10

[GaelVaroquaux]

With Python 3.9 hitting end of life in October, we should also change the minimum requirement to 3.10

Yes

[rcap107]

This will be closed by #1572 once it gets merged