Skip to content

allow passing kwargs in .skb.apply()#1642

Merged
rcap107 merged 7 commits into
skrub-data:mainfrom
jeromedockes:dataop-apply-kwargs
Oct 9, 2025
Merged

allow passing kwargs in .skb.apply()#1642
rcap107 merged 7 commits into
skrub-data:mainfrom
jeromedockes:dataop-apply-kwargs

Conversation

@jeromedockes

Copy link
Copy Markdown
Member

This allows passing extra named arguments to the methods of estimators passed to .skb.apply(). Still hesitating slightly about whether this is needed.

Also, we should agree on the exact interface: a single dict like kwargs={"fit": {"sample_weight": sw}}, several like fit_kwargs={"sample_weight": sw} (what is currently in this PR), other variations...

@jeromedockes jeromedockes added the data_ops Something related to the skrub DataOps label Oct 2, 2025
@rcap107

rcap107 commented Oct 2, 2025

Copy link
Copy Markdown
Member

I feel like I don't particularly like any of the solutions we discussed with @jeromedockes, but at the same time not having this feature would be too limiting, since people use other libraries that need these kwargs (and scikit-learn has some estimators that need them too).

I think this version with fit_kwargs and predict_kwargs is the better solution.

@jeromedockes

Copy link
Copy Markdown
Member Author

example passing sample weights (computed from X) to a ridge regressor:

import polars as pl
from sklearn.linear_model import Ridge

import skrub
import skrub.datasets
import skrub.selectors as s

dataset = pl.from_pandas(skrub.datasets.fetch_employee_salaries().employee_salaries)
# skrub.TableReport(dataset).open()
data = skrub.var("data")
y_col = "current_annual_salary"
X = data.skb.select(s.all() - y_col).skb.mark_as_X()
y = data[y_col].skb.mark_as_y()


def compute_weights(X):
    return (-(((X["year_first_hired"] - 2025) / 20) ** 2)).exp().rename("weight")


weight = X.skb.apply_func(compute_weights)
pred = X.skb.apply(skrub.TableVectorizer()).skb.apply(
    Ridge(), y=y, fit_kwargs={"sample_weight": weight}
)

graph

@jeromedockes jeromedockes changed the title discussion/wip allow passing kwargs in .skb.apply() allow passing kwargs in .skb.apply() Oct 2, 2025
@jeromedockes jeromedockes marked this pull request as ready for review October 2, 2025 22:38

@rcap107 rcap107 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

example passing sample weights (computed from X) to a ridge regressor:

import polars as pl
from sklearn.linear_model import Ridge

import skrub
import skrub.datasets
import skrub.selectors as s

dataset = pl.from_pandas(skrub.datasets.fetch_employee_salaries().employee_salaries)
# skrub.TableReport(dataset).open()
data = skrub.var("data")
y_col = "current_annual_salary"
X = data.skb.select(s.all() - y_col).skb.mark_as_X()
y = data[y_col].skb.mark_as_y()


def compute_weights(X):
    return (-(((X["year_first_hired"] - 2025) / 20) ** 2)).exp().rename("weight")


weight = X.skb.apply_func(compute_weights)
pred = X.skb.apply(skrub.TableVectorizer()).skb.apply(
    Ridge(), y=y, fit_kwargs={"sample_weight": weight}
)

graph

Could you add an example like this to the docstring? Maybe we could also add a mention to xgboost since it's likely what a lot of people will use this for.

Aside from that, I think it's good

@rcap107 rcap107 added this to the 0.7.0 milestone Oct 7, 2025

@rcap107 rcap107 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks @jeromedockes

@rcap107 rcap107 merged commit abec1ff into skrub-data:main Oct 9, 2025
29 checks passed
dierickxsimon pushed a commit to dierickxsimon/skrub that referenced this pull request Oct 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data_ops Something related to the skrub DataOps

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants