Is customized objective function supported?

[maycuiyan]

I want to train an LGBM regression model but with customized loss/objective function.
With the native API, I can do something like this:
model = lightgbm.LGBMRegressor(..., objective = customized_objective)
model.fit(...)
How can I do this with FLAML?

[qingyun-wu]

Hi maycuiyan,

Thank you for the question. You can do it in FLMAL by adding a customized learner:

1. Create a customized LGBM estimator class (in which you can specify the customized_objective)
2. Add this customized learner to the FLAML's AutoML instance through the add_learner() function.
3. Then, you can choose to use this newly added learner by specifying the estimator_list argument in FLAML AutoML's fit() function.
   You can find an example of how to add a customized learner in section 3 of this notebook https://github.com/microsoft/FLAML/blob/main/notebook/flaml_automl.ipynb

Please let me know if you need more help with adding your customized learner.

[qingyun-wu]

Hi Maycuiyan,

I just added a more concrete example (section 4) in the notebook about lightgbm for your reference: https://github.com/microsoft/FLAML/blob/lgbm_w_customized_obj/notebook/flaml_lightgbm.ipynb

[qingyun-wu]

Hi maycuiyan,

I hope you find the notebook helpful in creating an LGBM regression model with your customized objective.

Just a reminder that the customized objective specified in the LGBM estimator class is the training objective for this learner. It does not affect the optimization objective, which guides the search of learners and hyperparameters, in FLAML's AutoML class. If you also want to specify a customized metric for the optimization process in FLAML, you can do it by passing a customized metric through the 'metric' argument in the fit() function of the AutoML instance created (ref. https://github.com/microsoft/FLAML/blob/main/flaml/automl.py#L811).

Thanks,
Qingyun

[maycuiyan]

Hi @qingyun-wu, thanks a lot for putting an example in the notebook. I'm able to use both a customized objective and metric function, and it works nicely for me.

I also managed to use customized objective for xgboost (regression) and it seems that I have to override the fit() method, instead of init() as in the lgbm case. Not sure this is the best approach but at least it also works.

[qingyun-wu]

Hi maycuiyan,

Thank you for your feedback. We have updated the library (v0.3.1), and now you should be able to use a customized objective in xgboost (regression) similar to the way in LGBM. Please find an example in this notebook: https://github.com/microsoft/FLAML/blob/main/notebook/flaml_xgboost.ipynb

Please check it out, and we welcome more feedback.

Thanks,
Qingyun

[maycuiyan]

I have upgraded to v0.3.1 and it works just as expected. The unified interface is very nice. Thanks a lot for the update and taking my feedback into consideration!