Adding transformer param to HoeffdingTreeRegressor class

[caiogimenes]

Hey there!
I was wondering about adding a transformer param to HoeffdingTreeRegressor so instead of running the whole feature space it would restrict to a particular subspace based on a metric (like PersonCorr()). The feature_selection would take place at the best split evaluation.

It would be a slighly broader implementation of this paper's algorithm:

Mastelini, S.M., Ponce de Leon Ferreira de Carvalho, A.C. (2020). 2CS: Correlation-Guided Split Candidate Selection in Hoeffding Tree Regressors. In: Cerri, R., Prati, R.C. (eds) Intelligent Systems. BRACIS 2020. Lecture Notes in Computer Science(), vol 12320. Springer, Cham. https://doi.org/10.1007/978-3-030-61380-8_23

[e10e3]

Would this be different from creating a pipeline with feature_selection.SelectKBest as a step? It can use PearsonCorr to select a feature subspace.

[caiogimenes]

I think the main difference is at the timing and the scope.
Using the pipeline the model will effectively see only the k best features selected by the transformer at every instance.
When embedded to the model, the SelectKBest will only be called at a split decision. The model still see all features.
The selection process is used only to make the splitting decision more efficient and robust at a specific point in the tree's structure.