Using automatminer models as featurizers

[computron]

Typically, a matbench model would be considered an "output" of the study.

However, let's say you are trying to create a model that relates crystal structure to band gap. You could of course have the standard matminer structure/composition features and use that as your candidate feature set.

But another option is to use a different matbench (or other ML) model as a feature as well. For example, let's say you previously made a matbench model to relate structure/composition->bulk modulus. Now, when you are predicting band gap, you can use that previous matbench model to get a value for bulk modulus which becomes a new feature for band gap prediction.

Related ideas include the papers:
[1] M.L. Hutchinson, E. Antono, B.M. Gibbons, S. Paradiso, J. Ling, B. Meredig, Overcoming data scarcity with transfer learning, ArXiv. (2017).
[2] Y. Zhang, C. Ling, A strategy to apply machine learning to small datasets in materials science, Npj Comput. Mater. 25 (2018) 28–33.

and the goal of the propnet project in our group is to do similar things as well.

[Doppe1g4nger]

This is an interesting concept. "Feature learning" is a pretty common topic in deep learning. Usually as an implicit part of a multiplayer neural network or as a goal in itself with things like word2vec and autoencoders. It seems like there's not really enough data to apply that to property prediction yet though as far as I can tell. It would be interesting to see how valuable to the models these chained predicted features would be.

[albalu]

We'll certainly do this. I can imagine something like this should go in matminer as it would still get a composition/structure/etc and return a set of features. Perhaps we need to have a set of qualification criteria for a model added by us or the community such as ncompounds>2000 score > 90 %, diverse chemistry and/or structure or something like that? The reason I think we need to include only relatively accurate models is not to introduce more noise into our feature space.

[computron]

@Doppe1g4nger I'd lump this more into "transfer learning" than "feature learning" - i.e., you are using models trained on a different data set to inform your ML on your current data set. Although some transfer learning is about transferring information on hyperparameters, this is about transferring feature data

[ardunn]

@albalu I believe Logan Ward already implemented something for this purpose in matminer called StackedFeaturizer

[computron]

@ardunn good point. We might want to have a set of pre-fit StackedFeaturizers in matbench that matminer can use

[ardunn]

note to self: We should also outputs of pretrained use graph-net models

edit: this can be an option on autofeaturizer