Clarification for PET-OAM Training Procedure

[Naisargi-Goyal]

Hi everyone,

I am seeking some clarification on the training procedure for PET-OAM model, as there seems to some inconsistencies between the paper and the README.

The paper (section IV.A) states that PET-OAM is "trained on all three datasets" (OMat, sAlex, MPtrj) and that "the model were pre-trained using non-conservative forces and stresses, then fine-tuned using their conservative counterparts." This could be interpreted as pre-training and fine-tuning being done on three datasets combined.

However, the README says that the training set is OMat --> sAlex + MPtrj which to me implies that it was first trained on OMat and then fine-tuned on sAlex + MPtrj, which is consistent with how other OAM models (such as eSEN-30M-OAM) were trained. On Matbench Discovery, this is the description for eSEN-30M-OAM: "Training was done by: (1) 2-epoch direct pre-training on OMat24; (2) 2-epoch conservative fine-tuning on OMat24; (3) 1-epoch conservative fine-tuning on MPtrj+sAlex." On the contrary, Matbench discovery does not explicitly mention a fine-tuning stage on sAlex + MPtrj for PET-OAM which again implies that it was trained from scratch on a joint set of the three datasets.

Could you please clarify if PET-OAM pre-trained on OMat24 alone and then fine-tuned on MPtrj+sAlex as the README suggests, or it is was trained on all three datasets together as the paper suggests? If trained on the combined pool, how were the differing reference DFT settings between OMat24 and MPtrj accounted for? Thanks!

[frostedoyster]

Hello @Naisargi-Goyal, thanks for the question and we apologize if this wasn't clear from the paper. Indeed, the PET-OAM models were first pre-trained on OMat24 and then fine-tuned on sAlex+MPtrj. This is what you get from Matbench (https://github.com/janosh/matbench-discovery/tree/main/models/pet/training/pet-oam-xl-1.0.0). Note that the pre-training is itself split into a non-conservative stage and a conservative stage. This is also documented in Appendix A4 of the paper (picture here below).