RL generated molecules or post-RL sampling generated molecules

[Khanhnguyen2027]

Hello,

When curating a final set of molecules, should we sample only from the final trained agent checkpoint, or just include top scoring molecules encountered during RL training, or combine both sets? If a combination is preferred, what proportions would you suggest, and what post processing steps do you recommend

[halx]

As you would filter out interesting molecules as per a desired criterium, typically the scoring function or a subset, in post-processing you would do that on all the molecules generated during RL. "Good" molecules, as per your definition, can be produced at any time during RL. REINVENT also has a scoring runmode in case you wish to apply additional scoring functions different from the ones used in RL.

A newly trained model is only biased towards the scoring functions you have used. There is no principal guarantee that generated molecules sampled from a freshly trained model will be "good". It may seem a better strategy in practice to run RL far into convergence as that increases probablity of "good" molecules. Sampling seems rather unnecessary in practice. GIven the stochastic nature of RL I would rather recommend to run multiple independent runs.