What is the output if I have multiple training stage

[IpShingHeiLucas]

So in my code, I have set up 3 different stages, each with different loss weight and all training is set to disregard_previous_best=False,
Stage 1: Only data loss
Stage 2: all loss with Adam optimiser
Stage 3: all loss with L-BFGS-B

But when I run it, the loss in stage 1 is often the lowest. So when I used the trained PINN, which version will it be? The best model in stage 1 or best model in stage 3?

(If the answer is best model in stage 1)
If I want the to use best model in stage 3 as my final model, what should I change? (changing disregard_previous_best to True)

[lululxvi]

By default, the model in the last training step.