Inference results have a big difference

[hillaric1]

I use two method to infer the result.

1. merge two lora models, sft and RLHF, then use merged model to infer the result
2. directly use adapters to infer the results

There two way have a very different result.

[Jintao-Huang]

I think you only need to merge the RLHF LoRA

[hillaric1]

@Jintao-Huang There is answer which means I need to merge two lora，#7377