Skip to content

Fix the precision error compared to the Megatron-LM caused by the data type of moe_act_func and mhc recompute. #272

Fix the precision error compared to the Megatron-LM caused by the data type of moe_act_func and mhc recompute.

Fix the precision error compared to the Megatron-LM caused by the data type of moe_act_func and mhc recompute. #272