Refactor optimizer implementations and improve multi_tensor ops#36

lxd-cumt merged 2 commits into

flagos-ai:mainflagos-ai/TransformerEngine-FL:mainfrom

lihongyang1990:fix/refactor-optimizer-impl-and-multi-tensorlihongyang1990/TransformerEngine-FL:fix/refactor-optimizer-impl-and-multi-tensorCopy head branch name to clipboard