[dist] reuse comm groups when possible for memory saving - #5111

[dist] reuse comm groups when possible for memory saving#5111

gbyu-amd wants to merge 8 commits into

mainROCm/aiter:mainfrom

guanbao/reuse-comm-groupsROCm/aiter:guanbao/reuse-comm-groupsCopy head branch name to clipboard