Skip to content

Commit 4057404

Browse files
author
Jinliang Li
committed
add hybrid_fsdp_expt_group in pg_collection
1 parent 143113a commit 4057404

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

megatron/core/distributed/fsdp/mcore_fsdp_adapter.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -236,6 +236,7 @@ def _init_dist_index(self, pg_collection):
236236
hybrid_fsdp_group = pg_collection.dp_cp
237237
# This has not been tested yet.
238238
expt_dp_group = getattr(pg_collection, 'intra_expt_dp', None)
239+
hybrid_fsdp_expt_group = getattr(pg_collection, 'expt_dp', None)
239240
ep_group = getattr(pg_collection, 'ep', None)
240241
else:
241242
dp_cp_group = pg_collection.dp_cp

0 commit comments

Comments
 (0)