fix cg acess issue by using dict instead of list to iteratively acces…#3867
fix cg acess issue by using dict instead of list to iteratively acces…#3867yaox12 merged 2 commits intoNVIDIA:devfrom
Conversation
|
https://github.com/NVIDIA/Megatron-LM/pull/3625/changes There is an equivalent change on main, why not reflect this PR? update: main has the exact code now except it misses the test, and doesnt refactor the setter and getter into their own functions. When we go to merge dev into main I can take an AI to make sure these get into main as well |
|
/ok to test bddcdb3 |
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/23161562822 |
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/23165894341 |
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/23167652445 |
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/23170581013 |
|
/ok to test 00d2b4b |
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/23220473186 |
Fix nested CUDA-graph attribute access for Flex MoE token dispatchers.
Summary
token_dispatcher.cudagraph_attrs, so entries like_comm_manager.token_probs,_comm_manager.token_indices, and_comm_manager.routing_mapare handled correctly.getattr/setattrusage in MoE CUDA-graph capture and replay paths with shared nested-attribute helpers.This keeps existing flat attribute handling unchanged while fixing Flex dispatcher backends such as
deepepandhybridep.What does this PR do ?
Contribution process
Pre-checks
Code review
Feel free to message or comment the @mcore-oncall to help accelerate your merge into main. The less complex your PR is, the faster it will be approved and merged!
All PRs start as draft. If you open a non-draft PR, it will be automatically converted to draft.
Step 1: Mark PR as "Ready for Review"
.github/CODEOWNERS.Final Review might get declined if these requirements are not fulfilled.
Step 2: Final Review
For PRs that change
megatron/core, once all expert reviewers have approved, theFinal Reviewlabel is applied automatically and final reviewers are assigned.For PRs outside
megatron/core, this step is skipped.Step 3: Approved
Once all required reviewers have approved, the
Approvedlabel is applied automatically.Merge
Any member of mcore-engineers will be able to merge your PR.
For MRs into `dev` branch
The proposed review process for `dev` branch is under active discussion.MRs are mergable after one approval by either
eharper@nvidia.comorzijiey@nvidia.com.