[yueming-yuan]

Enable --experts-shared-outer-loras for multi-LoRA/Tinker. Each tenant keeps independent factors and Adam state; sampling snapshots use the existing shared-outer adapter format.

Record the expert layout in checkpoint metadata and reject restores under the other layout. Checkpoints written before the field existed are per-expert, since multi-LoRA rejected the shared-outer flag until this PR. Boundary: checkpoints saved by earlier heads of this branch (before 4ba90fe) carry no field and load as per-expert too; the H200 validation checkpoints below are the only such files and are scratch.

Stacked on #4480, which carries the per-request gradient accumulation fix this PR originally included (re-syncing retained expert gradients scaled earlier requests by 1/EP). This PR keeps the shared-outer part: the shared factors are replicated across EP, so their gradients are summed across EP once per request (allreduce_expert_parallel_replicated_grads).

Pins Megatron-Bridge to 2d0aa270 in all three Dockerfiles: radixark/Megatron-Bridge#45 as merged, on top of #50's fused-layer TP gradient fixes. The previous pin, 8cd3466d, rejects experts_shared_outer_loras for MultiLoRA. The H200 table below was measured on this branch at bc020cb with Bridge#45 at cd417d03, i.e. without those fixes; it is kept as the pre-fix record and is not validation of the current heads. At the current head, tests/e2e/lora/test_shared_outer_slots.py passes against Bridge 2d0aa270 on 2× B200 (TP1/EP2), and at Bridge#45's head on TP1/EP2 and TP2/EP2. No Megatron-Core or SGLang changes required. SGLang's fused shared-expert combination remains unsupported.

H200 validation: Qwen3.5-35B-A3B, 3 tenants, rank16/alpha32, train TP2/EP4 + inference TP4. Each tenant trained on a distinct fixed 8,192-token DAPO sequence with CE and Adam lr=1e-5 for 50 steps.

Tenant	Training NLL: step 1 → 50	SGLang prompt NLL: base → trained
0	0.926204 → 0.214607	1.159517 → 0.318825
1	0.950774 → 0.242959	1.241800 → 0.333805
2	0.978503 → 0.233334	1.058517 → 0.212997

Sampling used 512-token prefixes and a resumed step-51 snapshot. All tenants generated successfully. Checkpoint restore matched all 1,920 tensors/tenant exactly, including FP32 masters and Adam state. Repeated-forward logprobs varied even without an update: ~0.02 mean absolute difference after training, and ~0.08 in a separate serial zero-adapter control. Checkpoint tensors are exact; forward determinism has not been established or localized.

Other checks: 36 Bridge GPU cases passed; EP2 and TP2/EP2 integration passed accumulation, norm, isolation, cross-slot restore and resumed Adam. The old accumulation path fails the same test. Pre-commit and all CPU CI shards passed (15,693 tests). GPU CI scope: run-ci-multi-lora.

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}