fix(validator): isolate each NCCL benchmark run in its own namespace - #2443

Open

mikecook wants to merge 13 commits into

NVIDIA:mainfrom

mikecook:split/nccl-namespace-isolation

Commits on Aug 29, 2026

fix(validator): isolate each NCCL benchmark run in its own namespace
mikecook
committed
1cd1255
fix(validator): preserve structured errors from namespace lifecycle
mikecook
committed
9e54920
fix: verify NCCL namespace ownership before reuse or delete
mikecook
committed
2e76cc2
fix: run namespace cleanup before Trainer teardown on self-install path
mikecook
committed
39fbf35
fix: don't fail the benchmark on a namespace-termination wait timeout
mikecook
committed
8290cb7
fix: fold NCCL variant into the per-run namespace, not runID alone
mikecook
committed
0a364b7
fix: prune stale orphaned NCCL benchmark namespaces on next run
mikecook
committed
23a23a2
test: harden the finalizer-wait regression guard's timing check
mikecook
committed
5964fe1
fix: ignore stale Deleted and terminal pod events in the launcher watch
mikecook
committed
da236f9
fix: reclaim a stale TrainJob by delete-and-recreate, not update
mikecook
committed
a1b8e57
fix: don't prune a stale-looking NCCL namespace that still has a live pod
mikecook
committed
5503000
fix: reject an empty owning UID in cleanupNCCLResources
mikecook
committed
6092710
docs: correct a stale doc comment on the namespace-gone timeout test
mikecook
committed
a105267