Skip to content

Use local rank for Responses Triton CUDA binding - #274

Open
sylvesterkaczmarek wants to merge 3 commits into
openai:mainfrom
sylvesterkaczmarek:fix/responses-triton-local-rank
Open

Use local rank for Responses Triton CUDA binding#274
sylvesterkaczmarek wants to merge 3 commits into
openai:mainfrom
sylvesterkaczmarek:fix/responses-triton-local-rank

Conversation

@sylvesterkaczmarek

Copy link
Copy Markdown

Summary

Bind the Responses API Triton backend to its node-local CUDA device instead of using global distributed rank as a device index.

This backend selects its device independently and does not call gpt_oss.torch.utils.init_distributed(), so the local-rank correction in #267 does not cover it.

Fixes #273.

Fix

  • keep global RANK for distributed identity and log prefixes;
  • use LOCAL_RANK for torch.cuda.set_device() and the CUDA device object;
  • fall back to global RANK when LOCAL_RANK is absent, preserving existing single-node/custom-launch behavior.

Regression coverage

Adds a CPU-only mocked regression with RANK=5 and LOCAL_RANK=1 verifying that:

  • global rank remains 5;
  • local rank is 1;
  • model loading receives cuda:1;
  • CUDA binding uses device index 1.

No model loading, checkpoint format, inference math, or logging identity is otherwise changed.

@chatgpt-codex-connector

Copy link
Copy Markdown

Note

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Signed-off-by: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Responses API Triton backend binds CUDA devices using global rank

1 participant