Skip to content

Support dist.ReduceOp.AVG on XLA device #7782

Open
@miladm

Description

🚀 Feature

Enable dist.ReduceOp.AVG on XLA. This operation is available on NCCL devices only. (Reference). This op enables operations like torch.distributed.all_reduce(<tensor_name>, op=dist.ReduceOp.AVG)

Error when using the op

ValueError: Invalid reduce op <torch.distributed.distributed_c10d.ReduceOp object at 0x7f0cb2d3b830>

Metadata

Assignees

No one assigned

    Labels

    SPMD / DistributedusabilityBugs/features related to improving the usability of PyTorch/XLA

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions