Is the device parsing process redundant in backend configuration?

[fffffq00]

Link to code: backend

I've been debugging multi-GPU code and noticed some potentially redundant design in the device parsing process. Specifically looking at the sim_backend handling (and render_backend has the same issue):

Potential Bug in Device Creation

elif sim_backend == "physx_cuda":
    device = torch.device("cuda")    # !!! Bug: ignores device_id
    sim_device = (
        sapien.Device(f"cuda:{sim_device_id}") 
        if sim_device_id is not None
        else sapien.Device("cuda")
    )

Notice: torch.device("cuda") doesn't use the sim_device_id, while sapien.Device does - this seems inconsistent.

Redundant Mapping

sim_backend_name_mapping = {
    "cpu": "physx_cpu",      
    "cuda": "physx_cuda",    
    "gpu": "physx_cuda",
    "physx_cpu": "physx_cpu", 
    "physx_cuda": "physx_cuda",
}

Why the physx_ prefix? The mapping adds prefixes but the final devices still use base names (cpu/cuda). What's the purpose of this extra layer? Is this intentional design?
If this complexity isn't necessary, I'd be happy to submit a PR to simplify the code and fix the potential device_id inconsistency.

Looking forward to understanding the design rationale!

[StoneT2000]

That's a good catch, it probably should be using device id for the torch device. Sapien device object is only for the rendering backend.

That being said people have gotten multi gpu rl working (through puffer lib, they have an example) with the code here using torch run. Not sure what you are using

The physx prefix is added as we may potentially support more than just the physx physics simulation engine backend. We had considered supporting MJwarp although the integration efforts for that have been put on pause for now.

[fffffq00]

That makes sense, thanks for the explanation! Keeping the prefix for future multi-physics engine support is indeed reasonable.

I actually encountered this issue while testing on multi-GPU setups—when specifying cuda:X to run, the torch.device here wasn't correctly using the device_id. This should be straightforward to fix, and I'll submit a PR shortly to ensure consistency between the SAPIEN device and PyTorch device handling.

Related to this, I found a similar device ID handling issue in the SAPIEN backend and already submitted a PR SAPIEN#273 to fix it. With those changes, multi-GPU operation is working well now.