Single Graph splitting across multiple GPUs

[citrix123]

Hello Team,

Thanks for the nice framework. I have noticed that with multiple GPU recipe (DeepSeek-V3-Chat-multi-gpu-8.yaml) we are splitting the nodes of single Graph across the GPUs, which looks to be inefficient. Is it intentional ?
I see that single Graph Launch happens, and the nodes are waiting to run across the GPUs sequentially, which is equal to running the graph on single GPU.

We might make the implementation which can launch multiple graphs across multiple GPUs. which can be more optimal solution ?