Part 1: Introduce multi-node SPMD support for Neuron #10126

Job	Run time
get-torch-commit	2s
Build XLA CUDA plugin / build	40m 43s
Build PyTorch/XLA / build	1h 7m 36s
Build PyTorch with CUDA / build	25m 1s
TPU tests	0s
GPU tests requiring torch CUDA / test (python_tests, linux.8xlarge.nvidia.gpu)	8m 9s
GPU tests / test (benchmark_tests)	13m 59s
GPU tests requiring torch CUDA / test (triton_tests, linux.g5.4xlarge.nvidia.gpu)	6m 12s
Build docs / build-docs	1m 46s
GPU tests / test (python_tests, xla_op1)	32m 1s
GPU tests / test (python_tests, xla_op2)	9m 12s
GPU tests / test (python_tests, xla_op3)	48m 50s
GPU tests / test (python_tests, torch_mp_op)	30m 6s
CPU tests / test (benchmark_tests)	8m 0s
GPU tests / test (cpp_tests, cpp_tests1)	10m 43s
CPU tests / test (python_tests, xla_op1)	13m 38s
GPU tests / test (cpp_tests, cpp_tests2)	15m 39s
CPU tests / test (python_tests, xla_op2)	4m 20s
CPU tests / test (python_tests, xla_op3)	6m 57s
CPU tests / test (python_tests, torch_mp_op)	10m 25s
CPU tests / test (cpp_tests, cpp_tests1)	4m 52s
CPU tests / test (cpp_tests, cpp_tests2)	5m 23s
	6h 3m 34s

Provide feedback