load pytorch model with onnx training on spark with gpu?

[mullerhai]

HI, we want to load pytorch model with onnx training on spark with gpu for training pytorch model ,could we do that

[mullerhai]

or use javacpp -pytorch write network model load to spark on GPU

[rishic3]

Hi @mullerhai ! Thanks for your interest in GPU-accelerated ML/DL on Spark. Are you interested primarily in distributed training or distributed inference?
For distributed inference we would recommend starting with the PyTorch DL inference examples. These will walk through how to train a model on single-node, save to file system, and load on workers for distributed inference on Spark.
For distributed training with PyTorch you can check out the TorchDistributor examples on Spark.

Let us know if this helps answer your question.

[mullerhai]

Hi @mullerhai ! Thanks for your interest in GPU-accelerated ML/DL on Spark. Are you interested primarily in distributed training or distributed inference? For distributed inference we would recommend starting with the PyTorch DL inference examples. These will walk through how to train a model on single-node, save to file system, and load on workers for distributed inference on Spark. For distributed training with PyTorch you can check out the TorchDistributor examples on Spark.

Let us know if this helps answer your question.

not what I want ,I need scala language code for that