Open
Description
Python version: 3.8.1 as running on an Azure ML notebook VM / SDK version azureml-core-1.32.0
There are several issues here relating to lack of notebook examples for use of MpiStep
to do a distributed training run in a pipeline, as the official documentation claims there should be. So I'm creating an example for internal use. When I create an MpiStep, I see this set of deprecation warnings:
'Estimator' is deprecated. Please use 'ScriptRunConfig' from 'azureml.core.script_run_config' with your own defined environment or an Azure ML curated environment.
WARNING:root:'distributed_backend' parameter will be deprecated. Please use 'distributed_training' instead.
WARNING:azureml.core.environment:'enabled' is deprecated. Please use the azureml.core.runconfig.DockerConfiguration object with the 'use_docker' param instead.
The step is set up like this. I'm explicitly trying to replace a ScriptRunConfig
with a pipeline step, and don't see another way to run a PythonScriptStep
in parallel than by using an MpiStep.
script_step = MpiStep(
name='distributed-train-step',
script_name=script,
compute_target=compute_target,
source_directory=project_folder,
node_count=2,
process_count_per_node=1,
arguments=['--train_data', train_data.as_mount(),
'--test_data', test_data.as_mount(),
'--epochs', 100,
'--lr', 0.005],
custom_docker_image=docker_base_image,
)
I understand if this isn't the place to log this kind of generic issue relating to the Azure ML SDK but if so, then where is?