Skip to content

MpiStep throws Estimator deprecation warnings #1558

Open
@metazool

Description

@metazool

Python version: 3.8.1 as running on an Azure ML notebook VM / SDK version azureml-core-1.32.0

There are several issues here relating to lack of notebook examples for use of MpiStep to do a distributed training run in a pipeline, as the official documentation claims there should be. So I'm creating an example for internal use. When I create an MpiStep, I see this set of deprecation warnings:

'Estimator' is deprecated. Please use 'ScriptRunConfig' from 'azureml.core.script_run_config' with your own defined environment or an Azure ML curated environment.
WARNING:root:'distributed_backend' parameter will be deprecated. Please use 'distributed_training' instead.
WARNING:azureml.core.environment:'enabled' is deprecated. Please use the azureml.core.runconfig.DockerConfiguration object with the 'use_docker' param instead.

The step is set up like this. I'm explicitly trying to replace a ScriptRunConfig with a pipeline step, and don't see another way to run a PythonScriptStep in parallel than by using an MpiStep.

script_step = MpiStep(
    name='distributed-train-step',
    script_name=script,
    compute_target=compute_target,
    source_directory=project_folder,
    node_count=2,
    process_count_per_node=1,
    arguments=['--train_data', train_data.as_mount(),
               '--test_data', test_data.as_mount(),
               '--epochs', 100,
               '--lr', 0.005],
    custom_docker_image=docker_base_image,
    )

I understand if this isn't the place to log this kind of generic issue relating to the Azure ML SDK but if so, then where is?

Metadata

Metadata

Assignees

No one assigned

    Labels

    ADOIssue is documented on MSFT ADO for internal trackingCore UIdoc-enhancement

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions