Az ml extension fails with ERROR: The extension ml already exists

[srinisudharsan]

Describe the bug

The same command was working until yesterday.
Az cli version 2.52.0
az ml version 2.30.0

Command run:
az ml online-endpoint show --subscription _{--resource-group --workspace --name --only-show-errors}

cannot import name '_T' from 'marshmallow.fields' (/opt/az/azcliextensions/ml/marshmallow/fields.py)

STDERR: ERROR: An error occurred whilst updating.
ERROR: The extension ml already exists.
ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: [Errno 17] File exists: '/opt/az/azcliextensions/ml'
Traceback (most recent call last):
File "/opt/az/lib/python3.10/site-packages/azure/cli/core/extension/operations.py", line 429, in update_extension
_add_whl_ext(cli_ctx=cmd_cli_ctx, source=download_url, ext_sha256=ext_sha256,
File "/opt/az/lib/python3.10/site-packages/azure/cli/core/extension/operations.py", line 113, in _add_whl_ext
raise CLIError('The extension {} already exists.'.format(extension_name))
knack.util.CLIError: The extension ml already exists.

Related command

az ml online-endpoint show --subscription _{--resource-group --workspace --name --only-show-errors}

Errors

cannot import name '_T' from 'marshmallow.fields' (/opt/az/azcliextensions/ml/marshmallow/fields.py)

STDERR: ERROR: An error occurred whilst updating.
ERROR: The extension ml already exists.
ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: [Errno 17] File exists: '/opt/az/azcliextensions/ml'
Traceback (most recent call last):
File "/opt/az/lib/python3.10/site-packages/azure/cli/core/extension/operations.py", line 429, in update_extension
_add_whl_ext(cli_ctx=cmd_cli_ctx, source=download_url, ext_sha256=ext_sha256,
File "/opt/az/lib/python3.10/site-packages/azure/cli/core/extension/operations.py", line 113, in _add_whl_ext
raise CLIError('The extension {} already exists.'.format(extension_name))
knack.util.CLIError: The extension ml already exists.

Issue script & Debug output

Command run:
az ml online-endpoint show --subscription _{--resource-group --workspace --name --only-show-errors}

cannot import name '_T' from 'marshmallow.fields' (/opt/az/azcliextensions/ml/marshmallow/fields.py)

STDERR: ERROR: An error occurred whilst updating.
ERROR: The extension ml already exists.
ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: [Errno 17] File exists: '/opt/az/azcliextensions/ml'
Traceback (most recent call last):
File "/opt/az/lib/python3.10/site-packages/azure/cli/core/extension/operations.py", line 429, in update_extension
_add_whl_ext(cli_ctx=cmd_cli_ctx, source=download_url, ext_sha256=ext_sha256,
File "/opt/az/lib/python3.10/site-packages/azure/cli/core/extension/operations.py", line 113, in _add_whl_ext
raise CLIError('The extension {} already exists.'.format(extension_name))
knack.util.CLIError: The extension ml already exists.

Expected behavior

Show the az ml endpoint

Environment Summary

Az cli version 2.52.0
az ml version 2.30.0
OS ubuntu 22.04

Additional context

No response

[yonzhan]

Thank you for opening this issue, we will look into it.

[microsoft-github-policy-service]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github.

[srinisudharsan]

May be related to
microsoft/promptflow#3902

[singankit]

Thanks @srinisudharsan for raising the issue. The issue is already being tracked and fix is being worked on. Will update the bug once fix is released.

[srinisudharsan]

@singankit could you please provide the issue where it is being tracked

[frank-dong-ms]

@singankit this issue is blocking our production pipeline, is there any workaround for this issue? Like can we pin azure cli version or ml extension version anything like that?

[singankit]

@frank-dong-ms downgrading marshmallow will certainly help. Following command can be used as workaround:

pip install "marshmallow<3.24.0"

[frank-dong-ms]

@singankit thanks, does that mean error "knack.util.CLIError: The extension ml already exists." is also caused by marshmallow upgrade?