Error when using `to_pandas_dataframe` method on input datatsets ot a Run

[bhargav-kanda]

Hi,

I am passing a DatasetConsumptionConfig (Dataset.as_named_input()) object as one of the inputs to my CommandStep. In the script file that runs on the compute target, when I try to get the dataframe from the dataset using run.input_datasets['name_of_dataset'].to_pandas_dataframe(), I get the following error:

Traceback (most recent call last):
  File "/home/app/pyenv/lib/python3.8/site-packages/azureml/data/dataset_error_handling.py", line 101, in _try_execute
    return action()
  File "/home/app/pyenv/lib/python3.8/site-packages/azureml/data/tabular_dataset.py", line 168, in <lambda>
    df = _try_execute(lambda: dataflow.to_pandas_dataframe(on_error=on_error,
  File "/home/app/pyenv/lib/python3.8/site-packages/azureml/dataprep/api/_loggerfactory.py", line 210, in wrapper
    return func(*args, **kwargs)
  File "/home/app/pyenv/lib/python3.8/site-packages/azureml/dataprep/api/dataflow.py", line 696, in to_pandas_dataframe
    return get_dataframe_reader().to_pandas_dataframe(self,
  File "/home/app/pyenv/lib/python3.8/site-packages/azureml/dataprep/api/_dataframereader.py", line 295, in to_pandas_dataframe
    return to_pandas_feather()
  File "/home/app/pyenv/lib/python3.8/site-packages/azureml/dataprep/api/_dataframereader.py", line 231, in to_pandas_feather
    dataflow._engine_api.execute_anonymous_activity(
  File "/home/app/pyenv/lib/python3.8/site-packages/azureml/dataprep/api/_aml_helper.py", line 38, in wrapper
    return send_message_func(op_code, message, cancellation_token)
  File "/home/app/pyenv/lib/python3.8/site-packages/azureml/dataprep/api/engineapi/api.py", line 120, in execute_anonymous_activity
    response = self._message_channel.send_message('Engine.ExecuteActivity', message_args, cancellation_token)
  File "/home/app/pyenv/lib/python3.8/site-packages/azureml/dataprep/api/engineapi/engine.py", line 291, in send_message
    raise_engine_error(response['error'])
  File "/home/app/pyenv/lib/python3.8/site-packages/azureml/dataprep/api/errorhandlers.py", line 10, in raise_engine_error
    raise ExecutionError(error_response)
azureml.dataprep.api.errorhandlers.ExecutionError: 
Error Code: ScriptExecution.DatastoreResolution.Unexpected
Failed Step: 3fba0a5d-426b-476e-9efd-9a6a2271b5a7
Error Message: ScriptExecutionException was caused by DatastoreResolutionException.
  DatastoreResolutionException was caused by UnexpectedException.
    Unexpected failure making request to fetching info for Datastore 'workspaceblobstore' in subscription: Using base service url: https://eastus2.experiments.azureml.net. HResult: 0x80131501.
      The SSL connection could not be established, see inner exception.
| session_id=

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "data_cleaning/clean_orders.py", line 20, in <module>
    orders_df = run.input_datasets['raw_orders'].to_pandas_dataframe()
  File "/home/app/pyenv/lib/python3.8/site-packages/azureml/data/_loggerfactory.py", line 129, in wrapper
    return func(*args, **kwargs)
  File "/home/app/pyenv/lib/python3.8/site-packages/azureml/data/tabular_dataset.py", line 168, in to_pandas_dataframe
    df = _try_execute(lambda: dataflow.to_pandas_dataframe(on_error=on_error,
  File "/home/app/pyenv/lib/python3.8/site-packages/azureml/data/dataset_error_handling.py", line 104, in _try_execute
    _dataprep_error_handler(e, message, is_dprep_exception)
  File "/home/app/pyenv/lib/python3.8/site-packages/azureml/data/dataset_error_handling.py", line 156, in _dataprep_error_handler
    raise AzureMLException(message, inner_exception=e)
azureml._common.exceptions.AzureMLException: AzureMLException:
	Message: Execution failed in operation 'to_pandas_dataframe' for Dataset(id='', name='raw_orders', version=1, error_code=ScriptExecution.DatastoreResolution.Unexpected,error_message=ScriptExecutionException was caused by DatastoreResolutionException.
  DatastoreResolutionException was caused by UnexpectedException.
    Unexpected failure making request to fetching info for Datastore 'workspaceblobstore' in subscription:  Using base service url: https://eastus2.experiments.azureml.net. HResult: 0x80131501.
      Failed due to inner exception of type: HttpRequestException
| session_id=) ErrorCode: ScriptExecution.DatastoreResolution.Unexpected
	InnerException 
Error Code: ScriptExecution.DatastoreResolution.Unexpected
Failed Step: 3fba0a5d-426b-476e-9efd-9a6a2271b5a7
Error Message: ScriptExecutionException was caused by DatastoreResolutionException.
  DatastoreResolutionException was caused by UnexpectedException.
    Unexpected failure making request to fetching info for Datastore 'workspaceblobstore' in subscription:. Using base service url: https://eastus2.experiments.azureml.net. HResult: 0x80131501.
      The SSL connection could not be established, see inner exception.
| session_id=
	ErrorResponse 
{
    "error": {
        "message": "Execution failed in operation 'to_pandas_dataframe' for Dataset(id='', name='raw_orders', version=1, error_code=ScriptExecution.DatastoreResolution.Unexpected,error_message=ScriptExecutionException was caused by DatastoreResolutionException.\n  DatastoreResolutionException was caused by UnexpectedException.\n    Unexpected failure making request to fetching info for Datastore 'workspaceblobstore' in subscription: '. Using base service url: https://eastus2.experiments.azureml.net. HResult: 0x80131501.\n      Failed due to inner exception of type: HttpRequestException\n| session_id=) ErrorCode: ScriptExecution.DatastoreResolution.Unexpected"
    }
}

Can you please tell me why this error occurs? Serached online for this but wasn't able to figure out the issue.

TIA.

[ChunyuMSFT]

@bhargav-kanda , sorry for late response, this usually happens if the base image is not our supported OS version, would you please help provide the OS version in your base image?