Name of training data file

[NRauschmayr]

Reference: 0174624067

When calling fit, I indicated input_data.npinstead of input_data.npy e.g.

mxnet_estimator.fit({'train': 's3://rauscn-data/data/input_data.np')

The input_data.npy was neverthelss downloaded but the filename was then y. Initially I thought SageMaker had renamed the file, until I figured out that the filename was caused by the typo. When indicating for instance input_data. then the name of the downloaded file would be npy. Is this intended?

[ChoiByungWook]

Hello @NRauschmayr,

Thank you for bringing this to our attention.

If I understand this correctly, when calling fit(), the name of the dataset doesn't stay as originally attended and instead for the following examples, this happened:

mxnet_estimator.fit({'train': 's3://rauscn-data/data/input_data.np') = /opt/ml/input/data/train/y
mxnet_estimator.fit({'train': 's3://rauscn-data/data/input_data.npy') = /opt/ml/input/data/train/npy

The dataset that was used to train was renamed to y and npy when referenced respectively for input_data.np and input_data.npy.

Did this happen when training on SageMaker or through local mode?

I was looking through our Python SDK and I don't think we modify the URI provided by the user.

1. https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/job.py#L95
2. https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/job.py#L117

Would you be able to provide more information on how you are instantiating your estimator?

Please let me know if there is anything I can clarify.

Thanks!

[NRauschmayr]

Hello @ChoiByungWook,

yes exactly that is what happened
mxnet_estimator.fit({'train': 's3://rauscn-data/data/input_data.np') lead to /opt/ml/input/data/train/y

I am creating the Estimator, like this:

mxnet_estimator = MXNet('train.py',
                        role=role,
                        train_instance_type='local',#'ml.m5.xlarge',
                        train_instance_count=1,
                        output_path='s3://sagemaker-rauscn',
                        code_location='s3://sagemaker-rauscn',
                        framework_version='1.3.0', py_version='py2',
                        hyperparameters={'batch_size': 16,
                         'epochs': 10,
                         'learning_rate': 0.0001,
                         'wd': 0.0})

mxnet_estimator.fit({'train': 's3://sagemaker-rauscn/data/input_data.npy'})

[nadiaya]

If you go to the corresponding training job you ran in the sagemaker console what URI do you have specified in Input data configuration section?
Also it would be really useful to know what values do you have for Channel name, S3 data type, S3 data distribution type in the same exact section?

Thanks!

[NRauschmayr]

Hello @nadiaya,

When calling mxnet_estimator.fit({'train': 's3://sagemaker-rauscn/data/input_data.np'})
URI : s3://sagemaker-rauscn/data/input_data.np
Channel name: train
S3 data type: S3Prefix
S3 data distirbution type: FullyReplicated

[nadiaya]

But inside of the container for this training job you got: /opt/ml/input/data/train/y instead of the expected file?

Let me escalate this issue to the appropriate service team.

[NRauschmayr]

Exactly. In my training script the following crashes: train = np.load("/opt/ml/input/data/train/input_data.npy")
However this works: train = np.load("/opt/ml/input/data/train/y")

From a user perspective, it is probably best to just get an error message if there is such a typo in the filename.