-
Notifications
You must be signed in to change notification settings - Fork 67
Open
Description
I tried running the following command:
python generate_custom_image.py --image-name some-name --dataproc-version 2.0.36-ubuntu18 --zone us-east1-c --disk-size 100 --metadata somekey=somevalue --customization-script /tmp/script.sh --gcs-bucket some-bucket
And got the following error:
INFO:custom_image_utils.args_inferer:Getting Dataproc base image name...
Traceback (most recent call last):
File "generate_custom_image.py", line 95, in <module>
main()
File "generate_custom_image.py", line 86, in main
args = parse_args(sys.argv[1:])
File "generate_custom_image.py", line 57, in parse_args
args_inferer.infer_args(args)
File "/home/jakepromisel/Github/custom-images/custom_image_utils/args_inferer.py", line 225, in infer_args
_infer_base_image(args)
File "/home/jakepromisel/Github/custom-images/custom_image_utils/args_inferer.py", line 191, in _infer_base_image
args.dataproc_version)
File "/home/jakepromisel/Github/custom-images/custom_image_utils/args_inferer.py", line 175, in _get_dataproc_image_path_by_version
"Cannot find dataproc base image with dataproc-version=%s." % version)
RuntimeError: Cannot find dataproc base image with dataproc-version=2.0.36-ubuntu18.
But I believe this version does exist. Looking into this further, I tried recreating the gcloud command run to verify the dataproc version:
gcloud compute images list --project cloud-dataproc --filter "labels.goog-dataproc-version = 2-0-36 AND NOT name ~ -eap$ AND status = READY" --format "csv[no-heading=true](name,labels.goog-dataproc-version)" --sort-by=~creationTimestamp
And I get the following warning:
WARNING: --filter : operator evaluation is changing for consistency across Google APIs. labels.goog-dataproc-version=2-0-36 currently does not match but will match in the near future. Run `gcloud topic filters` for details.
If this repository is no longer maintained and shouldn't be used, can someone point me to how I should go about creating a custom image? From what I can tell, the docs still say to use this script. I even tried on a vm where I installed the recommended GCloud version (181.0.0) which is from 2017 and it still didn't work.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels