Skip to content

generate_custom_image script doesn't work #77

@JSPromisel

Description

@JSPromisel

I tried running the following command:

python generate_custom_image.py --image-name some-name --dataproc-version 2.0.36-ubuntu18 --zone us-east1-c --disk-size 100 --metadata somekey=somevalue --customization-script /tmp/script.sh --gcs-bucket some-bucket

And got the following error:

INFO:custom_image_utils.args_inferer:Getting Dataproc base image name...
Traceback (most recent call last):
  File "generate_custom_image.py", line 95, in <module>
    main()
  File "generate_custom_image.py", line 86, in main
    args = parse_args(sys.argv[1:])
  File "generate_custom_image.py", line 57, in parse_args
    args_inferer.infer_args(args)
  File "/home/jakepromisel/Github/custom-images/custom_image_utils/args_inferer.py", line 225, in infer_args
    _infer_base_image(args)
  File "/home/jakepromisel/Github/custom-images/custom_image_utils/args_inferer.py", line 191, in _infer_base_image
    args.dataproc_version)
  File "/home/jakepromisel/Github/custom-images/custom_image_utils/args_inferer.py", line 175, in _get_dataproc_image_path_by_version
    "Cannot find dataproc base image with dataproc-version=%s." % version)
RuntimeError: Cannot find dataproc base image with dataproc-version=2.0.36-ubuntu18.

But I believe this version does exist. Looking into this further, I tried recreating the gcloud command run to verify the dataproc version:

gcloud compute images list --project cloud-dataproc --filter "labels.goog-dataproc-version = 2-0-36 AND NOT name ~ -eap$ AND status = READY" --format "csv[no-heading=true](name,labels.goog-dataproc-version)" --sort-by=~creationTimestamp

And I get the following warning:

WARNING: --filter : operator evaluation is changing for consistency across Google APIs.  labels.goog-dataproc-version=2-0-36 currently does not match but will match in the near future.  Run `gcloud topic filters` for details.

If this repository is no longer maintained and shouldn't be used, can someone point me to how I should go about creating a custom image? From what I can tell, the docs still say to use this script. I even tried on a vm where I installed the recommended GCloud version (181.0.0) which is from 2017 and it still didn't work.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions