`sky launch` using the wrong GCP project, despite using `gcloud config set project`

[turian]

Using a small variation of hello_sky.yaml that allows GCP, I get a bunch of provisioning errors that reference the wrong GCP project:

W 12-17 14:46:04 common_utils.py:216] Caught <HttpError 403 when requesting https://compute.googleapis.com/compute/v1/projects/wrong-project-name?alt=json returned "Required 'compute.projects.get' permission for 'projects/wrong-project-name'". Details: "[{'message': "Required 'compute.projects.get' permission for 'projects/wrong-project-name'", 'domain': 'global', 'reason': 'forbidden'}]">. Retrying.
W 12-17 14:46:06 common_utils.py:216] Caught <HttpError 403 when requesting https://compute.googleapis.com/compute/v1/projects/wrong-project-name?alt=json returned "Required 'compute.projects.get' permission for 'projects/wrong-project-name'". Details: "[{'message': "Required 'compute.projects.get' permission for 'projects/wrong-project-name'", 'domain': 'global', 'reason': 'forbidden'}]">. Retrying.

In the shell I try:

gcloud config set project correct-project-name

I confirm that ~/.config/gcloud/configurations/config_default has project = correct-project-name.

But still, sky launch throws these errors? Why and how do I fix?

I have searched for previous github issues, but found nothing related.

[infwinston]

Hi @turian thanks for the question!
Could you check whether your ~/.config/gcloud/active_config indicates to the correct configuration file under ~/.config/gcloud/configurations?

[infwinston]

This command should be able to help to activate the correct project :)

gcloud config configurations activate [project-name]

https://cloud.google.com/sdk/gcloud/reference/config/configurations/activate

[infwinston]

Hi @turian

Is your problem solved? We added a note on our doc https://skypilot.readthedocs.io/en/latest/getting-started/installation.html

We'd appreciate it if you could confirm it works, thanks!

[turian]

~/.config/gcloud/active_config contains default. As above ~/.config/gcloud/configurations/config_default has project = correct-project-name.

I run:

gcloud config configurations activate default

But I still get the following error from above:

W 12-17 14:46:04 common_utils.py:216] Caught <HttpError 403 when requesting https://compute.googleapis.com/compute/v1/projects/wrong-project-name?alt=json returned "Required 'compute.projects.get' permission for 'projects/wrong-project-name'". Details: "[{'message': "Required 'compute.projects.get' permission for 'projects/wrong-project-name'", 'domain': 'global', 'reason': 'forbidden'}]">. Retrying.
W 12-17 14:46:06 common_utils.py:216] Caught <HttpError 403 when requesting https://compute.googleapis.com/compute/v1/projects/wrong-project-name?alt=json returned "Required 'compute.projects.get' permission for 'projects/wrong-project-name'". Details: "[{'message': "Required 'compute.projects.get' permission for 'projects/wrong-project-name'", 'domain': 'global', 'reason': 'forbidden'}]">. Retrying.

Note that I am using the latest pip-installed skypilot. Tell me if I should pip install from git main?

[turian]

If I run:

gcloud config configurations list

I get ONLY correct-project-name:

NAME     IS_ACTIVE  ACCOUNT           PROJECT        COMPUTE_DEFAULT_ZONE  COMPUTE_DEFAULT_REGION
default  True       LASTNAME@gmail.com  correct-project-name  europe-west3-b        europe-west3

[turian]

I also note that the log file in your output does not exist after the command fails:

I 12-22 22:00:52 cloud_vm_ray_backend.py:903] To view detailed progress: tail -n100 -f /Users/joseph/sky_logs/sky-2022-12-22-22-00-50-311494/provision.log

yourHostName:hello-sky joseph$ tail -n100 -f /Users/joseph/sky_logs/sky-2022-12-22-22-00-50-311494/provision.log
tail: /Users/joseph/sky_logs/sky-2022-12-22-22-00-50-311494/provision.log: No such file or directory

[concretevitamin]

@turian Could you please try switching the project using the following commands:

• Run gcloud init to choose the new project
• Then gcloud auth application-default login

(The provision log not being written is likely due to the provision errors.)

[github-actions]

This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[github-actions]

This issue was closed because it has been stalled for 10 days with no activity.

[Michaelvll]

This should stay open.