Skip to content

Streamline VM bootstrap. #11462

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
fasaxc wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Streamline VM bootstrap. #11462

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 1 addition & 10 deletions .semaphore/vms/configure-test-vm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,13 +67,4 @@ if [ "$startup_success" = false ]; then
exit 23
fi

set +x
echo "$DOCKERHUB_PASSWORD" | ssh "ubuntu@${vm_ip}" -- docker login --username "$DOCKERHUB_USERNAME" --password-stdin
scp -r -C "$HOME/secrets" "ubuntu@${vm_ip}:/home/ubuntu/secrets"
set -x
${ssh_cmd} "gcloud auth activate-service-account --key-file=/home/ubuntu/secrets/secret.google-service-account-key.json && \
gcloud config set project unique-caldron-775 && \
gcloud storage cp '${GCS_WORKFLOW_DIR}/${COMPONENT}/fv-artifacts/*' /tmp && \
tar -xzf /tmp/working-copy.tgz && \
${CALICO_DIR_NAME}/${COMPONENT}/.semaphore/load-test-artifacts"

echo "VM $vm_name configured successfully."
9 changes: 8 additions & 1 deletion .semaphore/vms/create-test-vms
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,12 @@ for batch in "${batches[@]}"; do
fi
done

cat $my_dir/vm-bootstrap.sh |
sed "s|__GCS_WORKFLOW_DIR__|${GCS_WORKFLOW_DIR}|g" |
sed "s|__DOCKERHUB_USERNAME__|${DOCKERHUB_USERNAME}|g" |
sed "s|__CALICO_DIR_NAME__|${CALICO_DIR_NAME}|g" |
sed "s|__COMPONENT__|${COMPONENT}|g" > "/tmp/vm-bootstrap.sh"

# Do a bulk create; this is faster and it saves API quota.
gcloud --quiet compute instances bulk create \
--predefined-names="$names" \
Expand All @@ -51,7 +57,8 @@ gcloud --quiet compute instances bulk create \
--boot-disk-type=hyperdisk-balanced \
--max-run-duration="${MAX_RUN_DURATION}" \
--instance-termination-action=DELETE \
--metadata-from-file startup-script="$my_dir/vm-bootstrap.sh" \
--metadata-from-file startup-script="/tmp/vm-bootstrap.sh" \
--service-account=ci-test-vm@unique-caldron-775.iam.gserviceaccount.com \
Copy link

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Hardcoded service account and project ID. The value [email protected] contains a hardcoded project ID. Consider parameterizing this using a variable like ${GCP_PROJECT} for consistency with configure-test-vm (line 22).

Suggested change
--service-account=ci-test-vm@unique-caldron-775.iam.gserviceaccount.com \
--service-account=ci-test-vm@${GCP_PROJECT}.iam.gserviceaccount.com \

Copilot uses AI. Check for mistakes.
--metadata block-project-ssh-keys=TRUE,ssh-keys="ubuntu:$(ssh-keygen -y -f $HOME/.ssh/id_rsa)",enable-guest-attributes=TRUE

for batch in "${batches[@]}"; do
Expand Down
11 changes: 11 additions & 0 deletions .semaphore/vms/vm-bootstrap.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,4 +79,15 @@ EOF

mv /tmp/daemon.json /etc/docker/daemon.json
systemctl restart docker

set +x
DOCKERHUB_USERNAME=__DOCKERHUB_USERNAME__
DOCKERHUB_PASSWORD=$(gcloud secrets versions access latest --project=unique-caldron-775 --secret=ci-dockerhub-password)
Copy link

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Hardcoded project ID. The project ID unique-caldron-775 is hardcoded in both the secret access (line 85) and the service account (line 61 in create-test-vms). Consider using a variable like ${GCP_PROJECT} (already used in configure-test-vm line 22) for consistency and easier configuration management.

Suggested change
DOCKERHUB_PASSWORD=$(gcloud secrets versions access latest --project=unique-caldron-775 --secret=ci-dockerhub-password)
DOCKERHUB_PASSWORD=$(gcloud secrets versions access latest --project="${GCP_PROJECT}" --secret=ci-dockerhub-password)

Copilot uses AI. Check for mistakes.
echo "$DOCKERHUB_PASSWORD" | docker login --username "$DOCKERHUB_USERNAME" --password-stdin
set -x

gcloud storage cp '__GCS_WORKFLOW_DIR__/__COMPONENT__/fv-artifacts/*' /tmp
tar -xzf /tmp/working-copy.tgz
__CALICO_DIR_NAME__/__COMPONENT__/.semaphore/load-test-artifacts

touch /var/run/startup-script-complete
Loading