Skip to content

Commit c21bf3d

Browse files
committed
Updated log messages and scripts
1 parent 964de52 commit c21bf3d

File tree

5 files changed

+79
-79
lines changed

5 files changed

+79
-79
lines changed

Makefile

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,23 @@
11
LAUNCH_SCRIPT := ./scripts/launch.sh
2-
CLEAN_CLOUD_SCRIPT := ./scripts/clean_cloud.sh
2+
CLEAN_SCRIPT := ./scripts/clean.sh
33
CONFIG_FILE := config.json
44

55
all: launch
66

77
launch: $(CONFIG_FILE)
8-
@echo "Creando instancias de VM con la configuración $(CONFIG_FILE)..."
8+
@echo "Creating VM instances with the configuration in $(CONFIG_FILE)..."
99
@bash $(LAUNCH_SCRIPT) $(CONFIG_FILE)
1010

1111
clean:
12-
@echo "Limpiando recursos en Google Cloud..."
13-
@bash $(CLEAN_CLOUD_SCRIPT)
12+
@echo "Clearing Google Cloud resources..."
13+
@bash $(CLEAN_SCRIPT) $(CONFIG_FILE)
1414

1515
reset: clean launch
1616

1717
help:
18-
@echo "Objetivos disponibles:"
19-
@echo " make launch [CONFIG_FILE=config.json] - Crea las instancias de VM según la configuración especificada (por defecto: config.json)."
20-
@echo " make clean - Limpia todas las instancias y buckets en Google Cloud."
21-
@echo " make reset - Limpia todos los recursos y luego crea nuevos."
22-
@echo " make help - Muestra esta ayuda."
18+
@echo "CLOUDY commands:"
19+
@echo " make launch [CONFIG_FILE=config.json] - Creates VM instances according to the specified configuration (default: config.json)."
20+
@echo " make clean [CONFIG_FILE=config.json] - Deletes all instances and buckets in Google Cloud (requires config.json)."
21+
@echo " make reset - Deletes all cloud resources and then creates new ones."
22+
@echo " make help - Displays this help."
23+

README.md

Lines changed: 34 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -7,42 +7,48 @@
77
[![Release](https://badgen.net/github/release/manjavacas/cloudy)]()
88
[![Contributors](https://badgen.net/github/contributors/manjavacas/cloudy)]()
99

10-
**CLOUDY** automatiza la ejecución de experimentos en máquinas virtuales de [Google Cloud](https://console.cloud.google.com). Crea instancias y *buckets*, instala dependencias, ejecuta *scripts* de Python y maneja la limpieza de recursos.
10+
**CLOUDY** automates the execution of experiments on [Google Cloud](https://console.cloud.google.com). It creates instances and buckets, installs dependencies, runs Python scripts, and handles resource cleanup.
1111

12-
## ⚙️ **¿Cómo funciona?**
12+
## ⚙️ **How it works?**
1313

14-
El flujo de trabajo de **CLOUDY** es el siguiente:
14+
The workflow of **CLOUDY** comprises the following steps:
1515

16-
1. El *script* `launch.sh` lanza un número determinado de instancias de máquinas virtuales, de acuerdo a las opciones especificadas en el fichero `config.json`.
16+
1. The script `launch.sh` launches a specified number of virtual machine instances, according to the options specified in the `config.json` file.
1717

18-
2. En cada máquina virtual se ejecuta el *script* `setup.sh`, que instala dependencias, descarga un repositorio y ejecuta un *script* especificado.
18+
2. The script `setup.sh` is executed in each VM to install dependencies, download the specified repository and run the Python script indicated.
1919

20-
3. La salida se guarda en un *bucket* existente, o lo crea en caso contrario.
20+
3. The output is saved to an existing bucket, or a new one is created as required.
2121

22-
4. Las máquinas se autoeliminan una vez finalizada su ejecución.
22+
4. The instances are automatically deleted once their execution has finished.
2323

2424
<p align="center">
25-
<img src="images/diagram.png" alt="DIAGRAMA" width=80% />
25+
<img src="images/diagram.png" alt="DIAGRAM" width=80% />
2626
</p>
2727

2828
## 📄 **Scripts**
2929

30-
Este proyecto consta de los siguientes *scripts*:
30+
This project consists of the following scripts:
3131

32-
- `launch.sh`: crea instancias de máquinas virtuales en Google Cloud según la configuración definida en `config.json`.
33-
- `setup.sh`: se ejecuta en cada instancia creada, instala dependencias, clona un repositorio, ejecuta un *script* de Python y guarda los resultados en un *bucket* de Google Cloud.
34-
- `clean_cloud.sh`: limpia todas las instancias de máquinas virtuales y *buckets* en Google Cloud.
35-
- `Makefile`: facilita la ejecución de los *scripts* mediante comandos simples.
32+
- `launch.sh`: creates VM instances on Google Cloud according to the configuration defined in `config.json`.
33+
- `setup.sh`: runs on each VM instance. Installs dependencies, clones a repository, runs a Python script, and saves the results to a Google Cloud bucket, creating it if necessary.
34+
- `clean.sh`: cleans up all virtual machine instances and buckets on Google Cloud.
35+
- `Makefile`: enables the execution of the scripts through simple commands.
3636

37-
## 💻 **¿Cómo utilizar CLOUDY?**
37+
## 💻 **How to use CLOUDY?**
3838

39-
0. **Prerrequisitos**.
39+
1. **Prerequisites**
4040

41-
Debes contar con una [cuenta de servicio en GCP](https://cloud.google.com/iam/docs/service-accounts-create?hl=es-419) con los permisos requeridos por *Compute engine* y *Cloud Storage* (ej. administrador de almacenamiento, administrador de instancias de compute).
41+
First, create a [service account on GCP](https://cloud.google.com/iam/docs/service-accounts-create?hl=en) with the required permissions for Compute Engine and Cloud Storage (e.g., *storage administrator*, *compute instances administrator*).
4242

43-
1. **Definir `config.json`.**
43+
Then, install the following dependencies:
4444

45-
Define tu configuración en el archivo `config.json`, ubicado en el directorio raíz del proyecto. Por ejemplo:
45+
- `Google Cloud SDK`: required to interact with Google Cloud from the command line.
46+
47+
- `jq`: used to read the JSON configuration file.
48+
49+
2. **Edit `config.json`**
50+
51+
Define your custom configuration in the `config.json` file, located in the root directory of the project. For example:
4652

4753
```json
4854
{
@@ -64,36 +70,29 @@ Este proyecto consta de los siguientes *scripts*:
6470
}
6571
```
6672

67-
Los principales campos a editar son:
68-
69-
- `INSTANCE_NAME_BASE` y `BUCKET_NAME`: identificador de las instancias y *bucket* creados.
70-
- `REPO_NAME` y `REPO_URL`: repositorio a clonar. En él se ubica el código que queremos ejecutar.
71-
- `DEPENDENCIES`: dependencias requeridas para ejecutar el *script*.
72-
- `SCRIPT_PATH` y `SCRIPT_ARGS`: ruta al *script* de Python que se encuentra en el repositorio y que queremos ejecutar, así como argumentos de entrada.
73-
- `SERVICE_ACCOUNT`: cuenta de servicio en GCP que vamos a utilizar. Debe contar con los permisos necesarios.
74-
75-
76-
2. **Instalar las dependencias necesarias.**
77-
78-
- `Google Cloud SDK`: necesario para interactuar con Google Cloud desde la línea de comandos.
73+
The main options to edit are:
7974

80-
- `jq`: empleado para leer el archivo JSON de configuración.
75+
- `INSTANCE_NAME_BASE` and `BUCKET_NAME`: identifiers for the created instances and bucket.
76+
- `REPO_NAME` and `REPO_URL`: repository to clone. This is where the code you want to execute is located.
77+
- `SCRIPT_PATH` and `SCRIPT_ARGS`: path to the Python script you want to execute in the repository, along with its input arguments.
78+
- `DEPENDENCIES`: dependencies required to run the Python script.
79+
- `SERVICE_ACCOUNT`: GCP service account to be used. It must have the necessary permissions.
8180

82-
3. **Ejecución.**
81+
3. **Run CLOUDY**
8382

84-
- Para crear las instancias de máquinas virtuales, ejecuta:
83+
- To launch the virtual machine instances, run:
8584

8685
```bash
8786
make launch
8887
```
8988

90-
- Para limpiar todas las instancias y *buckets*, ejecuta:
89+
- To clean up all instances and buckets, run:
9190

9291
```bash
9392
make clean
9493
```
9594

96-
- Para eliminar máquinas y *buckets* y volver a lanzar:
95+
- To delete machines and buckets and then relaunch, run:
9796

9897
```bash
9998
make reset

scripts/clean_cloud.sh renamed to scripts/clean.sh

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -14,32 +14,32 @@ CONFIG_FILE="config.json"
1414

1515
ZONE=$(jq -r '.ZONE' $CONFIG_FILE)
1616

17-
# Eliminar instancias de VM
17+
# Delete VM instances
1818
INSTANCES=$(gcloud compute instances list --format="value(name)" --filter="zone:($ZONE)" 2>/dev/null)
1919

2020
if [ -z "$INSTANCES" ]; then
21-
warn "No hay instancias de VM para eliminar."
21+
warn "No VM instances to delete."
2222
else
23-
info "Eliminando instancias de VM..."
23+
info "Deleting VM instances..."
2424
for INSTANCE in $INSTANCES; do
25-
info "Eliminando instancia: $INSTANCE..."
25+
info "Deleting instance: $INSTANCE..."
2626
gcloud compute instances delete "$INSTANCE" --zone="$ZONE" --quiet
2727
done
28-
info "Todas las instancias de VM han sido eliminadas."
28+
info "All VM instances have been deleted."
2929
fi
3030

31-
# Eliminar buckets
31+
# Delete buckets
3232
BUCKETS=$(gsutil ls)
3333

3434
if [ -z "$BUCKETS" ]; then
35-
warn "No hay buckets de GCS para eliminar."
35+
warn "No GCS buckets to delete."
3636
else
37-
info "Eliminando buckets de GCS..."
37+
info "Deleting GCS buckets..."
3838
for BUCKET in $BUCKETS; do
39-
info "Eliminando bucket: $BUCKET..."
39+
info "Deleting bucket: $BUCKET..."
4040
gsutil rm -r "$BUCKET"
4141
done
42-
info "Todos los buckets de GCS han sido eliminados."
42+
info "All GCS buckets have been deleted."
4343
fi
4444

45-
info "Proceso de eliminación completado."
45+
info "Deletion process completed."

scripts/launch.sh

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ SCRIPT_ARGS=$(jq -r '.SCRIPT_ARGS' $CONFIG_FILE)
2626
for ((i = 1; i <= N_VMS; i++)); do
2727
INSTANCE_NAME="$INSTANCE_NAME_BASE-$i"
2828

29-
info "Creando instancia: $INSTANCE_NAME..."
29+
info "Creating instance: $INSTANCE_NAME..."
3030
gcloud compute instances create "$INSTANCE_NAME" \
3131
--zone="$ZONE" \
3232
--machine-type="$MACHINE_TYPE" \
@@ -35,16 +35,16 @@ for ((i = 1; i <= N_VMS; i++)); do
3535
--service-account="$SERVICE_ACCOUNT" \
3636
--scopes https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/devstorage.full_control
3737

38-
while ! gcloud compute ssh "$INSTANCE_NAME" --zone="$ZONE" --command="info Instancia preparada" 2>/dev/null; do
39-
info "Esperando a que el servicio SSH esté disponible..."
38+
while ! gcloud compute ssh "$INSTANCE_NAME" --zone="$ZONE" --command="info Instance ready!" 2>/dev/null; do
39+
info "Waiting for SSH service to be available..."
4040
sleep 10
4141
done
4242

43-
info "Copiando el script de setup en la instancia..."
43+
info "Copying setup script to the VM instance..."
4444
gcloud compute scp scripts/$SETUP_SCRIPT "$INSTANCE_NAME:~/" --zone="$ZONE"
4545

46-
info "Ejecutando el script de setup en la instancia..."
46+
info "Running setup script on the VM instance..."
4747
gcloud compute ssh "$INSTANCE_NAME" --zone="$ZONE" --command="chmod +x ~/$SETUP_SCRIPT && ~/$SETUP_SCRIPT '$INSTANCE_NAME' '$BUCKET_NAME' '$BUCKET_ZONE' '$REPO_NAME' '$REPO_URL' '$SCRIPT_PATH' '$DEPENDENCIES' '$SCRIPT_ARGS' '$ZONE'"
4848

49-
info "Fin del proceso de creación de $INSTANCE_NAME."
49+
info "$INSTANCE_NAME finished!"
5050
done

scripts/setup.sh

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
#!/bin/bash
22

33
if [ "$#" -ne 9 ]; then
4-
error "Se deben proporcionar todos los argumentos necesarios."
4+
error "All required arguments must be provided."
55
exit 1
66
fi
77

@@ -17,68 +17,68 @@ ZONE="$9"
1717

1818
info() {
1919
local message="$1"
20-
echo -e "\033[32m[CLOUDY $MACHINE_NAME] $message\033[0m" # Verde
20+
echo -e "\033[32m[CLOUDY $MACHINE_NAME] $message\033[0m" # Green
2121
}
2222

2323
warn() {
2424
local message="$1"
25-
echo -e "\033[33m[CLOUDY $MACHINE_NAME] $message\033[0m" # Amarillo
25+
echo -e "\033[33m[CLOUDY $MACHINE_NAME] $message\033[0m" # Yellow
2626
}
2727

2828
error() {
2929
local message="$1"
30-
echo -e "\033[31m[CLOUDY $MACHINE_NAME] Error: $message\033[0m" # Rojo
30+
echo -e "\033[31m[CLOUDY $MACHINE_NAME] Error: $message\033[0m" # Red
3131
}
3232

3333
OUTPUT_FILE="output_${MACHINE_NAME}.txt"
3434

35-
info "Actualizando e instalando dependencias..."
35+
info "Updating and installing dependencies..."
3636

37-
# Redirigir stdout y stderr a /dev/null para ocultar la salida
3837
sudo apt-get update >/dev/null 2>&1
3938
sudo apt-get upgrade -y >/dev/null 2>&1
4039
sudo apt-get install -y python3 python3-pip >/dev/null 2>&1
4140

4241
pip3 install $DEPENDENCIES >/dev/null 2>&1
4342

44-
info "Clonando repositorio..."
43+
info "Cloning repository $REPO_NAME..."
4544

4645
git clone "$REPO_URL" >/dev/null 2>&1
4746
cd "$REPO_NAME"
4847

49-
info "Ejecutando script..."
48+
info "Running script: python3 $SCRIPT_PATH $SCRIPT_ARGS..."
5049

5150
python3 "$SCRIPT_PATH" $SCRIPT_ARGS >"$OUTPUT_FILE" 2>&1
5251

5352
if [ -f "$OUTPUT_FILE" ]; then
5453

5554
if ! command -v gsutil &>/dev/null; then
56-
info "Instalando Google Cloud SDK..."
55+
info "Installing Google Cloud SDK..."
5756
sudo apt-get install -y google-cloud-sdk >/dev/null 2>&1
5857
fi
5958

6059
if gsutil ls -b "gs://$BUCKET_NAME" &>/dev/null; then
61-
info "El bucket gs://$BUCKET_NAME existe."
60+
info "The bucket gs://$BUCKET_NAME already exists."
6261
else
63-
warn "El bucket gs://$BUCKET_NAME NO existe. Creando bucket..."
62+
warn "The bucket gs://$BUCKET_NAME does NOT exist. Creating bucket..."
6463
gsutil mb -l $BUCKET_ZONE "gs://$BUCKET_NAME/" >/dev/null 2>&1
6564
fi
6665

67-
info "Guardando $OUTPUT_FILE en el bucket gs://$BUCKET_NAME/..."
66+
info "Saving $OUTPUT_FILE to bucket gs://$BUCKET_NAME/..."
6867
gsutil cp "$OUTPUT_FILE" "gs://$BUCKET_NAME/" >/dev/null 2>&1
6968

7069
if [ $? -eq 0 ]; then
71-
info "Archivo guardado en gs://$BUCKET_NAME/$OUTPUT_FILE"
70+
info "File saved to gs://$BUCKET_NAME/$OUTPUT_FILE!"
7271
else
73-
error "No se pudo guardar el archivo en Google Cloud Storage."
72+
error "Failed to save the file in Google Cloud Storage."
7473
exit 1
7574
fi
7675

7776
else
78-
error "No se encontró el archivo de salida $OUTPUT_FILE"
77+
error "Output file $OUTPUT_FILE not found!"
7978
exit 1
8079
fi
8180

82-
info "Eliminando instancia..."
83-
gcloud compute instances delete "$MACHINE_NAME" --zone="$ZONE" --quiet 2>&1
84-
81+
info "Deleting instance..."
82+
if ! gcloud compute instances delete "$MACHINE_NAME" --zone="$ZONE" --quiet >/dev/null 2>&1; then
83+
echo "Failed to delete instance $MACHINE_NAME."
84+
fi

0 commit comments

Comments
 (0)