Errors when running in google cloud

[manimejia]

Has anybody found a way to run this in google cloud?
Here's what I have found:

---

The Provided Docker Image Failed to Start

Uploaded the provided docker image to gcloud

docker pull docker.io/jertel/vuegraf:latest
docker tag docker.io/jertel/vuegraf:latest \us-central1-docker.pkg.dev/myapp/myrepo/vuegraf
docker push us-central1-docker.pkg.dev/myapp/myrepo/vuegraf

Running as a gcloud job results in an exec format error

severity: "ERROR"
textPayload: "terminated: Application failed to start: invalid status ::14: could not start container: exec format error"

---

Gcloud built Image Started but Failed to Load Modules

Built a Python container using gcloud

echo "web: python3 src/vuegraf/vuegraf.py" > Procfile
gcloud builds submit --pack image=us-central1-docker.pkg.dev/myapp/myrepo/vuegraf

Running the same job results in ModuleNotFoundError

severity: "ERROR"
textPayload: "Traceback (most recent call last):
File "/workspace/src/vuegraf/vuegraf.py", line 12, in
import influxdb
ModuleNotFoundError: No module named 'influxdb'"

[jertel]

Note that my first reply accidentally mentioned "elastalert2" - that was a mistake, I meant Vuegraf. I manage multiple projects and still had the other name in my head as I was writing this response. I've updated it below with the correct project name. The response is still correct.

Vuegraf doesn't seem appropriate for a cloud "job", but I'll ignore that and assume you have a specific need to run a short-lived Vuegraf process.

The exec format error seems like you may be trying to run the wrong container image architecture on the chosen cloud infrastructure. The container image that this project uploads to Docker Hub contains both amd64 and arm64 architectures. Perhaps there's a mismatch occurring with your cloud usage.

[manimejia]

Thanks for the reply. I am rather noob with both python and containerized cloud services. So I appreciate your feedback and guidance as I try to get this script to run in a cloud environment, transferring data every minute (or so) from Emporia to InfluxDB.

I also came to the conclusion that the exec format error probably has something to do with the architecture of the pre-built container image. This is why I tried to build the same code using google's cloud build services. What baffles me is that BOTH builds seem to fail for different reasons. (The cloud built version seems to not have compiled with dependancies? How do I fix that?) Without expertise in python or cloud computing, I am really struggling to move forward in my project.

Thank you,
M.M.

P.S.
As I understand it, google cloud jobs are meant to execute a single service that will run continually (without having to be called periodically) until it is manually stopped. This seems to be what I want to execute a data transfer script that runs continually. Am I mistaken? If "cloud run" is suited better than "cloud jobs" for vuegraf, then I'd like to know more...

Unlike a Cloud Run service, which listens for and serves requests, a Cloud Run job only runs its tasks and exits when finished. A job does not listen for or serve requests, and cannot accept arbitrary parameters at execution.

[manimejia]

I'm gonna play around with adding a cloudbuild.yaml file, as documented for building a python container in google cloud, and report back.

[jertel]

I think a Kubernetes container deployment is closer to what you need.

A cloud job is meant to process a queue of tasks, and for each task a container is spun up. These are intended to be short-lived processes and I believe GCP limits the duration of the container to < 1h.