code-predict-cloudfunction

Cloud Function that wraps the PaLM Code Bison Models

Author(s)	Romin Irani

This application demonstrates a Cloud Function written in Python that initializes the Vertex AI module and then provides an endpoint to invoke PaLM Code Bison model.

NOTE: Before you move forward, ensure that you have followed the instructions in SETUP.md. Additionally, ensure that you have cloned this repository and are currently in the code-predict-cloudfunction folder. This should be your active working directory for the rest of the commands.

Environment variables required

Your Cloud Function requires access to two environment variables:

• GCP_PROJECT : This the Google Cloud Project Id.
• GCP_REGION : This is the region in which you are deploying your Cloud Function. For e.g. us-central1.

These variables are needed since the Vertex AI initialization needs the Google Cloud Project Id and the region. The specific code line from the main.py function is shown here: vertexai.init(project=PROJECT_ID, location=LOCATION)

In Cloud Shell, execute the following commands:

export GCP_PROJECT='<Your Google Cloud Project Id>'  # Change this
export GCP_REGION='us-central1'             # If you change this, make sure region is supported by Model Garden. When in doubt, keep this.

These variables can be set via the following instructions via any of the following ways:

1. At the time of deploying the Google Cloud Function. We will be using this method in the next section when we deploy the Cloud Function.
2. Updating the environment variables after deploying the Google Cloud Function.

Deploying the Cloud Function

Assuming that you have a copy of this project on your local machine with gcloud SDK setup on the machine, follow these steps:

1. Go to the root folder of this project.

2. You should have both the main.py and requirements.txt file present in this folder.

3. Provide the following command:

   gcloud functions deploy predictCode \
   --gen2 \
   --runtime=python311 \
   --region=$GCP_REGION \
   --source=. \
   --entry-point=predictCode \
   --trigger-http \
   --set-env-vars=GCP_PROJECT=$GCP_PROJECT,GCP_REGION=$GCP_REGION \
   --allow-unauthenticated

Invoking the Cloud Function

Since this Cloud Function is deployed with a HTTP trigger, you can directly invoke it. Sample calls are shown below:

curl -m 70 -X POST https://$GCP_REGION-$GCP_PROJECT.cloudfunctions.net/predictCode \
-H "Content-Type: application/json" \
-d '{
  "prompt": "Write a Python function to make a call to a URL?"
}'

If you'd like to see a better formatted version of the code, try the following invocation that formats the response using new lines:

curl -m 70 -X POST https://$GCP_REGION-$GCP_PROJECT.cloudfunctions.net/predictCode \
-H "Content-Type: application/json" \
-d '{
"prompt": "Write a Python function to make a call to a URL?"
}'  | sed -e 's/\\n/\n/g'