Skip to content

README.md

Latest commit

 

History

History
102 lines (77 loc) · 5.69 KB

README.md

File metadata and controls

102 lines (77 loc) · 5.69 KB

Advanced Solutions Lab

Overview

This repository contains AI and Machine Learning contents meant to be run on Google Cloud. This is maintained by Google Cloud’s Advanced Solutions Lab (ASL) team.

This repository contains 3 main modules to covers various AI/ML toipcs:

  • asl_core: A wide range of model architectures (DNN, CNN, RNN, transformers, SNGP, etc.) targeting many data modalities (tabular, image, text, time-series) implemented mainly in Tensorflow and Keras.
  • asl_mlops: Tools on Google Cloud’s Vertex AI for operationalizing Tensorflow, Scikit-learn and PyTorch models at scale (e.g. Vertex training, tuning, and serving, TFX and Kubeflow pipelines).
  • asl_genai: Generative AI and Agent System using Gemini and Agentic Frameworks like Google ADK.

Repository Structure

Each module (asl_core, asl_mlops, asl_genai) has separate environment and materials, which are organized in each directory.

All learning materials are in the contets folder. This folder is organized by different topics. Each folder contains a labs and a solutions folder. Use the labs notebooks to test your coding skills by filling in TODOs and refer to the notebooks in the solutions folder to verify your code.

We have three main folders described below:

├── asl_core
│   ├── notebooks - contains learning materials organized by topic
│   │   ├── building_production_ml_systems
│   │   │   ├── labs
│   │   │   └── solutions
│   │   ├── end-to-end-structured
│   │   ├── image_models
│   │   ├── ...
│   ├── kernels - contains kernel scripts needed for certain notebooks
│   ├── scaffolds - contains sample code to accelerate AI/ML projects
│   ├── requirements.txt - dependencies for this module
├── asl_mlops
│   ├── ...
├── asl_genai
│   ├── ...
├── ...

Environment Setup

Step 1. Run the Setup Script on Cloud Shell

This repository is tested on Vertex AI Workbench and Cloud Workstations. To begin, run the setup script in Cloud Shell to configure essential project infrastructure (APIs, IAM, Buckets).

Run the setup script in Cloud Shell to provision your environment.

git clone https://github.com/GoogleCloudPlatform/asl-ml-immersion.git
cd asl-ml-immersion
bash scripts/setup_env.sh

You will be prompted to select the environment to set up:

  • 1) Vertex AI Workbench: Setup Vertex AI Workbench.
  • 2) Cloud Workstations: Setup Cloud Workstations.
  • 3) Setup both: Setup both environments.
  • 4) Skip: Setup project infrastructure (APIs, IAM, Buckets) only.

By selecting the option 1-3, you can automatically setup the environment, or you can select 4 and manually set up the environment following the official documentation:

Next, you will be asked if you want to attach a GPU (Nvidia T4) to the environment. Select y or n depending on your preference.

Note: Accelerators (GPU/TPU) are not required in most of the notebooks, but some notebooks recommend using them.

Step 2. Build the Environemnt

Once your environment is running, open it. Then, run the commands below in Terminal inside the environment to clone this repository, and build the environemnt (venvs and jupyter kernels).

git clone https://github.com/GoogleCloudPlatform/asl-ml-immersion.git
cd asl-ml-immersion
make

On Cloud Workstations, click Open Folder -> asl-ml-immersion to open the repository window. If the folder is already opend, Command + Shift + P and type Developer: Reload Window to reflect the changes.

Using the Environment

Running a notebook

After the setup above, you can open a Jupyter notebook file, and execute on a module kernel (ASL Core, ASL MLOps, or ASL Agent).
If a correct kernel is not pre-selected, click Select Kernel and select a correct one.

On Cloud Workstations, you can find a kernel under Select Kernel -> Jupyter Kernels.
If you can't find Jupyter Kernels, click Python Environment -> <- (Left Arrow) to reload the environment.

Note: Some notebooks might require additional setup, please refer to the instructions in specific notebooks.

Running a command on Terminal

When running a command from the terminal, make sure to activate a venv for a specific environment.

E.g. (under asl-ml-immersion directory),

source ./asl_genai/.venv/bin/activate
adk web ./asl_genai/notebooks/vertex_genai/solutions/adk_agents

Contributions

Currently, only Googlers can contribute to this repo. See CONTRIBUTING.md for more details on the contribution workflow.

Disclaimer

This is not an officially supported Google product. Usage of Google Cloud products will incur charges. Learn more about pricing here.

Licensing

All the code in this repo is licensed under the Apache License, Version 2.0 (the "License"). You may obtain a copy of the License here.

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License