Skip to content

README.md

Latest commit

 

History

History
79 lines (57 loc) · 2.85 KB

README.md

File metadata and controls

79 lines (57 loc) · 2.85 KB
Raw

RHOAI Deployments

Automation and documentation for installing Red Hat OpenShift AI (RHOAI) on ROSA HCP clusters and deploying models using KServe.

Quick Start

# Install RHOAI on your cluster
./scripts/install-rhoai.sh

# Deploy a model
./scripts/deploy-model.sh scripts/models/granite-3-3-8b-instruct.conf

Repository Structure

scripts/
  install-rhoai.sh          # Automated RHOAI installation
  deploy-model.sh           # Model deployment (GPU/CPU, PVC/ModelCar)
  models/                   # Ready-to-use model configuration files
docs/
  RHOAI-ROSA-Installation-Guide.md
  deploy-granite-3-3-2b-instruct-cpu.md
  deploy-granite-embedding-modelcar.md

Scripts

install-rhoai.sh

Automates the full RHOAI installation: pull secret creation, Kyverno setup for secret distribution, CatalogSource, operator installation, and DataScienceCluster creation.

# Minimal install (Dashboard + KServe + Workbenches only)
./scripts/install-rhoai.sh --minimal --wait

# Specific RHOAI version
./scripts/install-rhoai.sh --catalog-image quay.io/rhoai/rhoai-fbc-fragment:rhoai-3.5

deploy-model.sh

Deploys a model via KServe InferenceService + vLLM. Supports PVC-based storage (downloads from HuggingFace) and ModelCar (OCI image). Automatically detects GPU availability and can fall back to an alternative config.

# Deploy with GPU fallback to CPU
./scripts/deploy-model.sh scripts/models/granite-3-3-8b-instruct.conf scripts/models/granite-3-3-2b-instruct-cpu.conf

# Undeploy
./scripts/deploy-model.sh -d scripts/models/granite-3-3-8b-instruct.conf

See scripts/README.md for the full reference (all flags, environment variables, config file format, and examples).

Available Model Configs

Config Model Mode GPU
granite-3-3-8b-instruct.conf Granite 3.3 8B Instruct PVC Yes
granite-3-3-2b-instruct-cpu.conf Granite 3.3 2B Instruct PVC No
granite-3-3-2b-instruct-cpu-minimal.conf Granite 3.3 2B Instruct (minimal) PVC No
granite-embedding.conf Granite Embedding English R2 PVC Yes
granite-embedding-cpu.conf Granite Embedding English R2 PVC No
granite-embedding-modelcar.conf Granite Embedding English R2 ModelCar No
llama-3-3-70b.conf Llama 3.3 70B PVC Yes
llama-3-1-8b-modelcar.conf Llama 3.1 8B ModelCar No
llama-3-2-1b-cpu.conf Llama 3.2 1B PVC No
opt-125m-cpu.conf OPT 125M PVC No

Documentation

Step-by-step guides covering manual installation and deployment procedures are available in the docs/ directory.

Prerequisites

  • ROSA HCP cluster with cluster-admin access
  • oc and jq installed
  • Docker config with registry credentials at ~/.docker/config.json