Skip to content

openshift-snc.md

Latest commit

 

History

History
136 lines (98 loc) · 7.25 KB

openshift-snc.md

File metadata and controls

136 lines (98 loc) · 7.25 KB
Raw

Overview

This feature of mapt allows to deploy a single node OpenShift cluster on the cloud, using bundles from the SNC project.

Providers and Platforms

Currently it allows to create the single node cluster on the AWS provider.

Prerequisite

To be able to create the single node cluster on AWS, an AMI needs to be generated from the SNC bundle, this can be done with the help of cloud-importer

To create and publish an AMI for the 4.19.0 OpenShift bundle, use the following command:

% cloud-importer openshift-local aws \
    --arch x86_64 \
    --bundle-url https://developers.redhat.com/content-gateway/file/pub/openshift-v4/clients/crc/bundles/openshift/4.19.0/crc_libvirt_4.19.0_amd64.crcbundle \
    --shasum-url https://developers.redhat.com/content-gateway/file/pub/openshift-v4/clients/crc/bundles/openshift/4.19.0/sha256sum.txt \
    --backed-url file:///Users/tester/workspace \
    --output /tmp/snc

Note

  • cloud-importer might fail to upload the disk image from the bundle to S3 in case the network is slow, it is often helpful to run cloud-importer in an ec2 instance instead of the local dev machine
  • if bundle is already downloaded then use file:///<bundle_path> with --bundle-url and file:///<shasum_path> with --shasum-url

Operations

After the AMI is published and accessible by the account, we can use the following mapt command to create an OpenShift cluster using spot instances from AWS.

% podman run -d --rm \
    -v ${PWD}:/workspace:z \
    -e AWS_ACCESS_KEY_ID=XXX \
    -e AWS_SECRET_ACCESS_KEY=XXX \
    -e AWS_DEFAULT_REGION=us-east-1 \
    quay.io/redhat-developer/mapt:v1.0.0-dev mapt aws openshift-snc create \
        --spot \
        --version 4.19.0 \
        --project-name mapt-snc \
        --backed-url file:///home/tester/workspace \
        --conn-details-output /tmp/snc \
        --pull-secret-file /home/tester/Downloads/pull-secret

After the above command succeeds the kubeconfig to access the deployed cluster will be available in /tmp/snc/kubeconfig

Profiles

Profiles are optional addons that are installed on the SNC cluster after it is ready. Use the --profile flag to enable one or more profiles:

mapt aws openshift-snc create \
    --spot \
    --version 4.21.0 \
    --project-name mapt-snc \
    --backed-url file:///home/tester/workspace \
    --conn-details-output /tmp/snc \
    --pull-secret-file /home/tester/Downloads/pull-secret \
    --profile virtualization

Multiple profiles can be specified as a comma-separated list (e.g., --profile virtualization,ai).

Available profiles

Profile Description
virtualization Installs OpenShift Virtualization (CNV) on the cluster, enabling virtual machines to run on the single-node cluster. When this profile is selected, nested virtualization is automatically enabled on the cloud instance. Because standard Nitro-based instances do not expose /dev/kvm, a bare metal instance is required.
serverless-serving Installs OpenShift Serverless and creates a KnativeServing instance, enabling serverless workloads (Knative Serving) on the cluster.
serverless-eventing Installs OpenShift Serverless and creates a KnativeEventing instance, enabling event-driven workloads (Knative Eventing) on the cluster.
serverless Installs OpenShift Serverless and creates both KnativeServing and KnativeEventing instances.
servicemesh Installs OpenShift Service Mesh 3 (Sail/Istio) on the cluster, deploying IstioCNI and an Istio control plane.
ai Installs Red Hat OpenShift AI (RHOAI) on the cluster. Automatically installs Service Mesh v2 (Maistra) and Serverless Serving as prerequisites for Kserve. All three operators install in parallel; the DataScienceCluster CR is only created once Service Mesh and Serverless are fully ready. The minimum instance size is raised to 16 vCPUs (from the default 8) to accommodate the additional operators. Cannot be combined with the servicemesh profile (which deploys Service Mesh v3/Sail).
nvidia Installs the NVIDIA GPU Operator on the cluster. Automatically installs Node Feature Discovery (NFD) as a prerequisite and creates a ClusterPolicy with the recommended OpenShift defaults (CRI-O runtime, OCP driver toolkit). The cluster must run on a GPU-capable instance type (e.g. g4dn, g5, p4d).

Operator overrides

Profiles install operators using the default OLM channel (stable) and catalog (redhat-operators). Two flags allow overriding these per operator, which is useful for testing pre-release operator builds:

--operator-channel

Override the OLM subscription channel for a specific operator:

mapt aws openshift-snc create \
    --profile serverless-serving \
    --operator-channel serverless-operator=candidate

Multiple operators can be overridden at once:

--operator-channel serverless-operator=preview,nfd=4.17

--catalog-source

Use a custom index image instead of the default catalog. This creates a CatalogSource CR in openshift-marketplace and points the operator's subscription to it:

mapt aws openshift-snc create \
    --profile nvidia \
    --catalog-source gpu-operator-certified=quay.io/my-team/gpu-operator-index:test-v1.0

Both flags can be combined:

mapt aws openshift-snc create \
    --profile ai \
    --operator-channel serverless-operator=candidate \
    --catalog-source rhods-operator=quay.io/my-team/rhoai-index:nightly

When neither flag is provided, operators use the defaults: channel stable and catalog redhat-operators (unless overridden in the profile definition, e.g. gpu-operator-certified and nfd use certified-operators).

The keys are operator package names as they appear in OLM. The operators installed by each profile are:

Profile Operator package names
serverless-serving / serverless-eventing / serverless serverless-operator
servicemesh servicemeshoperator3
ai rhods-operator, servicemeshoperator, authorino-operator, serverless-operator
nvidia gpu-operator-certified, nfd

Adding new profiles

To add a new profile:

  1. Create <name>.go under pkg/target/service/snc/profile/ — Go file with a deploy<Name>() function that uses the Pulumi Kubernetes provider to create the required resources (Namespace, OperatorGroup, Subscription, CRs, etc.)
  2. Register the profile in profile.go by adding it to the profileRegistry map