Scalingo Operator v1.0.0-alpha1

The Scalingo Operator can deploy and undeploy PostgreSQL instances hosted on dedicated resources on Scalingo platform from a Kubernetes cluster.

Definitions

Few helpful definitions to start with.

CRD (Custom Resource Definition)

A CRD is a Kubernetes object used to define a new custom resource type in the Kubernetes API. It tells the API server: “Here is a new kind (e.g., MyApp) and its schema.”

Example: config/crd/bases/databases.scalingo.com_postgresqls.yaml

CR (Custom Resource)

A CR is an actual instance of the type defined by the CRD.

Example: config/samples/databases_v1alpha1_postgresql.yaml

Usage

As a pre-requisite, first store your Scalingo API token in your cluster secrets. Then, execute the Kubernetes operator on your cluster and deploy the expected database resource.

Create Secret

# create secret
SCALINGO_TOKEN="tk-my_token"
kubectl create secret generic scalingo \
    --from-literal=api_token="$SCALINGO_TOKEN"

# verification
kubectl describe secret scalingo

Deploy the Operator

Gather and save the latest installer from the Scalingo Operator Github releases: https://github.com/Scalingo/scalingo-operator/releases

The installer file is install.yaml.

Then deploy the Operator controller on your Kubernetes cluster:

# install operator controller manager
kubectl apply --filename install.yaml

# verification
kubectl get all --namespace scalingo-operator-system

# follow operator logs
kubectl logs deploy/scalingo-operator-controller-manager --namespace scalingo-operator-system --follow

Deploy Database Resource

Once the operator is deployed and running, deploy the database resource using its descriptor.

Using PostgreSQL sample example:

kubectl apply --filename config/samples/databases_v1alpha1_postgresql.yaml

Read Database URL

At deployment, the database secret is written in Kubernetes secrets. The secret name and key are defined in Custom Resource connInfoSecretTarget fields.

To read the secret:

$NAME=my-postgresql-secret  # read from CR spec.connInfoSecretTarget.name
$PREFIX=PG                  # read from CR spec.connInfoSecretTarget.prefix

# list all secrets in all namespaces
kubectl get secrets -A

# secret details
kubectl describe secret $NAME

# get secret field
kubectl get secret $NAME -o jsonpath='{.data}' | grep $PREFIX

# gather the returned base64 content and decode, example:
echo "ZGJfY29ubmVjdGlvbl9zdHJpbmc=" | base64 --decode

Undeploy Database

Use almost the same command than deploy, with the same descriptor file: replace apply by delete.

Using PostgreSQL sample example:

kubectl delete --filename config/samples/databases_v1alpha1_postgresql.yaml

Undeploy the Operator

Not necessary, if Operator undeploy is needed execute the opposite deploy commands.

# Replace my-namespace and my-operator using your own names
kubectl delete deployment my-operator --namespace scalingo-operator-system
kubectl delete namespace scalingo-operator-system
kubectl delete --kustomize config/crd

# Verifications
kubectl get deploy,pods --namespace scalingo-operator-system
kubectl get namespaces
kubectl get crds

Architecture

This application follows a (slightly adapted version of) the clean architecture pattern.

Layers

• Boundaries > Out: output calls to go-scalingo API
• Controller (> In): handles input Kubernetes requests
• Usecases: business rules
• Domain: business entities

Note

Controller is generated by Kubebuilder, ready to handle Kubernetes requests. For this reason it is kept as is, ensuring the Controllers plus the Boundaries > In layers.

Development

Install Environment

sudo snap install microk8s --classic
sudo snap install kubectl --classic

microk8s.kubectl config view --raw > $HOME/.kube/microk8s.config

# then add in ~/.zshrc
export  KUBECONFIG=$HOME/.kube/config
export  KUBECONFIG=$KUBECONFIG:$HOME/.kube/microk8s.config

# verification: both commands must return the same informations
microk8s.kubectl config view
kubectl config view

# ensure some microk8s modules are enabled
microk8s enable dns
microk8s enable rbac
microk8s status

Download Kubebuilder and Install Locally

curl -L -o kubebuilder "https://go.kubebuilder.io/dl/latest/$(go env GOOS)/$(go env GOARCH)"
chmod +x kubebuilder && sudo mv kubebuilder /usr/local/bin/

Kubebuilder Commands

Operator Creation Example

These commands were executed to create this operator:

kubebuilder init --domain scalingo.com --repo github.com/Scalingo/scalingo-operator
kubebuilder create api --group databases --version v1alpha1 --kind PostgreSQL
make manifests
make install

Build Commands

# generate api/v1alpha/zz_generated.deepcopy.go
make generate

# generate the CRD manifests under config/crd/bases and a sample for it under config/samples
make manifests

Execute Command

# deploy the CRD on local cluster
make install

# verify using
kubectl get crd

# execute
make run

Build and Deploy Image

VERSION="1.0.0-alpha1"

# build and push your image to the location specified by IMG
make docker-build docker-push IMG=scalingo/scalingo-operator:v$VERSION

# deploy the controller to the cluster with image specified by IMG
make deploy IMG=scalingo/scalingo-operator:v$VERSION

List make Targets

To list the make targets with their description:

make help

Tips and Tricks

Force-delete the CRD

In case the kubernetes delete CRD hangs indefinetly, as with such command:

kubectl delete crd postgresqls.databases.scalingo.com

to force the deletion, remove the Finalizers manually:

# example
$KIND=PostgreSQL
$NAME=postgresql-sample
$CRD=postgresqls.databases.scalingo.com

# edit the current CRD descriptor, and remove this block:
#
#   metadata:
#       finalizers: []
#
kubectl edit $KIND $NAME

# then, retry the delete
kubectl delete crd $CRD

Force-pull the Operator image

Set the Manager parameter imagePullPolicy to Always (default value being IfNotPresent).

In config/manager/manager.yaml, add this line:

imagePullPolicy: Always

bellow:

image: controller:latest

then, execute make deploy IMG=....

Useful Links

• Kubebuilder book
• MicroK8s
• Kubernetes operator
• Kubernetes Custom Resource Definition

Release a New Version

Warning

You first need to execute the integration tests on main to ensure that everything is working.

Bump new version number in:

• CHANGELOG.md
• README.md: all VERSION contents.

Commit, tag and create a new release:

VERSION="1.0.0-alpha1"

git switch --create release/${VERSION}
git add CHANGELOG.md README.md
git commit --message="feat: bump v${VERSION}"
git push --set-upstream origin release/${VERSION}
gh pr create --reviewer=scalingo/team-ist --fill-first

Once the pull request merged, you can tag the main branch:

git tag v${VERSION}
git push origin main v${VERSION}

After tagging the branch, the process to release the archive with the installer is automatically handled by GitHub Actions. The release will be published on the Scalingo Operator page.

Integration Tests

Integration tests are minimal and remain in progress.

Execute with:

go test -tag integration ./...