Skip to content

Latest commit

 

History

History
245 lines (194 loc) · 9.98 KB

build-all.md

File metadata and controls

245 lines (194 loc) · 9.98 KB

Building and deploying Mayastor Helm Chart from scratch

This guide will walk you through the process of building all Mayastor images using Nix and Docker. Once these are ready we can then install the helm chart using our freshly baked images.

Mayastor is a multi-component Rust project that makes heavy use of Nix for our development and build process.

If you're coming from a non-Rust (or non-Nix) background, building Mayastor may be a bit different than you're used to. There is no Makefile, you won't need a build toolchain, you won't need to worry about cross compiler toolchains, and all builds are reproducible.

Table of Contents

Prerequisites

Mayastor only builds on modern Linuxes. We'd adore contributions to add support for Windows, FreeBSD, OpenWRT, or other server platforms.

If you do not have a Linux system:

  • Windows: We recommend using [WSL2][windows-wsl2] if you only need to build Mayastor. You'll need a [Hyper-V VM][windows-hyperv] if you want to use it.
  • Mac: We recommend you use [Docker for Mac][docker-install] and follow the Docker process described. Please let us know if you find a way to run it!
  • FreeBSD: We think this might actually work, SPDK is compatible! But, we haven't tried it yet.
  • Others: This is kind of a "Do-it-yourself" situation. Sorry, we can't be more help!

Build system

The only thing your system needs to build Mayastor is Nix.

Usually Nix can be installed via (Do not use sudo!):

curl -L https://nixos.org/nix/install | sh

Source Code

Mayastor is split across different GitHub repositories under the OpenEBS organization.

Here's a breakdown of the required repos for the task at hand:

NOTE: There are also a few other repositories which are pulled or submoduled by the repositories above

NOTE: Some say we should have remained as a mono-repo :)

In order to build all images, you may check out all repos, example:

mkdir ~/mayastor && cd ~/mayastor
git clone --recurse-submodules https://github.com/openebs/mayastor.git -- io-engine
git clone --recurse-submodules https://github.com/openebs/mayastor-control-plane.git -- controller
git clone --recurse-submodules https://github.com/openebs/mayastor-extensions.git -- extensions

Building and Pushing

Each of the repos contains a script for building and pushing all their respective container images. Usually this is located at ./scripts/release.sh The api for this script is generally the same as it leverages a common base script.

> ./scripts/release.sh --help
Usage: release.sh [OPTIONS]

  -d, --dry-run              Output actions that would be taken, but don't run them.
  -h, --help                 Display this text.
  --registry <host[:port]>   Push the built images to the provided registry.
                             To also replace the image org provide the full repository path, example: docker.io/org
  --debug                    Build debug version of images where possible.
  --skip-build               Don't perform nix-build.
  --skip-publish             Don't publish built images.
  --image           <image>  Specify what image to build and/or upload.
  --tar                      Decompress and load images as tar rather than tar.gz.
  --skip-images              Don't build nor upload any images.
  --alias-tag       <tag>    Explicit alias for short commit hash tag.
  --tag             <tag>    Explicit tag (overrides the git tag).
  --incremental              Builds components in two stages allowing for faster rebuilds during development.
  --build-bins               Builds all the static binaries.
  --no-static-linking        Don't build the binaries with static linking.
  --build-bin                Specify which binary to build.
  --skip-bins                Don't build the static binaries.
  --build-binary-out <path>  Specify the outlink path for the binaries (otherwise it's the current directory).
  --skopeo-copy              Don't load containers into host, simply copy them to registry with skopeo.
  --skip-cargo-deps          Don't prefetch the cargo build dependencies.

Environment Variables:
  RUSTFLAGS                  Set Rust compiler options when building binaries.

Examples:
  release.sh --registry 127.0.0.1:5000

Building

In order to build all images, we simply need to walk the repos.

If you want to see what happens under the hood, without building, you can use the --dry-run.

cd ~/mayastor
for repo in */; do
  $repo/scripts/release.sh --dry-run --alias-tag my-tag
done

Here's a snippet of what you'd actually see:

~/mayastor/controller ~/mayastor
nix-build --argstr img_tag my-tag --no-out-link -A control-plane.project-builder.cargoDeps
Cargo vendored dependencies pre-fetched after 1 attempt(s)
Building openebs/mayastor-agent-core:my-tag ...
nix-build --argstr img_tag my-tag --out-link agents.core-image -A images.release.agents.core --arg allInOne true --arg incremental false --argstr product_prefix  --argstr rustFlags
docker load -i agents.core-image
rm agents.core-image
Building openebs/mayastor-agent-ha-node:my-tag ...
nix-build --argstr img_tag my-tag --out-link agents.ha.node-image -A images.release.agents.ha.node --arg allInOne true --arg incremental false --argstr product_prefix  --argstr rustFlags
docker load -i agents.ha.node-image
rm agents.ha.node-image
Building openebs/mayastor-agent-ha-cluster:my-tag ...
nix-build --argstr img_tag my-tag --out-link agents.ha.cluster-image -A images.release.agents.ha.cluster --arg allInOne true --arg incremental false --argstr product_prefix  --argstr rustFlags
docker load -i agents.ha.cluster-image

If you want to build, but not push it anywhere, you can skip the publishing with --skip-publish.

NOTE: This will also build the static kubectl-mayastor plugin. You can avoid that with --skip-bins.

cd ~/mayastor
for repo in */; do
  $repo/scripts/release.sh --skip-publish --alias-tag my-tag
done

After some time (14 minutes on my system), all images should be built. The output is too long, but since the script loads the images locally, you can see them:

> docker image ls | grep my-tag
openebs/mayastor-fio-spdk                     my-tag    e577a4ae779b   49 minutes ago      142MB
openebs/mayastor-casperf                      my-tag    67b1997768fb   49 minutes ago      149MB
openebs/mayastor-io-engine                    my-tag    62c58727a721   54 minutes ago      315MB
openebs/mayastor-upgrade-job                  my-tag    4cb14b214fb4   54 minutes ago      154MB
openebs/mayastor-obs-callhome-stats           my-tag    d4f78e2c57fd   54 minutes ago      71.3MB
openebs/mayastor-obs-callhome                 my-tag    d44b0444883f   54 minutes ago      378MB
openebs/mayastor-metrics-exporter-io-engine   my-tag    416c1cfd3a64   56 minutes ago      60MB
openebs/mayastor-csi-node                     my-tag    50beac3984ee   58 minutes ago      408MB
openebs/mayastor-csi-controller               my-tag    4186d4ea8fbe   59 minutes ago      68.8MB
openebs/mayastor-api-rest                     my-tag    9bccb64557e0   59 minutes ago      70.4MB
openebs/mayastor-operator-diskpool            my-tag    d6f19ec945e0   59 minutes ago      67.5MB
openebs/mayastor-agent-ha-cluster             my-tag    146ed2c49a78   59 minutes ago      66.1MB
openebs/mayastor-agent-ha-node                my-tag    0a3f9b375ebd   59 minutes ago      103MB
openebs/mayastor-agent-core                   my-tag    938f7b481b2e   About an hour ago   87.6MB

Pushing

You can push the images to your required registry/namespace using the argument --registry.
For the purposes of this, we'll push my docker.io namespace: docker.io/tiagolobocastro.

cd ~/mayastor
for repo in */; do
  $repo/scripts/release.sh --registry docker.io/tiagolobocastro --alias-tag my-tag
done

NOTE: If you don't specify the namespace, the default openebs namespace is kept.

Installing

Installing the helm chart with the custom images is quite simple.

NOTE: One last step is required, mostly due to a bug or unexpected behaviour with the helm chart.
We'll need to manually push this container image:

docker pull docker.io/openebs/alpine-sh:4.1.0
docker tag docker.io/openebs/alpine-sh:4.1.0 docker.io/tiagolobocastro/alpine-sh:4.1.0
docker push docker.io/tiagolobocastro/alpine-sh:4.1.0
> helm install mayastor mayastor/mayastor -n mayastor --create-namespace --set="image.repo=tiagolobocastro,image.tag=my-tag" --wait
NAME: mayastor
LAST DEPLOYED: Fri Dec  6 15:42:16 2024
NAMESPACE: mayastor
STATUS: deployed
REVISION: 1
NOTES:
OpenEBS Mayastor has been installed. Check its status by running:
$ kubectl get pods -n mayastor

For more information or to view the documentation, visit our website at https://openebs.io/docs/