Mayastor: No Way to Disable Automatic Image-based Disk Pool

[AJMansfield]

Summary

When installing mayastor, the install script creates image-backed DiskPools, even when doing so makes no sense (e.g. --default-pool-size 0) and with no combination of arguments that can disable this behavior entirely.

What Should Happen Instead?

When running microk8s enable mayastor --default-pool-size 0, no DiskPool objects should be created during the install, and no /data/microk8s.img file should be created on the system.

Reproduction Steps

1. Set up a new cluster.
2. Run microk8s enable mayastor --default-pool-size 0
3. Wait for the mayastor diskpool operator deployment to finish.
4. Run microk8s.kubectl get diskpools -n mayastor and observe that there are a set of diskpools named like microk8s-<hostname>-pool with 0 capacity and 0 availability, stalled in the Creating state until eventually reaching the Error state after reaching the 200 retry limit.

Can you suggest a fix?

At present, it's not enough to simply change the python enable script: Even if the conditional checking for that argument in the enable script evaluates false, that just means mayastor.io_engine.autoCreateImageSize isn't overridden on the helm command-line, and the chart still falls through to its own default "20G" value.

In order to avoid creating a zero-size diskpool, the initialize-pool container in extraInitContainers needs to be disabled or to exit early when no diskpool should be created.

I'd propose restructuring autoCreateImageSize into a composite value like:

autoCreateImage:
    enabled: true
    size: "20G"

Then modify the enable script to add another flag like microk8s enable mayastor --no-default-pool that sets the enable flag false.
The initialize-pool script then needs to check this flag and terminate early the same way it does if /data/microk8s.img already exists.

Are you interested in contributing with a fix?

I don't expect time to submit a PR for this for likely several months, but if the annoying vestigial errored zero-size image-backed pools are still around once other critical-path issues on my project are resolved, I'll see about making a PR.