[RFE] Consider different base image

[mallardduck]

As title, this issue is tracking work looking into a different base image.

Currently BRO uses registry.suse.com/bci/bci-micro:latest as a base image for the final result. We could reduce the footprint by going down to the busybox BCI, and/or by using scratch. In either case having a reduced footprint will improve security concerns related to this image.

Here are the options, we have:

Option 1: Switch to Busybox base

This reduces the file size quite a bit - 5MB compressed, ~20MB on disk.

Option 2: Switch to scratch base

This reduces the file size quite a bit - 10MB compressed, ~40MB on disk.
This is most optimal from a security POV as the only thing in the container is our binary.
However, because the only thing in the image is our binary you cannot exec into the container.
Instead one would have to sidecar the pod with a shell specific image.

Option 3: Switch to scratch and busybox base

This is the "why not both" option. We pick one as the default and one as an opt-in.
Ultimately this could be the ideal solution as it gives us benefits of both options.

I would be in favor of picking scratch as default and suffixing the busybox version as different.
This way we know all our users are working with the minimal version by default, rather than opting in.
However, the idea can work just the same in the other direction with scratch based images being opt-in.

[mallardduck]

@alexandreLamarre - I wonder if option 3 is the way to go here, but with a delay in picking how we publish variants?

Essentially meaning we pick Option 1 to act on now, and then that doesn't prevent us from later picking Option 3. And down the road we can evaluate which base image should be default and which one opt-in?

[mallardduck]

Pushing this out to backlog since current images state makes this less concerning. (Maybe aim for 2.10)