more frequent Dockerfile builds

[glyg]

I hope it is OK to ask this here, and more appropriate than on the image.sc post

Is there a reason not to RUN yum update -y right after the CentOS image was pulled?

Best

Guillaume

[sbesson]

@glyg thanks for raising this issue. The forum post was equally as appropriate and is it on our radar to follow-up but the team is operating in reduced capacity with several of us attending ELMI this week.

Speaking from the IDR experience, we are using rolling deployments with the base packages being systematically upgraded after provisioning the VMs. So running yum update -y when building the Docker images would mirror this behavior and feels reasonable and fairly low-cost.
The primary downside from my perspective is whether this communicates the user expectation that the latest Docker image will always be up-to-date with the latest upstream packages. In that case, this is a necessary first step byt additional infrastructure will be required to rebuild the image periodically as discussed in the related post.

I'll leave @joshmoore to comment as he was more heavily involved in the initial work on these Docker images and there might be some rationale I am overlooking.

[glyg]

The forum post was equally as appropriate and is it on our radar to follow-up but the team is operating in reduced capacity with several of us attending ELMI this week.

Sure, I was not meaning to sound impatient sorry!

[glyg]

The primary downside from my perspective is whether this communicates the user expectation that the latest Docker image will always be up-to-date with the latest upstream packages. In that case, this is a necessary first step byt additional infrastructure will be required to rebuild the image periodically as discussed in the related post

Yes, I think we are going to try that on our side for the moment anyway, but I wonder about good tests to run when re-deploying so frequently

[sbesson]

but I wonder about good tests to run when re-deploying so frequently

At least on the OME side, the latest versions of these Docker images should be tested weekly via the scheduled GitHub Actions CI builds in:

• the Docker deployment example under https://github.com/ome/omero-deployment-examples
• the integration tests for most plugins under https://github.com/ome/omero-plugins using https://github.com/ome/omero-test-infra

[glyg]

Thanks @sbesson If I understand correctly, integration tests as ran in omero-test-infra but with our own docker-compose and set of plugins would be a first step.

[joshmoore]

Probably the biggest downside of running apt/yum update that I can think of is the size of the image. It means our "base" image is a custom centos rather than the upstream one that others may have already pulled.

[sbesson]

Ah forgot about the image size cost. A quick test certainly shows a 2.5 fold increase

(base) [sbesson@idr2-slot3 ~]$ docker images centos
REPOSITORY   TAG       IMAGE ID       CREATED        SIZE
centos       7         eeb6ee3f44bd   8 months ago   204MB
(base) [sbesson@idr2-slot3 ~]$ docker build -t centos_updated .
Sending build context to Docker daemon  101.1MB
Step 1/2 : FROM centos:7
 ---> eeb6ee3f44bd
Step 2/2 : RUN yum update -y
 ---> Running in abedb5d76066
...
Complete!
Removing intermediate container abedb5d76066
 ---> c9cbc8961302
Successfully built c9cbc8961302
Successfully tagged centos_updated:latest
(base) [sbesson@idr2-slot3 ~]$ docker images centos_updated
REPOSITORY       TAG       IMAGE ID       CREATED         SIZE
centos_updated   latest    c9cbc8961302   2 minutes ago   509MB

[glyg]

Thanks, this indeed is an issue. It is said here that they perform rolling updates monthly or "as needed for emergency fixes". Let's see whether my security conscious sysadmin finds this acceptable :)

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/omero-docker-images-security-upgrades/67815/6

[JulianHn]

While the size increase of course cannot be mitigated completely, it can at least be somewhat limited by performing a clean of all caches in the same Run command:

FROM centos:centos7
RUN yum update -y && yum -y clean all && rm -fr /var/cache

docker build -t centos-update-clean .
Cleaning up list of fastest mirrors
Removing intermediate container 755787065bb7
 ---> 25fe9406d9ef
Successfully built 25fe9406d9ef
Successfully tagged centos-update-clean:latest
❯ docker images centos-update-clean
REPOSITORY            TAG       IMAGE ID       CREATED         SIZE
centos-update-clean   latest    25fe9406d9ef   9 seconds ago   364MB

This should be closer to the actual amount of additional installed packages.

// Julian

[JulianHn]

Actually, concerning the sizes: Independent of what the decision on yum update is, there is opportunity to decrease the docker sizes by amending the RUN commands with a clean of the caches:

Dockerfile diff:

--- a/Dockerfile
+++ b/Dockerfile
@@ -8,9 +8,13 @@ ADD playbook.yml requirements.yml /opt/setup/
 
 RUN yum -y install epel-release \
     && yum -y install ansible sudo \
-    && ansible-galaxy install -p /opt/setup/roles -r requirements.yml
+    && ansible-galaxy install -p /opt/setup/roles -r requirements.yml \
+    && yum -y clean all \
+    && rm -fr /var/cache
 
-RUN ansible-playbook playbook.yml
+RUN ansible-playbook playbook.yml \
+    && yum -y clean all \
+    && rm -fr /var/cache

I get the following image sizes (Clean & Update has an additional yum update -y in the first RUN command):

Options	Image Size
Current	1.29GB
Clean	918MB
Clean & Update	1.05GB

So this might be something to think about.

// Julian

[joshmoore]

If we had an auto-redeploy (i.e. bumping the final -1 build number) every time that the upstream centos image is released, would there still be a desire to keep yum update in the build? If not, I'd be more in favor of more frequent rebuilds. (Either via a new issue or by updating this one)