Merge pull request #15 from hcguersoy/feature/py3-ghactions

Feature/py3 ghactions

Author: hcguersoy

.github/workflows/build-publish.yaml

@@ -0,0 +1,72 @@
+name: Build and Publish cleanreg
+
+on:
+  push:
+    branches:
+      - 'main'
+      - 'pr/**'
+      - 'feature/**'
+    paths-ignore:
+      - 'README.md'
+
+  release:
+    types: [published]
+
+  workflow_dispatch:
+
+jobs:
+
+  test-cleanreg:
+    runs-on: 'ubuntu-20.04'
+
+    steps:
+
+      - uses: actions/checkout@v2
+
+      - run: |
+          cd test
+          chmod +x runAllTests.sh
+          ./runAllTests.sh
+        name: Execute tests
+        shell: bash --noprofile --norc {0}
+
+  package:
+    runs-on: 'ubuntu-20.04'
+    needs: test-cleanreg
+    # create the images and push them only if on a release tag, starting with a 'v'
+    if: startsWith(github.ref, 'refs/tags/v')
+    steps:
+
+      # here we create the version information based on the tag and the commit sha
+      # but only if we are building from a tag reference
+      - name: Create tags
+        id: create_tags
+        run: |
+          DOCKER_IMAGE=${{ secrets.DOCKERHUB_USERNAME }}/cleanreg
+          SHORTREF=${GITHUB_SHA::8}
+          VERSION=${{ github.ref_name }}
+          RUN_NUMBER=${{ github.run_number }}
+
+          TAGS="${DOCKER_IMAGE}:${VERSION},${DOCKER_IMAGE}:${VERSION}-${SHORTREF}-${RUN_NUMBER}"
+
+          echo "::set-output name=tags::${TAGS}"
+        shell: bash
+
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v1
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v1
+
+      - name: Login to DockerHub
+        uses: docker/login-action@v1
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+
+      - name: Build and push
+        uses: docker/build-push-action@v2
+        with:
+          push: true
+          platforms: linux/amd64,linux/arm64
+          tags: ${{ steps.create_tags.outputs.tags }}

.gitignore

@@ -2,3 +2,4 @@
 test/testlog.txt
 .DS_Store
 venv
+.python-version

.travis.yml

@@ -1,4 +1,5 @@
-FROM python:2.7-alpine
+FROM python:3.10.2-alpine3.15
+RUN python -m pip install --upgrade pip
 RUN pip install requests PyYAML
 COPY cleanreg.py /cleanreg.py
 COPY LICENSE /LICENSE

Dockerfile

@@ -186,7 +186,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright {yyyy} {name of copyright owner}
+   Copyright 2022 All cleanreg authors
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

Dockerfile.dummybox

@@ -1,4 +1,4 @@
-[![Build Status](https://travis-ci.org/hcguersoy/cleanreg.svg?branch=master)](https://travis-ci.org/hcguersoy/cleanreg)
+
 # Registry Cleaner
 
 This is a small tool to delete tags, or, to be more correct, delete image manifests, from a Docker Registry implementing the API v2.
@@ -9,7 +9,8 @@ Information about the needed garbage collection is described at [https://docs.do
 
 ## History
 
-* v0.7.1 - Added a `--skip-tls-verify` attribute
+* v0.8.0 - Finally find the time to migrate to Python 3 and Github Actions. This comes this many changes in the code and the surrounding build system. Support for `linux\arm64` added.
+* v0.7.1 - Added a `--skip-tls-verify` attribute (_unreleased_)
 * v0.7 - This is a release which breaks some stuff (configuration file is now yaml based), adding new options for keeping images (e.g. `--since`, `--regex`) (thanks to @JulianSauer for his [PR10](https://github.com/hcguersoy/cleanreg/pull/10))
 * v0.6 - add `-cf` flag which allows to clean up all repos in a registry (thanks @kekru for his PR)
 * v0.5 - fix for issue [#8](https://github.com/hcguersoy/cleanreg/issues/8) which resulted in deleting more layers then intended; performance improvements; added `--metadata-workers` attribute
@@ -19,32 +20,34 @@ Information about the needed garbage collection is described at [https://docs.do
 * v0.2 - added support for registry server using self signed certificates
 * v0.1 - first version with basics
 
-## Prerequisites and supported Plattform
+## Prerequisites and supported Platform
 
 You need Docker and a Docker registry system which implements the Registry API v2.
 For Docker Registry v2 API specification see [https://docs.docker.com/registry/spec/api/](https://docs.docker.com/registry/spec/api/).
 
-Be sure to configure your registry server to allow deletion (see [https://docs.docker.com/registry/configuration/#/delete](https://docs.docker.com/registry/configuration/#/delete)). 
+Be sure to configure your registry server to allow deletion (see [https://docs.docker.com/registry/configuration/#/delete](https://docs.docker.com/registry/configuration/#/delete)).
 
 ## Usage
 
-Download the file *cleanreg.py* or clone this repository to a local directory or pull the docker image. This is the suggested way to run `cleanreg`! 
+Download the file _cleanreg.py_ or clone this repository to a local directory or pull the docker image.
+Beginning with version `v0.0.8`, images are supported for `linux/amd64` and (this is new) `linux\arm64`.
+This is the suggested way to run `cleanreg`!
 
-```
-docker pull hcguersoy/cleanreg:v0.7.0
+```shell
+docker pull hcguersoy/cleanreg:v0.8.0
 ```
 
 The image is hosted here: [https://hub.docker.com/r/hcguersoy/cleanreg/](https://hub.docker.com/r/hcguersoy/cleanreg/ "")
 
-*Hint:* `latest` tag is not supported (you know, `latest` is evil 😈).
+_Hint:_ `latest` tag is not supported (you know, `latest` is evil :imp:).
 
 To get a immediate help simply run it without any parameters:
 
-```
-$ docker run --rm hcguersoy/cleanreg:v0.7.0
+```shell
+$ docker run --rm hcguersoy/cleanreg:v0.8.0
 usage: cleanreg.py [-h] [-v] -r REGISTRY [-p] [-y] [-q] [-n REPONAME:TAG]
                    [-k KEEPIMAGES] [-re] [-d DATE]
-                   [-f REPOSFILE] [-c CACERT] [-i] [-u BASICAUTHUSER]
+                   [-f REPOSFILE] [-c CACERT] [-sv] [-i] [-u BASICAUTHUSER]
                    [-pw BASICAUTHPW] [-w MD_WORKERS]
 
 Removes images on a docker registry (v2).
@@ -117,68 +120,67 @@ optional arguments:
 
 ```
 
-
 If you've to use a configuration file (parameter `-f`) you should mount that file into your container:
 
-```
+```shell
 docker run --rm -it -v $(pwd)/cleanreg-example.conf:/cleanreg-example.yaml hcguersoy/cleanreg:<version> -r  http://192.168.56.2:5000 -f cleanreg-example.yaml -i
 ```
 
 As an alternative, you can create your own image, inheriting from `cleanreg`:
 
-```
-FROM hcguersoy/cleanreg:v0.7.0
+```shell
+FROM hcguersoy/cleanreg:<version>
 ADD myconfig.yaml /config.yaml
 ```
 
 ## Examples
 
-**Attention:** It is strongly recommended that you use the *-i* flag even it is more time and memory consuming. If not you can delete images / layers which you not wanted to delete because registry itself doesn't check if a digest is referenced by multiple tags!
+**Attention:** It is strongly recommended that you use the _-i_ flag even it is more time and memory consuming. If not you can delete images / layers which you not wanted to delete because registry itself doesn't check if a digest is referenced by multiple tags!
 
 Cleaning up a single repository called mysql on registry server 192.168.56.2:5000 and keeping 5 of the latest images:
 
-```
+```shell
 docker run --rm -it hcguersoy/cleanreg:<version> -r http://192.168.56.2:5000 -n mysql -k 5
 ```
+
 Be aware that you don't keep here the five last tags but digests/images. As a digest can be associated with multiple tags this can result in deletion of images which you not intended in!
 Again: to be secure use the `-i` flag:
 
-```
+```shell
 docker run --rm -it hcguersoy/cleanreg:<version> -r http://192.168.56.2:5000 -n mysql -k 5 -i
 ```
 
 Same as above but ignore images which are associated with multiple tags.
 
-```
+```shell
 docker run --rm -it hcguersoy/cleanreg:<version> -r http://192.168.56.2:5000 -n mysql:latest -i
 ```
 
 Will only delete the image mysql which is tagged as latest.
 
-```
+```shell
 docker run --rm -it hcguersoy/cleanreg:<version> -r http://192.168.56.2:5000 -n mysql:.*temp.* -re -d 2018-01-01 -k 5 -i
 ```
 
-Removes all images that contain the word "temp" in their tagnames and if they were created before 2018 but at least 5 will be kept in total.
+Removes all images that contain the word "temp" in their tag's and if they were created before 2018 but at least 5 will be kept in total.
 
-```
+```shell
 docker run --rm -it hcguersoy/cleanreg:<version> -r http://192.168.56.2:5000 -n myalpine -k 50 -i -w 12
 ```
 
-If you have a very large registry and enough bandwidth you can increase the parallel workers to retrieve the image metadata. The default is *6*. Be aware that you can generate a *DoS* on your registry server by increasing to much.
-
+If you have a very large registry and enough bandwidth you can increase the parallel workers to retrieve the image metadata. The default is _6_. Be aware that you can generate a _DoS_ on your registry server by increasing to much.
 
 Cleaning up all repositories of the registry:
 
-```
+```shell
 docker run --rm -it hcguersoy/cleanreg:<version> -r http://192.168.56.2:5000 -cf -k 5 -i
 ```
-This will clean up all repositories, keeping 5 images per repository.
 
+This will clean up all repositories, keeping 5 images per repository.
 
 Cleaning up multiple repositories defined in a configuration file:
 
-```
+```shell
 docker run --rm -it \
         -v $(pwd)/cleanreg-example.conf:/cleanreg-example.yaml \
        hcguersoy/cleanreg:<version>  \
@@ -187,9 +189,10 @@ docker run --rm -it \
        -re \
        -i
 ```
+
 The configuration file has the format
 
-```
+```yaml
 <repository name>:
     tag: <tag>
     keepimages: <number of images to keep>
@@ -198,7 +201,7 @@ The configuration file has the format
 
 The values for `tag`, `keepimages` and `keepsince` are optional. If the tag should be parsed as a regular expression use the `-re` flag as shown above. A simple example for the configuration file:
 
-```
+```yaml
 consul:
   tag: OnlyThisTag
   keepimages: 20
@@ -210,36 +213,38 @@ dummybox:
   keepimages: 0
 ```
 
-
 The configuration file can be used together with the clean-full-catalog option:
 
-```
+```shell
 ./cleanreg.py -r http://192.168.56.2:5000 -cf -d 20180101 -f cleanreg-example.conf -i
 ```
-This will clean the repositories with images to keep as defined in the configuration file and it will additionally clean all other repositories of the registry, keeping images per repository that were created since 2018.
 
+This will clean the repositories with images to keep as defined in the configuration file and it will additionally clean all other repositories of the registry, keeping images per repository that were created since 2018.
 
 There is a simple script added to create multiple image tags (based on `busybox`) on your registry server.
 
-If you have installed a *semi secure* registry server using TLS and self signed certificates you have to provide the path to the CA certificate file:
+If you have installed a _semi secure_ registry server using TLS and self signed certificates you have to provide the path to the CA certificate file:
 
-```
-./cleanreg.py -r https://192.168.56.3:5000 -c /my/certifacates/ca.pem -f cleanreg-example.conf -i
+```shell
+./cleanreg.py -r https://192.168.56.3:5000 -c /my/certificates/ca.pem -f cleanreg-example.conf -i
 ```
 
-If you run *cleanreg* in a container you should not forget to mount the certificate file into the container like the configuration file above.
+If you run _cleanreg_ in a container you should not forget to mount the certificate file into the container like the configuration file above.
+Alternatively you can set the option `--skip-tls-verify`. In this case, you don't need to provide a certificate.
 
-If your registry is protected with basic auth and the username is `test` and the password is `secret`, you have to pass these credentials to *cleanreg*.
+> :exclamation: be aware that this should only used for (local) testing but not for productive environments.
 
-```
+If your registry is protected with basic auth and the username is `test` and the password is `secret`, you have to pass these credentials to _cleanreg_.
+
+```shell
 ./cleanreg.py -r https://192.168.56.3:5000 -u test -pw secret -f cleanreg-example.conf
 ```
 
-## Runing Garbage Collection
+## Running Garbage Collection
 
 Example on running the garbage collection:
 
-```
+```shell
 $ docker run --rm \
   -v /docker/registry2:/var/lib/registry:rw \
   registry:latest bin/registry \
@@ -254,27 +259,28 @@ The registry itself should be stopped before running this.
 
 Feel free to contribute your changes as a PR. Please ensure that the tests run without errors and provide tests for additional functionality.
 
-This tool was implemented and tested on Ubuntu Linux 16.04 and on MacOS 10.13 using Python 2.7. It is developed against Docker Registry version [2.5.1](https://github.com/docker/distribution/releases/tag/v2.5.1), but tested against [2.6.1](https://github.com/docker/distribution/releases/tag/v2.6.1) and *latest* (see [https://hub.docker.com/r/library/registry/](https://hub.docker.com/r/library/registry/)).
+This tool was initially implemented and tested on Ubuntu Linux 16.04 and on MacOS 10.13 using Python 2.7 and migrated to Python 3.10 on macOS 12 (arm64).
+It is developed and tested against Docker Registry versions [v2.7.1](https://github.com/docker/distribution/releases/tag/v2.7.1) and [v2.8.0](https://github.com/docker/distribution/releases/tag/v2.8.8).
+It should work with older registry versions which are supporting the API, but they are not tested anymore.
 
-You need to install the Python modules *requests* and *PyYAML*:
+You need to install the Python modules _requests_ and _PyYAML_:
 
-```
+```shell
 $ pip install requests PyYAML
 ```
 
-Be sure to configure your registry server to allow deletion (see [https://docs.docker.com/registry/configuration/#/delete](https://docs.docker.com/registry/configuration/#/delete)).
-
+Be sure to configure your registry server to allow deletion (see [https://docs.docker.com/registry/configuration/#delete](https://docs.docker.com/registry/configuration/#delete)).
 
-### Run tests
+### Run tests locally
 
 Prerequisites:
 
 * Bash
-* Locally installed Docker engine (remote execution is not yet implemented; runs with Docker for MacOS fine)
+* Locally installed Docker engine. Remote execution is not yet implemented. Runs with [Rancher Desktop](https://rancherdesktop.io) on macOS fine (configured to use _dockerd_ as runtime).
 
-You can run all tests, with the *runAllTests.sh* script:
+You can run all tests, with the _runAllTests.sh_ script:
 
-```
+```shell
 cd test
 ./runAllTests.sh
 ```
@@ -283,14 +289,25 @@ This will run all tests and repeat them for different versions of the Docker Reg
 
 To run a single test, change to the `test/tests` directory and run a test script:
 
-```
+```shell
 cd test/tests
 ./simple_clean.sh
 ```
-By default the test will start the Docker Registry from Docker Hub with the tag `latest`. To specify another registry version, write its Docker Hub tag to the environment variable `REGISTRYTAG`.
 
-```
+By default the test will start the Docker Registry from Docker Hub with the tag `latest`.
+To specify another registry version, set it using the environment variable `REGISTRYTAG`.
+
+```shell
 cd test/tests
 export REGISTRYTAG=2.5.1
 ./simple_clean.sh
 ```
+
+### GitHub Actions
+
+After pushing to `master` or a branch `feature\*` or `pr\*` a GitHub Action will be triggered which runs the tests defined in `test/runAllTests.sh`. If one of these tests fail, the action will fail, too.
+If the build was triggered due to the creation of a release, a GitHub Action will be triggered, too. In this case, after running successfully the tests, the Docker image will be build with the help of _buildx_.
+Currently, two kinds of architectures are supported: `linux/amd64` and `linux/arm64`.
+The later one could be used on Apple Silicon systems this the according runtime, e. g. Rancher Desktop.
+
+> Due to lack of knowledge regarding other ARM systems (e. g. Raspberry Pi) you are very welcome to provide a PR.