chore(utils): AS-788 speed up docker image checking

[afoley587]

Rationale

Last release, we realized that the docker pull to validate if an image exists or not takes a LONG time. Luckily, there is an experimental docker command: docker manifest

This can help us check the manifest, not the pull, to see if an image exists on the remote.

Changes

Changes the docker pull command to docker manifest inspect
Checklist

• This PR maintains parity between Docker Compose and Helm

Testing

Testing Failures

INFO: Validated docker.io/busybox:stable-glibc successfully.
ERROR: Could not validate voxel51/fiftyone-app:v2.11.1!
ERROR: Could not validate voxel51/fiftyone-app-gpt:v2.11.1!
ERROR: Could not validate voxel51/fiftyone-app-torch:v2.11.1!
ERROR: Could not validate voxel51/fiftyone-teams-api:v2.11.1!
ERROR: Could not validate voxel51/fiftyone-teams-app:v2.11.1!
ERROR: Could not validate voxel51/fiftyone-teams-cas:v2.11.1!
ERROR: Could not validate voxel51/fiftyone-teams-cv-full:v2.11.1!
alexanderfoley@Alexanders-MacBook-Pro fiftyone-teams-app-deploy % echo $?
1
alexanderfoley@Alexanders-MacBook-Pro fiftyone-teams-app-deploy % 

Testing Success (With Timing)

alexanderfoley@Alexanders-MacBook-Pro fiftyone-teams-app-deploy % time ./utils/validate-docker-pulls.sh             
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.oci.image.index.v1+json",
   "manifests": [
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 2388,
         "digest": "sha256:4f2f118665f9dba3c9c606b8ce99b853c519f09d842d20d8b25d5674c03569ba",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 841,
         "digest": "sha256:95e654c7405396dedcc1c80cd3f63522017462b824d93586ae32b3412b26d256",
         "platform": {
            "architecture": "unknown",
            "os": "unknown"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 2388,
         "digest": "sha256:046ace30fd4d5e1e7abaf4a357ed794e9c8d0823b446e075da67f18704c5396d",
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 841,
         "digest": "sha256:7ea8ab2ef138afd763cf1200e1c706f3e5c9f466869dc235e817e998b9750f5f",
         "platform": {
            "architecture": "unknown",
            "os": "unknown"
         }
      }
   ]
}
.
.
.
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.oci.image.index.v1+json",
   "manifests": [
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 610,
         "digest": "sha256:8cac9c34e220de273cc3f255b4472cdb639d59fe848f51df8d7d5ddd3dbd378e",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 559,
         "digest": "sha256:5c2e689a7668fd09709a6f1fd14b759cc766be41e8c3f503ce4adf53f65f2fc0",
         "platform": {
            "architecture": "unknown",
            "os": "unknown"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 610,
         "digest": "sha256:948ea15a937713077ab2955415c85d55206ff8ed96f262ae91c3d1a6f6fcde38",
         "platform": {
            "architecture": "arm",
            "os": "linux",
            "variant": "v5"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 559,
         "digest": "sha256:fee06e90ba796e1d27400476d60455882b15fe60264a2cca34d51f865a3276ef",
         "platform": {
            "architecture": "unknown",
            "os": "unknown"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 610,
         "digest": "sha256:460c140b2d4643a05eb6b024385d6bb41cec14bff465565a692ee4d6dd2aca7b",
         "platform": {
            "architecture": "arm",
            "os": "linux",
            "variant": "v7"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 559,
         "digest": "sha256:4cb5c53b46e8037f59aefeb9f6481dff31601a44aef7487d8a52e01578592bdf",
         "platform": {
            "architecture": "unknown",
            "os": "unknown"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 610,
         "digest": "sha256:2394760438e0c02f5a979042f0960599460998092991252e7c23cb00595be3a2",
         "platform": {
            "architecture": "arm64",
            "os": "linux",
            "variant": "v8"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 559,
         "digest": "sha256:3466615137c2774262a59d40a95035af44168f44da21a55c6191553acf4ee0c6",
         "platform": {
            "architecture": "unknown",
            "os": "unknown"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 610,
         "digest": "sha256:88fc722c54c256ed1f13cc9f2f89ffc6cea57346b43d92217a678ca09fdd4d58",
         "platform": {
            "architecture": "386",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 559,
         "digest": "sha256:7e1ff63b992110a2ad5d2d10dbe12889c67d6edb354f59bfb8f7711fc184876b",
         "platform": {
            "architecture": "unknown",
            "os": "unknown"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 610,
         "digest": "sha256:aef434d7e582fb1c99d2bb39fb8f0a999e35d24e4e9e2b7de6ab2c0e2eca7f9e",
         "platform": {
            "architecture": "mips64le",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 610,
         "digest": "sha256:df4040317ece1fa085bd77fa8259e09436088d7bfe05ed1a7c7a5d849ec01817",
         "platform": {
            "architecture": "ppc64le",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 559,
         "digest": "sha256:795b40bd0eaae6edd224be8a13d4fe605948a9c5f22eac1ff13cc9975383ff14",
         "platform": {
            "architecture": "unknown",
            "os": "unknown"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 610,
         "digest": "sha256:72d669cd4d95998ccba3db5a1988e18878a540847a0a859ef82ee6c3633b04ff",
         "platform": {
            "architecture": "riscv64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 559,
         "digest": "sha256:7b94e7f9d67b5c4da4333cb0c394bf3dc77e261237d03560a3d115b6f08aee6a",
         "platform": {
            "architecture": "unknown",
            "os": "unknown"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 610,
         "digest": "sha256:cbfd8328c46ebbf0844a5c6e291bb776c4e2bf922a86e5f8bc4e9f741e9ddf9e",
         "platform": {
            "architecture": "s390x",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 559,
         "digest": "sha256:742186e80389f41bc2fe0b84ce8e3d45180080e545861f5df6615e5eda016805",
         "platform": {
            "architecture": "unknown",
            "os": "unknown"
         }
      }
   ]
}
INFO: Validated docker.io/busybox:stable-glibc successfully.
INFO: Validated voxel51/fiftyone-app:v2.10.0 successfully.
INFO: Validated voxel51/fiftyone-app-gpt:v2.10.0 successfully.
INFO: Validated voxel51/fiftyone-app-torch:v2.10.0 successfully.
INFO: Validated voxel51/fiftyone-teams-api:v2.10.0 successfully.
INFO: Validated voxel51/fiftyone-teams-app:v2.10.0 successfully.
INFO: Validated voxel51/fiftyone-teams-cas:v2.10.0 successfully.
INFO: Validated voxel51/fiftyone-teams-cv-full:v2.10.0 successfully.
./utils/validate-docker-pulls.sh  0.67s user 0.66s system 16% cpu 8.304 total

[findtopher]

I like 8 seconds instead of several minutes!

[kevin-dimichel]

I thought the "why" of the script wasn't to validate that the images were in our private Docker Hub repository, but to validate that using a PAT, that the images could be downloaded. 🤔

[afoley587]

@kevin-dimichel the intent of the script was originally to validate that our images are available and pullable before merging our helm chart (ie finalizing our release).

I think that validating that the manifests exists accomplishes that!

Original PR

We require a way to make sure that all of our docker images are pullable before releasing artifacts to the public. We spoke internally about a script that performs this validation - essentially doing a docker pull on every image in the output of a helm template.