CI: Example-Unit-Testing service dependencies are unpinned and unwaited (Zipkin :latest, no health checks, bitnamilegacy Kafka)

[akshat-kumar-singhal]

Summary

Example-Unit-Testing depends on five backing services. Two of them (MinIO, and MinIO alone properly) are pinned and waited on; the other four are neither. This is a plausible contributor to the intermittent failures that the nick-fields/retry wrapper currently absorbs.

1. Zipkin is :latest with no readiness wait

- name: Start Zipkin
  run: docker run -d -p 2005:9411 openzipkin/zipkin:latest

go.yml:98-99. Compare the MinIO step immediately below it (go.yml:104-123), which is digest-pinned and polls /minio/health/live for 60s with an explicit ::error:: if it never comes up — with a comment explaining exactly why that matters. Zipkin gets neither: the step returns as soon as docker run -d has started the container, and :latest means the image can change under the repo without any commit.

The same treatment the MinIO step already has should apply here — a digest pin and a poll on Zipkin's /health.

2. The services: containers declare no health checks

Kafka, Redis and MySQL (go.yml:39-72) are all declared without options: --health-cmd/--health-interval/--health-retries. Without those, the runner waits for the container to be created and started, not for the service inside it to be accepting connections. MySQL is the one that takes meaningfully long to become ready after start.

mysql:
  image: mysql:8.2.0
  ports: ["2001:3306"]
  env: { MYSQL_ROOT_PASSWORD: "password", MYSQL_DATABASE: "test" }
  options: >-
    --health-cmd="mysqladmin ping -h 127.0.0.1 -ppassword"
    --health-interval=10s --health-timeout=5s --health-retries=5

To be clear about the evidence: I have not traced a specific CI failure to a service not being ready — the retry wrapper makes that hard to see, since a first-attempt failure is only visible as a ##[warning]Attempt 1 failed line. This is a known-unsound pattern rather than a diagnosed flake, and adding health checks is also how you'd find out whether it is one.

3. bitnamilegacy/kafka:3.4.1 is a frozen archive

kafka:
  image: bitnamilegacy/kafka:3.4.1

Verified against Docker Hub (hub.docker.com/v2/repositories/bitnamilegacy/kafka, last updated 2025-08-28), where the description reads:

⚠️ This repository is no longer updated. In this Bitnami Legacy repository, you can find a backup of all existing container images, which will receive no further updates or support and should only be used for temporary migration purposes. If you rely on any of these legacy images, we suggest pulling and storing them in your own container registry to ensure continued availability. Please note that this repository may be removed in the future.

So this is a dependency on an image the publisher has said may be deleted, with no security updates in the meantime. It works today; the day it stops working, every example test that touches pub/sub fails at once and the cause will not be obvious from the logs.

Options, roughly in order of effort: move to apache/kafka (official, but different env-var surface — KAFKA_CFG_* is a Bitnami convention and the whole env: block would need rewriting), mirror the current image into GHCR under gofr-dev, or take Bitnami Secure Images.

Expected

• Every image used by CI pinned by digest or at minimum an immutable tag.
• Every service waited on for readiness, not just for container start.
• No dependency on a registry namespace whose publisher has announced it may be removed.

Context

Found while reviewing run 31266842627. Related to #3818 (readiness polling in the example tests themselves) — same class of problem one layer up.

[akshat-kumar-singhal]

PR #3939 covers items 1 and 2 of this issue — Zipkin is now digest-pinned with a /health readiness poll (the same treatment the MinIO step already had), and Kafka, Redis and MySQL declare health checks so the runner waits for the service rather than for the container to start. Each health command was verified against the real image; redis-cli ping was checked in both directions (0 up, 1 down) so the check can genuinely fail rather than passing unconditionally.

Leaving this issue open for item 3, the bitnamilegacy/kafka:3.4.1 migration. It was deliberately kept out of that PR: apache/kafka has a different env-var surface (KAFKA_CFG_* is a Bitnami convention, so the whole env: block gets rewritten), and it cannot be validated without a live CI run — a wrong block turns every pub/sub example test red at once. Bundling it with the verifiable readiness fixes would have made a red run ambiguous.

The three options from the issue still stand: move to apache/kafka, mirror the current image into GHCR under gofr-dev, or take Bitnami Secure Images. The mirror is the lowest-risk of the three since it keeps the env block byte-identical, but it needs a maintainer to create the package and grant packages: write — that is not something I can do from a fork. Happy to take whichever direction you prefer.

The reasoning is also recorded in a comment above the image in go.yml so it survives if this issue scrolls out of view.