CLOUDP-407997: Publish MCK promoted MCK images

[josvazg]

Summary

Prepare CI tool to publish previously promoted images and unify naming.

New mckci release publish and mckci release publish images commands for publishing single and all release images at a specific commit (or promoted-latest). Stomp protection added to mckci release promote images.

Stomp protection (both promote images + publish images)

Every image's immutable tag is checked for conflicts BEFORE any writes. If any
image conflicts and --force is not set, the entire operation is refused — no
partial writes.

Examples

# Publish all release images from latest promoted image/commit
mckci release publish images 

# Publish all release images at a specific commit
mckci release publish images \
  --commit a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2

# Publish single image (omit --commit for promoted-latest)
mckci release publish \
  --staging-image quay.io/mongodb/staging/mongodb-kubernetes \
  --commit a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2

# Promote all release images with force override
mckci release promote images --commit a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2 --force

# Dry-run all images
mckci release promote images --commit a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2 --dry-run

Proof of Work

CI must pass, changes fully self tested.

Checklist

• Have you added changelog file?
  • use skip-changelog label if not needed

[github-actions]

⚠️ (this preview might not be accurate if the PR is not rebased on current master branch)

MCK 1.10.0 Release Notes

New Features

• MongoDBOpsManager, AppDB: The mongodb-agent-monitoring sidecar container has been merged into the main
  mongodb-agent container. Both automation and monitoring now run as a single process in a single container. The
  spec.appDB.monitoringAgent field is now deprecated and has no effect; Monitoring Agent options for the AppDB —
  including log level and rotation — are configured via spec.applicationDatabase.agent and
  spec.applicationDatabase.agent.monitoringAgent, the same fields used by MongoDB resources. Enabling AppDB
  monitoring still results in rolling restarts of AppDB pods.
• MongoDBSearch: Added support to show cluster level status of Search, Managed LoadBalancer and MetricsForwarder in the MongoDBSearch resource's status.clusters field.
• MongoDBSearch: Added support to configure the node affinity of the MongoDB Search (mongot) pods using the MongoDBSearch CR fields spec.clusters[].nodeAffinity or spec.clusters[].shardOverrides[].nodeAffinity.

Bug Fixes

• MongoDBUser: Fixed a bug where the connection string Secret created in the central cluster did not carry a controller owner reference to the MongoDBUser CR. The missing reference prevented Kubernetes garbage collection from cleaning up the Secret when the MongoDBUser was deleted.
• MongoDBUser: Fixed a bug where the ownership guard on the connection string Secret was always satisfied regardless of the actual owner, allowing the operator to silently overwrite a Secret controlled by a different resource.
• Fixed a bug where all MongoDBCommunity deployment telemetry rows incorrectly reported IsRunningEnterpriseImage = true. The field was being evaluated against the operator-level enterprise image rather than the image configured in each CR's spec, causing a misleading ~100% enterprise rate for the Community deployment type.
• IsRunningEnterpriseImage for Community deployments is now derived from the mongod container image override in spec.statefulSet.spec.template.spec.containers, if present. When no override is set, the field correctly defaults to false, reflecting that Community CRs use the community MongoDB server image by default.
• MongoDBOpsManager, AppDB: Fix for spec.applicationDatabase.agent.monitoringAgent.logRotate field not being
  handled properly. Additionally, added defaults for
  spec.applicationDatabase.agent.monitoringAgent.logRotate.sizeThresholdMB to 1000 and
  spec.applicationDatabase.agent.monitoringAgent.logRotate.timeThresholdHrs to 24, which are same defaults Ops Manager
  would set.
• MongoDBSearch: The default JVM heap size (half of the memory request) is now capped at 30GB, following the mongot sizing guidance. Heap sizes above ~30GB prevent the JVM from using compressed object pointers and degrade performance. User-provided heap flags are not affected, if more than 30GB heap is required, we recommend using jvmFlags.
• MongoDB: Fixed a bug where switching a MongoDB ReplicaSet or ShardedCluster to a new Ops Manager project could cause the automation agent to generate a random keyfile in the empty target project, breaking internal cluster authentication.

[josvazg]

Need to fix promotion first upstream, we need to promote and release all associated images not just the operator one.

[m1kola]

I think something is wrong around logic with latest. See questions below.

[m1kola]

Is it a behaviour we want with backporting?

So if we have 2.0.1 right now as latest and then publish 2.0.2 it becomes latest. So far so good. But I I understand the description correctly, backporting something to 1.x and say publishing 1.10.1 - will move latest to that version.

Is my understanding correct? If yes - then I think it is not what should be happening and latest should always post to the highest version possible within the current branch. In the example above it will be 2.0.2 because it is the latest version despite 1.10.1 being the latest in the timeline of releases.

[josvazg]

Yes I think we are going to need more than one "latest" I can add a flag to pass a custom "latest" marker such as latestv1.

The latest marker is useful as it allows to pin the most up to date promoted image without knowing the commit in advance. But you are right for backports we need to extend this a bit.

My proposal is a new optional flag names --latest-marker (or similarly named) that defaults to latest but we can set to "latestv1", "latestv2" etc the "latest" remains the marker for the current release.

[m1kola]

I see. If we need to have some sort of latest marker, I think we can use major version for that. E.g. if 2.0.2 is the latest, it will be tagged with 2.0.2, 2.x (or just 2), latest and anything in 1.x tagged with 1.<minor>.<patch> (say 1.10.1 from example above), 1.x (or just 1) and that's it.

[josvazg]

At the tool level we can accept anything. This PR is at the tool level, the wiring is a separate PR. We can discuss conventions there.

[m1kola]

nit: I would not encourage use of --force

[josvazg]

I can soften the wording, but still, the flag exists and is a separate thing to ensure safe usage. The assumption is if you are using it, is because you must.

double check and use --force to override if required would be test here?

[m1kola]

My logic is that if you must use it - you are aware of it or you at least looked at --help. So it is more of a deliberate decision because it requires some research before using it.

But don't worry about this comment too much. it is ok as is. Not a big deal.