[CI] Two-step EPR package promotion to prevent image pulls

[delanni]

Summary

• We cache the EPR (elastic package registry) image on the VM image, so we don’t need to pull on every FTR
• We run tests daily on the new EPR image, and promote it to the kibana-ci domain when the tests pass
• Then we trigger a cache-image refresh build (but that takes ~1hr)
• So there's a time window, when the VM with the refreshed image is building, within this window FTRs who want to start EPR need to pull the layer differences between the cached and the newly promoted image.
• This probably causes the disk space to deplete.

Symptom

The result is we have some daily, unfortunately scheduled jobs that will run out of disk space while running.

Fix suggestion

Change .buildkite/scripts/steps/fleet/promote_package_registry.sh to promote the verified EPR image as an alternative URL - the VM image build should pick up this alternative/temp URL (so while the image is building, FTR tests using the final promoted image URL, that still points to the previous image version - thus no pull should be triggered). Once the VM image is built, we can move the promoted image URL to point to the alternative/temp URL, and thus the FTR tests from then on out will use the new version of the image in FTRs.

[elasticmachine]

Pinging @elastic/kibana-operations (Team:Operations)

[mistic]

#244766 was also a follow up of this

[delanni]

Closing this, as it seems resolved