chart-release workflow does not comply with immutable releases

[furkatgofurov7]

Problem:
Currently, in chart-release workflow cr upload creates and publishes the GitHub release immediately. Any steps that run after it, specifically the recovery step: Verify chart asset present, which does gh release upload --clobber attempts to modify an already published release, violating GitHub's immutable releases requirement.

Additional note:
The recovery step was introduced (by me 😢) in #2141 to handle the case where cr upload succeeds in creating the release but the chart asset ends up missing. Whatever solution is implemented must continue to address that scenario.

Thoughts on possible fix:
We could replace the current cr upload + recovery pattern with a proper draft and then publish flow so that the release is only published after all assets are confirmed present, making both immutable releases compliance and #2138 concerns addressed in one go. Rougly the fix could look like:

• Create the release as a draft first
• Upload the chart asset explicitly to the draft
• Update the helm index
• Publish the release as the very last step

[anmazzotti]

FYI I did some experimenting with this.
As long as the immutable release is in Draft state, artifacts can be added to it at any time.
So a release can be a 2 step process, where:

1. The draft release is generated: gh release create ${{ env.TAG }} --draft --latest=false --generate-notes
2. The chart is built and added to the release: gh release upload ${{ github.event.inputs.release_version }} _assets/dummychart.tar.gz

BIG NOTES:

• --latest=false does not affect the web UI, which will always enable the latest option for any release. The user un-drafting the release must pay a lot of attention to not accidentally tag a release as latest. This needs big warning in the release documentation.
• I suggest doing one or a few releases with the new process, before we activate immutable releases. Let's leave room for mistakes first, otherwise we won't be able to fix without cutting a new one. At very least let's have the first RCs still mutable.

[salasberryfin]

Indeed your proposed solution @furkatgofurov7 @anmazzotti is also the approach recommended by GitHub: https://docs.github.com/en/code-security/concepts/supply-chain-security/immutable-releases#best-practices-for-publishing-immutable-releases.

[furkatgofurov7]

Path to immutable releases

This PR #2416 is Phase 1 of a gradual migration:

Phase 1 (this PR): Draft-first flow, manual undraft. Gives us confidence that the suggested process works reliably without the risk of being stuck with a broken immutable release.

Phase 2 (follow-up, once Phase 1 is validated over a few releases): Uncomment the Publish release step already included in the workflow. This automates the undraft as the very last step, only after all assets are confirmed present, fully following GitHub's recommended immutable release workflow

Phase 3 (no code change needed, a follow-up once Phase 2 is validated): Enable immutable releases in the
repository settings under Settings -> Releases -> "Enable release immutability".

I will create a follow-up issue to track Phase 2 and 3 which should not block this work and anyway, it will take time to get there.