Make regular (pre/post upgrade,other) backup of all the secrets

[chaen]

A lot of secrets are generated once (signing keys, etc), and are then stored in the cluster only.
They should be backed up in case the cluster dies.
To complete

[chaen]

Todo:
write a utility hosted here for dumping all the important secrets.

A custom CI job can be run daily to use that utility, encrypt it and keep it as an artifact