Document how to use the hashivault secret manager in the operator

[viveklak]

Affected feature

Secret manager integration with hashicorp vault

Details

The operator's integration with hashicorp vault secret manager is a bit more complicated than other cloud secret managers since it typically needs a renewable token. Pulumi's cloud secret manager support relies on https://gocloud.dev/ which in the case of Hashi vault, requires that the token be specified using a static environment variable VAULT_SERVER_TOKEN. In the case of the Pulumi CLI this is workable since the CLI is shortlived but this is not workable in the Operator.

[viveklak]

The recommended approach for this is to use Vault injector to inject a vault agent sidecar that can act as a proxy to an internal or external vault. The operator uses the sidecar through VAULT_SERVER_URL but no VAULT_SERVER_TOKEN is used. This way the operator is able to work without having to inject tokens while the vault agent handles authentication and token renewal.

An initial approach is documented here: https://gist.github.com/viveklak/05344c9c684dce4dea41bb09915903e0#file-operatorwithvaultsecretmanager-md which I validated locally. This should be moved to regular operator docs.