How to use ArgoCD to add dex auth config

[trashhead]

I want to use argocd to add an auth config(dex.config).
The dex.config is today part of argocd-cm.
And as far as I know I cannot patch an already existing manifest in the cluster.
Would be nice if the dex.config was its own configmap without all the other default config map keys.

Does anyone have a good solution to this?