ArgoCD pull integration (basic model) does not work with ArgoCD running as EKS "capability"

[alexoughton]

Hello,

AWS have recently released a feature for EKS where ArgoCD can run as an EKS "capability":

https://docs.aws.amazon.com/eks/latest/userguide/argocd.html
https://docs.aws.amazon.com/eks/latest/userguide/argocd-comparison.html

In this feature, ArgoCD is effectively running as part of the EKS control-plane. While you will see ArgoCD CRDs on the cluster, and can create Application(set) manifests and cluster secrets, you will not see any ArgoCD-related pods running on your cluster.

I have attempted to use the ArgoCD pull integration (basic model) with a spoke cluster using this form of ArgoCD. This does not work due to differences in how ArgoCD works here. The pull integration is functioning correctly on the hub, creating ManifestWorks as appropriate. The spoke cluster is correctly creating Application manifests from the AppliedManifestWork. However, the application does not deploy.

The cause is that the OCM ArgoCD pull integration is hard-coded to set spec.destination.server to https://kubernetes.default.svc. The use of Kubernetes https URLs is not supported by the EKS ArgoCD capability. There are two supported options:

1. Set spec.destination.server to the ARN of the cluster.
2. Set spec.desintation.name to the name of a cluster registered with ArgoCD (which will itself be properly configured to use the ARN instead of the https URL).

It therefore seems that the pull integration could work here if we had the option to change the behavior of the spec.destination set on the ManifestWork the integration creates. I would suggest setting spec.destination.name to in-cluster. If we then create an appropriate cluster secret registering in-cluster as the spoke cluster's ARN, then this will allow application deployment.

This change should be allowed to operate simultaneously with the current behavior, so that the same hub cluster can manage non-EKS-capability clusters (which still want the https URL as the destination), as well as clusters migrated to the EKS version. I would therefore suggest that this behavior be controlled by an annotation on the Application(set) manifest on the hub.

Thanks in advance for considering this feature request.

Alex

[mikeshng]

I agree that this behavior should be controlled via annotations on the Application(set) manifest on the Hub cluster. We are already utilizing specific labels and annotations for the pull integration, such as:

labels:
  apps.open-cluster-management.io/pull-to-ocm-managed-cluster: 'true'
annotations:
  argocd.argoproj.io/skip-reconcile: 'true'
  apps.open-cluster-management.io/ocm-managed-cluster: '{{name}}'

I propose introducing two optional annotations:

apps.open-cluster-management.io/destination-name Allows the user to specify a custom cluster name (ie, in-cluster).

apps.open-cluster-management.io/destination-server Allows the user to specify a custom API server URL (ie, https://kubernetes.default.svc).

If either of these annotations is present, the integration will use the provided value to override the spec.destination.name/server.
If neither annotation is present, it will fall back to the existing default behavior.

@alexoughton WDYT?

/assign @mikeshng

[alexoughton]

@mikeshng Yes, I believe that would solve this entirely.

[mikeshng]

As discussed with Alex, the presence of any new annotation overrides the default behavior.