Skip to content

Enhancement Proposal: KubeConfigExecProvider for Dynamic Cluster Credentials #26729

@kushwiz

Description

@kushwiz

Summary

Add support for a new cluster authentication method called KubeConfigExecProvider that executes an external command to produce a full kubeconfig file dynamically. Unlike the existing ExecProviderConfig which only generates credentials, this provider generates an entire kubeconfig (including server, CA, and auth) that ArgoCD reads to build a REST config. The cluster cache will also refresh the config on each sync cycle when this provider is used, ensuring rotated credentials and CA certificates are picked up automatically.

Motivation

Use Cases

  1. Dynamic Credential Rotation in Air-Gapped/Secure Environments

    In secure enterprise environments, cluster credentials (certificates, tokens) are frequently rotated for security compliance. Organizations often use internal PKI systems or secrets management tools that require executing a command to fetch fresh credentials. Currently, ArgoCD's ExecProviderConfig only supports generating auth credentials, not the full kubeconfig including CA certificates.

  2. Integration with Custom Authentication Proxies

    Many organizations use authentication proxies (e.g., corporate identity providers, bastion-style access systems) that generate short-lived kubeconfig files. These systems often provide a CLI tool that produces a complete kubeconfig with all necessary connection details.

  3. Multi-Tenant Platforms with Dynamic CA Certificates

    In multi-tenant Kubernetes platforms, CA certificates may be rotated or differ per tenant. The existing ExecProviderConfig cannot handle CA certificate rotation since it only refreshes auth credentials, not the full TLS configuration.

  4. HashiCorp Vault Integration for Full Cluster Configs

    When using Vault to store complete cluster configurations (not just tokens), operators need a way to dynamically fetch the entire cluster config including server URL verification, CA certificate bundles, and client certificates or tokens.

  5. Cloud Provider CLI Integration

    Some cloud providers offer CLI tools that generate complete kubeconfig files with auto-refreshing credentials.

Current Limitations

  • ExecProviderConfig: Only produces credentials (tokens/certs), not full kubeconfig. Cannot handle CA rotation.
  • Static cluster configuration: Requires manual updates when credentials or CAs rotate.
  • Cluster cache: Uses a static REST config, meaning credential expiration causes connection failures until the cluster is re-registered.

Proposal

1. New Type: KubeConfigExecProvider

Add a new configuration type in ClusterConfig that specifies:

  • Command: The executable to run that generates the kubeconfig
  • Args: Arguments to pass to the command (temp file path appended automatically)
  • Env: Additional environment variables for the command
  • APIVersion: Preferred API version of the kubeconfig output
  • InstallHint: Help text shown when the command is not found

Additionally, add a KubeConfigContext field to ClusterConfig to specify which context to use from the generated kubeconfig.

2. Dynamic Config Refresh in Cluster Cache

Add a config provider callback mechanism to the gitops-engine cluster cache. When KubeConfigExecProvider is configured, the controller sets this provider, and on each cache sync cycle, the provider is invoked to get fresh credentials before reconnecting to the cluster.

3. Implementation Flow

  1. User configures cluster with KubeConfigExecProvider pointing to their credential-generation tool
  2. When ArgoCD needs to connect to the cluster:
    • Create a temp file
    • Execute the command with the temp file path as the last argument
    • Command writes kubeconfig to the temp file
    • ArgoCD loads the kubeconfig and builds a REST config
    • Server URL from the Cluster resource is preserved (ArgoCD's identity model)
    • Temp file is deleted
  3. On each cluster cache sync, the process repeats to pick up fresh credentials

4. Security Considerations

  • The exec command runs with the same privileges as the ArgoCD application controller
  • Temp files are created with restricted permissions and immediately deleted after use
  • Command output (stderr) is captured and included in error messages for debugging

5. Example Usage

apiVersion: v1
kind: Secret
metadata:
  name: my-cluster
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: cluster
type: Opaque
stringData:
  name: production-cluster
  server: https://kubernetes.prod.example.com
  config: |
    {
      "kubeConfigExecProvider": {
        "command": "/usr/local/bin/mycorp-k8s-auth",
        "args": ["generate-kubeconfig", "--cluster", "prod"],
        "env": {
          "VAULT_ADDR": "https://vault.example.com"
        },
        "installHint": "Install mycorp-k8s-auth from https://internal.example.com/tools"
      },
      "kubeConfigContext": "prod-admin"
    }

6. Backwards Compatibility

  • This is a purely additive change
  • Existing ExecProviderConfig continues to work unchanged
  • KubeConfigExecProvider takes precedence when both are configured (with a warning log)

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesttriage/pendingThis issue needs further triage to be correctly classified

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions