Dynamic input for GitOpsTemplates

[sympatheticmoose]

WIP

As a Template author, I want to be able to define a parameter which can be set based on a response from a query to a WGE API, for example to select a Cluster from a returned set of GitOpsCluster objects.

As a Template author, I can easily discover available APIs which would be useful in Template authoring.

As a Template user, I want to create a resource which references existing installed components available to me, to simplify filling out a template, to reduce the chance of inputting incorrect values, and to help simplify adherence to RBAC.

Example based on Pipeline Template
A Pipeline references both an Application (HelmRelease) and set of DeploymentTargets (GitOpsCluster + Application Namespace). GitOpsClusters can be hardcoded into the template, including as set options, requiring the Template author to ensure these are valid, or specified by the template user as a string value, requiring them to provide the correct name and namespace combinations (error-prone).

This feature would allow for the values to be set based on a response to an API query, therefore making it easier to correctly pass on the values into the template form.

Below is for illustrative purposes only, showing how a team could specify their Dev Cluster for the application, but staging and production remain hardcoded as more stable and shared environments.

apiVersion: clustertemplates.weave.works/v1alpha1
kind: GitOpsTemplate
metadata:
  name: pipeline
  namespace: default
annotations: 
  templates.weave.works/profiles-enabled: true
  templates.weave.works/kustomizations-enabled: true
  templates.weave.works/credentials-enabled: true
labels:
  weave.works/template-type: pipeline
spec:
  description: Creates an application Pipeline
  params:
  #metadata
    - name: PIPELINE_NAME
      description: Name of Pipeline
    - name: PIPELINE_NAMESPACE
      description: Namespace for the Pipeline
  #Appdata
    - name: APP_NAME
      description: Name of the HelmRelease for your application
    - name: APP_NAMESPACE
      description: Namespace for the HelmRelease
    - name: APP_REPO_URL
      description: URL for source of HelmRelease in Git
    - name: APP_REPO_BRANCH
      description: Branch to write updates to in Git
    - name: GIT_TOKEN_SECRET
      description: Name of the secret containing PAT for Git
 #Environments 
   - name: DEV_CLUSTER
     description: Development Cluster where the application is deployed
     query: clusters #Maps to https://<dashboard>/v1/clusters, UI shows a drop down menu similar to Add App today
     values: #based on user selection of a returned object, these values become accessible in the template
        DEV_CLUSTER.NAME: "gitopsClusters.name"
        DEV_CLUSTER.NAMESPACE: "gitopsClusters.namespace"
resourcetemplates:
- apiVersion: pipelines.weave.works/v1alpha1
  kind: ${PIPELINE_NAME}
  metadata:
    name: ${PIPELINE_NAME}
    namespace: ${PIPELINE_NAMESPACE}
  spec:
    appRef:
      apiVersion: helm.toolkit.fluxcd.io/v2beta1
      kind: HelmRelease
      name: ${APP_NAME}
    environments:
    - name: dev
      targets:
        - namespace: ${APP_NAMESPACE}
          clusterRef:
            kind: GitopsCluster
            name: ${ DEV_CLUSTER.NAME}
            namespace: ${DEV_CLUSTER.NAMESPACE}
    - name: stage
      targets:
        - namespace: ${APP_NAMESPACE}
          clusterRef:
            kind: GitopsCluster
            name: QA
            namespace: flux-system
    - name: prod
      targets:
        - namespace: ${APP_NAMESPACE}
          clusterRef:
            kind: GitopsCluster
            name: prod
            namespace: flux-system
  promotion:
    pull-request:
      branch: ${APP_REPO_BRANCH}
      secretRef:
        name: ${GIT_TOKEN_SECRET}
      url: ${APP_REPO_URL}

[enekofb]

it feels that to add a type field to params would makes sense

weave-gitops-enterprise/charts/mccp/crds/clustertemplates.weave.works_gitopstemplates.yaml

Line 43 in b61176f

params:

so having

              type:
                default: string
                description: indicates the parameter type
                enum:
                - string
                - gitopscluster 
                type: string

than in the target example would be

apiVersion: clustertemplates.weave.works/v1alpha1
kind: GitOpsTemplate
metadata:
  name: pipeline
  namespace: default
annotations: 
  templates.weave.works/profiles-enabled: true
  templates.weave.works/kustomizations-enabled: true
  templates.weave.works/credentials-enabled: true
labels:
  weave.works/template-type: pipeline
spec:
  description: Creates an application Pipeline
  params:
  #metadata
   - name: DEV_CLUSTER
     description: Development Cluster where the application is deployed
     type: gitopscluster

it is then up to the templates api to resolve the type gitopscluster for the environment ...

btw, there would be also a need to map to the answer to a type
....

[davidstauffer]

Thanks for the input :) Its matching our thoughts and conversations we have for quite some time. Once we wrap up the first initial release of GitOps templates (very soooon) we will start to look at it in more detail.

[foot]

Yes, this would be super cool to have, similarly spec'd out here #1201

Also: How backstage handles hinting the UI more in their templates as inspiration. https://backstage.io/docs/features/software-templates/adding-templates

e.g.

    - title: Choose a location
      required:
        - repoUrl
      properties:
        repoUrl:
          title: Repository Location
          type: string
          ui:field: RepoUrlPicker
          ui:options:
            allowedHosts:
              - github.com

To declare how the field should be rendered in the UI, + options to it,

[davidstauffer]

One challenge is that we want to keep the syntax simple, using a single language to provide those expressions.
Another option using go-templating syntax: (Provided by Kevin)

apiVersion: capi.weave.works/v1alpha1
kind: CAPITemplate
metadata:
  name: cluster-template
spec:
  description: this is test template 1
  renderType: templating·
  params:
  - name: DYNAMIC_PARAMETER
    description: dynamic parameter
    templateOptions: |-
      {{- range testList }}
      - {{ . | quote }}
      {{- end }}

[steve-fraser]

Hi! One thing that could be really useful is to allow us to define functions similar to a _helpers.tpl file in a helm to allow reuse. There are instances where we come up with a large if then statements, but want to use that across the 15-20 instances across the template.

[foot]

Spike: Try and recreate the functionality of add-app in the template

[bigkevmcd]

I suspect this is still needed, we are looking at what we can do with Backstage Software Templates, and I'd be reticent to reimplement them, but, if we need to we need to.