RHIDP-9088: Enhance Developer Experience with Orchestrator

assemblies/assembly-automating-workflow-deployment-with-orchestrator.adoc

@@ -0,0 +1,39 @@
+:_mod-docs-content-type: ASSEMBLY
+
+[id="assembly-automating-workflow-deployment-with-orchestrator"]
+= Automating workflow deployment with Orchestrator
+
+The Orchestrator plugin for {product} ({product-very-short}) automates the software development lifecycle (SDLC) for serverless workflows. By using the Orchestrator software templates, you bootstrap a complete serverless workflow project that includes a fully operational Git repository, deployment configurations, and automated Continuous Integration (CI) and Continuous Deployment (CD) pipelines.
+
+The Orchestrator plugin integrates these components:
+
+{product-very-short} Helm chart:: Installs the {product-very-short} Orchestrator.
+
+Tekton or Red Hat OpenShift Pipelines:: Manages the `Kubernetes-native` CI pipeline to build images.
+
+ArgoCD or Red Hat OpenShift GitOps:: Manages the CD pipeline to deploy the workflow on the {product-very-short} instance.
+
+Quay.io:: Stores the container images generated by the pipelines.
+
+OpenShift Serverless Logic operator:: Implements serverless workflow specifications
+
+// installing Orchestrator software templates
+include::modules/orchestrator/proc-installing-orchestrator-software-templates.adoc[leveloffset=+1]
+
+// installing the Orchestrator Software Templates Infra chart
+include::modules/orchestrator/proc-installing-the-orchestrator-software-templates-infra-chart.adoc[leveloffset=+2]
+
+// installing the Orchestrator Software Templates chart
+include::modules/orchestrator/proc-installing-the-orchestrator-software-templates-chart.adoc[leveloffset=+2]
+
+// creating a serverless workflow project
+include::modules/orchestrator/proc-creating-a-serverless-workflow-project.adoc[leveloffset=+1]
+
+// bootstrapping GitOps resources and triggering pipelines
+include::modules/orchestrator/proc-bootstrapping-gitops-resources-and-triggering-pipelines.adoc[leveloffset=+1]
+
+// verifying the deployment
+include::modules/orchestrator/proc-verifying-the-deployment.adoc[leveloffset=+1]
+
+// troubleshooting the CI/CD pipeline
+include::modules/orchestrator/proc-troubleshooting-the-pipelines.adoc[leveloffset=+1]

modules/orchestrator/proc-bootstrapping-gitops-resources-and-triggering-pipelines.adoc

@@ -0,0 +1,42 @@
+:_mod-docs-content-type: PROCEDURE
+
+[id="proc-bootstrapping-gitops-resources-and-triggering-pipelines_{context}"]
+= Bootstrapping GitOps resources and triggering pipelines
+
+After running the template and creating the source code repository and the GitOps repository, the CI pipeline does not start immediately. You must manually bootstrap the GitOps resources through the generated repository to trigger the pipeline.
+
+.Procedure
+
+. Open the generated *GitOps repository*.
+
+. Clone the repository and navigate to the `bootstrap` directory:
++
+[source,terminal,subs="attributes+"]
+----
+$ git clone https://<git_host>/<org_name>/<repo_name>.git
+cd <repo_name>/bootstrap
+----
+
+. Open `${{values.workflowId}}-argocd-repo.yaml` and replace the `__REPLACE_SSH_PRIVATE_KEY__` string with your SSH private key.
+
+. Apply the manifests to the cluster:
++
+[source,bash]
+----
+$ kubectl apply -f .
+----
++
+Applying these manifests triggers the following automated sequence:
+
+.. CI Pipeline (Tekton): Builds the workflow image and pushes it to your `Quay.io` registry.
+.. CD Pipeline (ArgoCD): Deploys the serverless workflow manifests to the cluster.
++
+[NOTE]
+====
+If you are not logged into the repository, provide a personal access token (PAT) in the clone URL. The token must have the necessary permissions to access the repository.
+
+[source,terminal,subs="attributes+"]
+----
+$ git clone https://token:<PAT>@${{ values.gitHost }}/${{ values.orgName }}/${{ values.repoName }}.git # Replace <PAT> with your personal access token.
+----
+====

modules/orchestrator/proc-creating-a-serverless-workflow-project.adoc

@@ -0,0 +1,59 @@
+:_mod-docs-content-type: PROCEDURE
+
+[id="proc-creating-a-serverless-workflow-project_{context}"]
+= Creating a serverless workflow project
+
+Use the Orchestrator software templates to generate a project with workflow definitions, `Kustomize` configurations, and CI/CD pipelines.
+
+.Prerequisites
+
+* You have installed `orchestrator-software-templates-infra` and `orchestrator-software-templates` Helm charts to enable templates.
+
+* You have installed {product-very-short} and the Orchestrator plugin by using the Helm chart.
+
+* You have a `Quay.io` organization and repository to store the workflow image.
+
+* You have a GitHub personal access token with repository creation permissions.
+
+* You have configured a GitOps secret for the target cluster.
+
+[NOTE]
+====
+Set the target namespace for both the pipeline and the workflow to the {product-very-short} namespace (for example, `rhdh`).
+====
+
+.Procedure
+
+. Prepare the image registry. Before creating the template, configure the target repository in `Quay.io`.
+
+.. Log in to your `Quay.io` organization (for example, `orchestrator-testing`).
+
+.. Create a new repository (for example, `serverless-workflow-demo`).
+
+.. Add robot account permissions to the repository settings.
+
+. Open the {product} Catalog.
+
+. Select the *Basic workflow bootstrap project* template and click *Launch Template*.
+
+. Follow the template form to enter required details, including the GitHub organization, source code repository name, and a unique Workflow ID.
+
+. Select *Tekton with ArgoCD* as the CI/CD method to generate GitOps resources.
+
+. Set the *Workflow Namespace* to `rhdh` and the *GitOps Namespace* to `orchestrator-gitops`.
+
+. Enter your `Quay.io` registry details, and click *Review*.
+
+. Click *Create*. The system generates the source code repository and the GitOps configuration repository.
+
+. Optional: Enable persistence and provide database connection details if the workflow requires a database schema.
+
+The system creates two repositories:
+
+. Source code repository: Contains the actual serverless workflow project.
+. GitOps repository: Contains GitOps configurations, Tekton pipeline templates, and bootstrap instructions.
+
+[role="_additional-resources"]
+.Additional resources
+
+* {orchestrator-book-link}#assembly-building-and-deploying-serverless-workflows[Building and deploying serverless workflows]

modules/orchestrator/proc-installing-orchestrator-software-templates.adoc

@@ -0,0 +1,16 @@
+:_mod-docs-content-type: PROCEDURE
+
+[id="proc-installing-orchestrator-software-templates_{context}"]
+= Installing Orchestrator software templates
+
+You must install two additional Helm charts to enable software templates on {product-very-short}.
+
+.Prerequisites
+
+* You have installed the `orchestrator-infra` chart.
+* You have installed {product-very-short} and the Orchestrator plugin by using the Helm chart.
+
+.Procedure
+
+. Install the `orchestrator-software-templates-infra` chart.
+. Install the `orchestrator-software-templates` chart.

modules/orchestrator/proc-installing-the-orchestrator-software-templates-chart.adoc

@@ -0,0 +1,44 @@
+:_mod-docs-content-type: PROCEDURE
+
+[id="proc-installing-the-orchestrator-software-templates-chart_{context}"]
+= Installing the Orchestrator Software Templates chart
+
+The `orchestrator-software-templates` chart loads the actual software templates into your {product-very-short} instance. This allows users to select workflow templates from the {product-very-short} Catalog.
+
+.Prerequisites
+
+* You have installed the `orchestrator-software-templates-infra` chart for deploying OpenShift Pipelines (Tekton) operator and OpenShift GitOps (ArgoCD) operator in the same namespace as {product-very-short}.
+
+* You have labeled the {product-very-short} namespace to enable GitOps sync:
++
+[source,terminal]
+----
+$ oc label ns rhdh rhdh.redhat.com/argocd-namespace=true
+----
+
+* You have created a secret named `orchestrator-auth-secret` in the {product-very-short} namespace containing the following keys:
+
+** `BACKEND_SECRET`: Backend authentication secret
+** `K8S_CLUSTER_TOKEN`: Kubernetes cluster token
+** `K8S_CLUSTER_URL`: Kubernetes cluster URL
+** `GITHUB_TOKEN`: GitHub access token (optional)
+** `GITHUB_CLIENT_ID`: GitHub OAuth client ID (optional)
+** `GITHUB_CLIENT_SECRET`: GitHub OAuth client secret (optional)
+** `GITLAB_HOST`: GitLab host URL (optional)
+** `GITLAB_TOKEN`: GitLab access token (optional)
+** `ARGOCD_URL`: ArgoCD server URL (optional)
+** `ARGOCD_USERNAME`: ArgoCD username (optional)
+** `ARGOCD_PASSWORD`: ArgoCD password (optional)
+
+.Procedure
+. Install the software templates chart:
++
+[source,terminal]
+----
+$ helm repo add redhat-developer https://redhat-developer.github.io/rhdh-chart
+$ helm install my-orchestrator-templates redhat-developer/orchestrator-software-templates --version 0.2.0
+----
+
+. Follow the post-installation notes to create a custom `values.yaml` file to update your {product-very-short} configuration.
+
+. Make sure the templates appear in the {product-very-short} UI under the *Create* sidebar menu.

modules/orchestrator/proc-installing-the-orchestrator-software-templates-infra-chart.adoc

@@ -0,0 +1,38 @@
+:_mod-docs-content-type: PROCEDURE
+
+[id="proc-installing-the-orchestrator-software-templates-infra-chart_{context}"]
+= Installing the Orchestrator Software Templates Infra chart
+
+The `orchestrator-software-templates-infra` chart installs the Custom Resource Definitions (CRDs) and operators for Tekton (OpenShift Pipelines) and ArgoCD (OpenShift GitOps). These are required to handle the CI/CD automation for serverless workflows.
+
+.Prerequisites
+
+* You have `cluster-admin` privileges.
+* You have installed the Helm CLI.
+* You have included the following required plugins:
+
+** `backstage-plugin-scaffolder-backend-module-github-dynamic`
+** `backstage-plugin-scaffolder-backend-module-gitlab-dynamic`
+** `backstage-plugin-kubernetes-backend-dynamic`
+** `backstage-plugin-kubernetes`
+** `backstage-community-plugin-tekton`
+** `backstage-community-plugin-redhat-argocd`
+** `roadiehq-backstage-plugin-argo-cd-backend-dynamic`
+** `roadiehq-scaffolder-backend-argocd-dynamic`
+
+.Procedure
+
+. Install the infrastructure chart:
++
+[source,terminal,subs="attributes+"]
+----
+$ helm install <release_name> redhat-developer/redhat-developer-hub-orchestrator-infra
+----
+
+. Verify the installation:
++
+[source,terminal]
+----
+$ helm test redhat-developer-hub-orchestrator-infra
+----
+

modules/orchestrator/proc-troubleshooting-the-pipelines.adoc

@@ -0,0 +1,39 @@
+:_mod-docs-content-type: PROCEDURE
+
+[id="proc-troubleshooting-the-pipelines_{context}"]
+= Troubleshooting the CI/CD pipeline
+
+Deployment failures involve plugin visibility, running pipeline, or resource synchronization.
+
+. Visibility issues
+
+Missing Orchestrator plugin:: If Orchestrator features do not appear in {product-very-short}, make sure you have updated the {product-very-short} Helm chart with the required plugins.
+
+Software templates not appearing:: Make sure the `orchestrator-software-templates` chart is installed and the `orchestrator-auth-secret` exists in the correct namespace.
+
+. Pipeline failure (CI)
+
+Build or push issues:: Check the *CI* tab in the {product-very-short} Catalog.
++
+If the pipeline status does not appear in the {product} console, check the CI status directly in the source code repository (GitHub or GitLab).
++
+If the Tekton pipeline fails during the build or push stages:
+
+* Verify that your `Quay.io` robot account has Write permissions.
+
+* Make sure the `docker-registry-credentials` secret is present in the `sonataflow-infra` namespace.
+
+. Resource visibility and Sync issues (CD)
+
+Pipeline succeeds but workflows are missing:: If the CI pipeline succeeds but the workflow does not appear in the *CD* tab:
+
+* Make sure the target namespace is labeled for ArgoCD:
++
+[source,terminal]
+----
+$ oc label ns sonataflow-infra rhdh.redhat.com/argocd-namespace=true
+----
+
+* Make sure the ArgoCD ServiceAccount has the required permissions to manage resources in the `sonataflow-infra` namespace.
+
+ArgoCD sync failure:: If resources appear but stay in an `OutOfSync` state, click *Refresh* in the ArgoCD UI or make sure the *AppProject* exists in the `orchestrator-gitops` namespace.

modules/orchestrator/proc-verifying-the-deployment.adoc

@@ -0,0 +1,22 @@
+:_mod-docs-content-type: PROCEDURE
+
+[id="proc-verifying-the-deployment_{context}"]
+= Verifying the deployment
+
+After bootstrapping the GitOps resources, verify the status of the pipelines in the {product-very-short} component catalog.
+
+.Verifying the CI pipeline
+
+.. Open the Source Code Repository component in the {product-very-short} Catalog (for example, `onboardings`).
+
+.. Click the *CI* tab and make sure the pipeline run shows a status is `Succeeded`. This confirms that the image was built and pushed to the Quay registry.
+
+.. If the pipeline status does not appear in the {product} console, check the CI status directly in the source code repository (GitHub or GitLab).
+
+.. If the pipeline failed, click the *run* name to view the logs and identify build errors.
+
+.Verifying the CD pipeline
+
+.. Open the GitOps Resources Repository component in the Catalog (for example, `onboarding-gitops`).
+
+.. Click the *CD* tab and make sure the Kubernetes resources are synced and healthy. This confirms that ArgoCD deployed the workflow to the cluster.

titles/orchestrator/master.adoc

@@ -20,6 +20,8 @@ include::assemblies/assembly-install-rhdh-orchestrator-plugin-in-an-air-gapped-e
 
 include::assemblies/assembly-building-and-deploying-serverless-workflows.adoc[leveloffset=+1]
 
+include::assemblies/assembly-automating-workflow-deployment-with-orchestrator.adoc[leveloffset=+1]
+
 include::assemblies/assembly-orchestrator-troubleshooting-serverless-workflows.adoc[leveloffset=+1]
 
 include::assemblies/assembly-orchestrator-plugins-technical-appendixes.adoc[leveloffset=+1]