Skip to content

docs: added initital first draft of the complete epic workflow automation guide #1890

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added assets/action-pointer.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/actions.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions docs/workflows-guide/_section.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
title: Github action workflow guide
weight: 20
---
323 changes: 323 additions & 0 deletions docs/workflows-guide/available-global-jobs.md

Large diffs are not rendered by default.

93 changes: 93 additions & 0 deletions docs/workflows-guide/global-replicator-job.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
title: 'The Global Replicator Workflow Job'
weight: 70
---

By simply extending the pattern of our workflow automations, we introduced a custom workflow called `Global Replicator` with a simple of goal of replicating generic files in different repositories from a single source.

The Global Replicator Workflow provides the flexibility of having a subset of reusable workflows shared across multiple repositories, so you don’t have to maintain the same workflows in every project manually.

Once a repository opts in to a specific replicator job (`via topics`), it will automatically receive updates whenever that workflow file/files in the replicator job changes. This means your repo always stays in sync with the latest improvements or fixes, no extra work required.

The `global replicator` workflow consists of multiple jobs such as `Replicate Code of Conduct in all repositories` job configuration which is responsible for propagating the `Code of Conduct.md` file across all relevant repositories.

There are two categories of jobs in the `global replicator workflow` which are `Default workflow jobs` and `Topic-to-Replicator workflow jobs`.

## Default Workflow Jobs

These jobs handle the distribution of **organization-wide files**, such as `CODE_OF_CONDUCT.md` or `CONTRIBUTING.md`, which are essential for maintaining consistency and compliance across all repositories.

For example, when a project joins the AsyncAPI organization, it must adopt our organization-wide policies, such as the [Code of Conduct](https://github.com/asyncapi/.github/blob/master/CODE_OF_CONDUCT.md). But with over 50 repositories, how do we ensure each one always has the most up-to-date version?

That’s where the `Replicate Code of Conduct in all repositories` job configuration in the `global replicator` comes in which is considered as a default job, which main purpose is to automate the propagation of the`CODE_OF_CONDUCT.md` file across all relevant repositories.

Here’s a snippet of the job from the replicator workflow:

```
replicate_coc:
if: startsWith(github.repository, 'asyncapi/')
name: Replicate Code of Conduct in all repositories
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Replicating file
uses: derberg/manage-files-in-multiple-repositories@beecbe897cf5ed7f3de5a791a3f2d70102fe7c25
with:
github_token: ${{ secrets.GH_TOKEN }}
patterns_to_include: CODE_OF_CONDUCT.md
repos_to_ignore: shape-up-process,glee-hello-world,saunter
committer_username: asyncapi-bot
committer_email: [email protected]
commit_message: "ci: update of files from global .github repo"
bot_branch_name: bot/update-files-from-global-repo
```

This job uses the `derberg/manage-files-in-multiple-repositories` action to replicate files across all repositories, except for those explicitly excluded via the `repos_to_ignore` input. It ensures every project receives a local and up-to-date copy of the `CODE_OF_CONDUCT.md`, instead of just linking to it, which might be ignored or missed by contributors and automation.

## Topic-to-Replicator Jobs

Topic-to-Replicator Workflow jobs are tailored to **specific use cases** or **project types** such as language-specific test workflows that are only needed in a subset of repositories.

For instance, when a new Go-based project joins the AsyncAPI ecosystem, we may want to enable our centralized Go PR testing workflow for that repository. However, this workflow should **only** be applied to Go projects.

To support this, the `global replicator` includes jobs that uses the `topics_to_include` filter. Repositories must have a specified global workflow topic to be eligible for the workflow:

```
replicate_go_workflows:
if: startsWith(github.repository, 'asyncapi/')
name: Replicate workflows for Go projects
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Replicating file
uses: derberg/manage-files-in-multiple-repositories@beecbe897cf5ed7f3de5a791a3f2d70102fe7c25
with:
github_token: ${{ secrets.GH_TOKEN }}
patterns_to_include: .github/workflows/if-go-pr-testing.yml
topics_to_include: golang
committer_username: asyncapi-bot
committer_email: [email protected]
commit_message: "ci: update of files from global .github repo"
bot_branch_name: bot/update-files-from-global-repo

```

In this setup, **only repositories tagged with the `golang` topic** will receive the `go PR testing workflow`. This gives maintainers the flexibility to opt into shared workflows without affecting repositories that don't need them.

Below is a list of workflows with their corresponding topics and descriptions. By including each topic in your repository in AsyncAPI organisation, you will subscribe to get a workflow or a group of workflows that perform a specific task.

| Topic to include | Workflows you will get | Description |
| --- | --- | --- |
| `get-workflows-validation` | [validate-workflow-schema](.github/workflows/validate-workflow-schema.yml) | Validates the YAML schema of a workflow files in your repository
| `golang` | [if-go-pr-testing](.github/workflows/if-go-pr-testing.yml) | Compiles and tests Go code using multiple versions of Go
| `nodejs` | [if-nodejs-pr-testing](.github/workflows/if-nodejs-pr-testing.yml) | Builds and tests Node.js projects using multiple Node.js versions
| `get-global-node-release-workflows` | [if-nodejs-release](.github/workflows/if-nodejs-release.yml), [if-nodejs-version-bump.yml](.github/workflows/if-nodejs-version-bump.yml) , [bump.yml](.github/workflows/bump.yml) | Fetches and publishes Node.js release information to the project's website
| `get-global-releaserc` | [.releaserc](.github/workflows/.releaserc) | Fetches release configuration files from a remote repository andUpdates the documentation of a project using `generate:assets` command (if it exists) when the commit starts with `docs makes them available to other workflows
| `get-global-docs-autoupdate` | [update-docs-on-docs-commits](.github/workflows/update-docs-on-docs-commits.yml) | Updates the documentation of a project using `generate:assets` command (if it exists) when the commit starts with `docs:`
| `get-global-prettierignore` | [replicate-prettierignore-file](./.github/workflows/global-replicator.yml#L202C3-L218C61) | Replicates `.prettierignore` file in the repository which has this topic.

## Why This Matters

Having local copies of key files like the Code of Conduct ensures better visibility and enforcement. More importantly, it removes the burden from maintainers to keep these files in sync manually. Updates made in the `.github` repo can be automatically and reliably distributed across all relevant projects.
77 changes: 77 additions & 0 deletions docs/workflows-guide/global-workflow-distribution.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
---
title: 'Global Workflows Distribution'
weight: 60
---

Once a global workflow is created or exists within the `.github` repository, the next step is to make them available to other repositories across the organization. This process is referred to as **Workflow Distribution**.

The goal of the distribution is simple: `enable any repository to consume the predefined workflow with minimal setup`. Instead of copying and pasting the same workflow logic into multiple repositories, each project can simply reference a shared workflow directly from the `.github` repository.

To streamline this process, we introduced two custom workflows: `global replicator` and `global remover`. These workflows automate the distribution and removal of global workflow files across all repositories based on predefined conditions.

## How Workflow Distribution Run

If you opt in to get a generic workflow like the `Replicate workflows for Go projects` by adding the `golang` topic to your repository, the workflow doesn’t just magically appear in your Go project. For that to happen, the `global replicator` workflow distributor needs to be triggered.

There are two ways the global workflow distributor can be triggered:

### Automatic Workflow Distribution:

Workflows are distributed automatically **only** when there’s a push to the `.github` repository’s `master` branch.

So, even if you’ve opted in for the Golang workflow by adding the `golang` topic, it won’t be distributed to your repo just yet, not until there’s an actual update pushed to the `.github` repo.

```
on:
push:
branches: [master]
paths:
- ".github/workflows/**"
- "CODE_OF_CONDUCT.md"
- "CONTRIBUTING.md"
```

As shown above, the distribution workflow only triggers if changes are made in specific paths: the `.github/workflows` directory, `CODE_OF_CONDUCT.md`, or `CONTRIBUTING.md`. If no updates land in those files, the distributor won't kick in.

### Manual Workflow Distribution:

The second option is manual and it's only available to `.github` repository maintainers.

Say you've added the `golang` topic to your Go repository and want the Golang workflow applied immediately. In that case, you’ll need to reach out to one of the `.github` repo maintainers and ask them to manually trigger the **global workflow to rule them all** via the **Actions** tab in the `.github` repo.

From there, the maintainer can input the specific repository name and run the distribution for your project manually—no need to wait for a push to the `master` branch.

## Accepting Global Workflows

The real heroes behind the global workflows are the AsyncAPI bots: `Eve` and `Chan`. These bots are responsible for pushing changes and opening pull requests across all repositories when workflows are distributed.

But just opting in to a generic global workflow doesn’t automatically guarantee that the workflow will show up in your repository. A few things need to be in place first:

### Step 1 - Add the AsyncAPI Bots Team as Collaborators

Every repository within the AsyncAPI organization that wants to accept global files from the `.github` repository **must** add the [`asyncapi-bots`](https://github.com/asyncapi-bot) team as a collaborator with at least **maintainer** access.

In other words, if you’re maintaining a repository in the AsyncAPI GitHub org and want to take advantage of the global workflow distribution, you’ll need to make sure the `asyncapi-bots` team is added under your repo’s settings, just like in the example below:

### Step 2 - Add Bot as Part of Code-owners

One of the bots in the `asyncapi-bots` team is [`asyncapi-bot-eve`](https://github.com/asyncapi-bot-eve), who handles pull request approvals coming from the other bot, `Chan`.

To make this work smoothly, you need to add `Eve` as a code owner in your repository. This allows the bot to automatically review and approve any PRs initiated by `Chan`.

### Step 3 - Trigger Initial Workflow Automation

Adding the bots and setting up codeowners isn’t enough by itself. You still need to trigger the initial global workflow distribution.

To do this, reach out to a maintainer of the `.github` repository and ask them to manually trigger the automation. They can do this by going to the **Actions** tab in the `.github` repo, selecting the `Global workflow to rule them all` action, clicking **Run workflow**, and providing your repository name.

This kicks off the bot process and pushes all the global files that your repo has opted into, based on its topics.

### Step 4 - Merge Pull Requests

Once the automation is triggered, you’ll see a few pull requests open in your repo, containing the files distributed by the global workflows. These PRs need to be manually reviewed and merged by your repository’s maintainers.

By default, you'll receive PRs to add the `CODE_OF_CONDUCT.md` and `CONTRIBUTING.md` files, unless your repo is listed in the `repos_to_ignore` section of the global workflow job.

Other workflow-related files will only be included if your project has the right workflow topics set **before** the workflow was triggered.

14 changes: 14 additions & 0 deletions docs/workflows-guide/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
title: 'Introduction'
weight: 5
---

## AsyncAPI Workflow Guide

The AsyncAPI workflow guide helps contributors and maintainers understand how shared GitHub Actions workflows are used and managed across repositories within the AsyncAPI ecosystem. It explains the purpose behind each replicated workflow, how to opt into them, and the standards they enforce.

By centralizing and documenting these workflows, we ensure consistency, reduce duplicated effort, and streamline CI/CD practices across all AsyncAPI projects.

The guide empowers contributors to make informed decisions when customizing or adopting workflows, while also supporting maintainers in keeping their repositories aligned with community best practices.

> For a broader understanding of contributing to the AsyncAPI Initiative, refer to our [contributing guidelines](https://github.com/asyncapi/community/blob/master/CONTRIBUTING.md).
23 changes: 23 additions & 0 deletions docs/workflows-guide/introduction-to-github-repository.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
title: 'Introduction to the .github repository'
weight: 60
---

With over `90%` of all available workflows in all repositories across the organization being considered as global workflows, managing them will be a nightmare without proper centralization of these workflows.

Github has many special repositories. For example, you can create a repository that matches your username, add a README file to it, and all the information in that file will be visible on your Github profile.

The `.github` repository is one of the many special repository available on Github. It serves as the central hub for reusable Github configurations that can be automatically inherited by other repositories within the org, especially repositories without existing of pre-defined `.github` directory which can they act as a fallback for such repositories.

The AsyncAPI [.github](https://github.com/asyncapi/.github) repository serves as the central source for all reusable community health files and global workflow automation. It includes shared GitHub Actions, issue and pull request templates, and metadata for the entire organization. For instance, the profile description you see when visiting [github.com/asyncapi](https://github.com/asyncapi) is powered by the `/profile/README.md` file within this repository.

## Key Function of the `.github` Repository

The `.github` repository lies at the heart of our global workflow strategy. It plays a vital role in ensuring consistency, automation, and maintainability across all AsyncAPI repositories. Below are its primary functions:

| Function | Desctiption |
| ------------------------------- | ---------------------- |
| Shared Github Actions Workflows | Hosts reusable workflows that can be referenced by other repositories using, enabling centralized automation and reducing duplication. This is the foundation of AsyncAPI’s global workflows. |
| Default Community Health Files | Provides default files such as `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`, `ISSUE_TEMPLATE/`, and more. These are automatically applied to any repository in the organization that doesn't define its own versions. |
| Org-wide Metadata | Stores files like `/profile/README.md` to define the org-wide profile page, and `FUNDING.yml` to manage sponsorship links. This centralizes important metadata and presents a unified organizational identity. |
| Centralized Maintainance | Updates to workflows, templates, or documentation only need to be made once in this repository. All consuming repositories automatically reflect the changes, simplifying maintenance and ensuring uniformity across projects. |
22 changes: 22 additions & 0 deletions docs/workflows-guide/local-and-global-worfklows-overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
title: 'Overview of Local vs Global Workflows'
weight: 30
---

`We automate pretty much everything we possibly can.` But that doesn’t mean we spin up an automation for every little thing that comes our way. Why? Because we know automation only works when it’s solving the right problem, not when it’s just adding more noise.

That’s why we categorised our workflows into two main types: **Local Workflows** and **Global Workflows**.

### Local Workflows

Local workflows are types of workflow automations that are repository-specific. This means they live inside a single repository and only handle tasks related to that repo. They don’t care what’s going on elsewhere, they just focus on doing their job in one place.

For example, you might have a local workflow that runs tests on every pull request. So whenever someone opens a PR, the workflow kicks in to run the necessary checks and make sure everything meets the standards set by the maintainers. It’s all scoped to that one repo, and nothing beyond it.

### Global Workflows

Global workflows are automations that are shared across multiple repositories. These workflows allow new projects to instantly benefit from existing automations within the organization, so maintainers don’t have to build everything from scratch.

For example, say you’re donating a new project to the org. That means you’ll need things like a `Code of Conduct` and a `Contributing Guide`. But instead of writing them from scratch, you can just hook into one of our global workflows that handles that for you, it automatically syncs those docs from a central source and keeps them updated whenever changes are made.

That way, you spend less time reinventing the wheel, and more time actually working on your project.
Loading