Optimize CI workflow structure for better efficiency

[mgajek-cern]

Problem

The current Update Documentation workflow has several inefficiencies:

1. Redundant checkouts: Duplicate actions/checkout@v4 in the build job
2. Mixed concerns: Quality checks and deployment logic in single workflow
3. Resource waste: 4 separate runners for simple linting tasks that could run sequentially
4. Slow feedback: PRs must wait for all parallel jobs instead of getting quick validation

Proposed Solution

Split the workflow into focused, efficient workflows:

Quality Checks Workflow (quality-checks.yml)

• Runs on every push/PR
• Single job that runs all linting/validation sequentially
• Faster feedback for contributors
• Combines: markdown lint, file name checks, shell checks, Python checks

E.g.

name: Quality Checks
on: [push, pull_request]
jobs:
  lint-and-validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Setup dependencies
        run: |
          # Install all linting tools in one step
          gem install mdl
          sudo apt-get install -y shellcheck
          pip install -r tools/requirements.txt
      - name: Check markdown
        run: ./tools/check-docs.sh
      - name: Check file names  
        run: ./tools/check-file-names.sh
      - name: Check shell scripts
        run: shellcheck **/*.sh
      - name: Check Python
        run: |
          black --check .
          isort --profile black --filter-files --check .
          flake8 --config tools/.flake8
          mypy --ignore-missing-imports .

Build & Deploy Workflow (build-deploy.yml)

• Runs only on main branch pushes and scheduled
• Handles documentation generation and deployment
• Separated from validation concerns

E.g.

name: Build and Deploy
on:
  schedule:
    - cron: '0 4 * * 1-5'
  push:
    branches: [main]
# ... build and deploy logic

Benefits

• Faster PR feedback - single job vs 4 parallel jobs with setup overhead
• Resource efficiency - reduced runner usage and setup duplication
• Clearer separation - validation vs deployment logic separated
• Easier maintenance - focused workflows are simpler to debug and modify

Implementation

Would involve restructuring .github/workflows/ to replace current workflow with the two focused workflows described above.

[rdimaio]

Resource waste: 4 separate runners for simple linting tasks that could run sequentially

Why is this marked as an inefficiency? We could flip the other way around and say "efficient usage of resources: separate tasks are run in parallel so that CI completes faster". If we have enough free workers, it makes sense to parallelize and split the workload when possible, so that the CI as a whole completes faster to get quicker feedback on the PR

[mgajek-cern]

Resource waste: 4 separate runners for simple linting tasks that could run sequentially

Why is this marked as an inefficiency? We could flip the other way around and say "efficient usage of resources: separate tasks are run in parallel so that CI completes faster". If we have enough free workers, it makes sense to parallelize and split the workload when possible, so that the CI as a whole completes faster to get quicker feedback on the PR

Good point. I was too hasty calling it "inefficient" - if you have free runners and the tasks take meaningful time, parallel is definitely faster for PR feedback. The main benefit I'm after is separating quality checks from the build/deploy pipeline (PRs don't need the full doc build). But within quality checks, parallel vs sequential is worth measuring rather than assuming.
Want to benchmark both approaches to see what actually performs better?

[rdimaio]

Resource waste: 4 separate runners for simple linting tasks that could run sequentially

Why is this marked as an inefficiency? We could flip the other way around and say "efficient usage of resources: separate tasks are run in parallel so that CI completes faster". If we have enough free workers, it makes sense to parallelize and split the workload when possible, so that the CI as a whole completes faster to get quicker feedback on the PR

Good point. I was too hasty calling it "inefficient" - if you have free runners and the tasks take meaningful time, parallel is definitely faster for PR feedback. The main benefit I'm after is separating quality checks from the build/deploy pipeline (PRs don't need the full doc build). But within quality checks, parallel vs sequential is worth measuring rather than assuming. Want to benchmark both approaches to see what actually performs better?

separating quality checks from the build/deploy pipeline (PRs don't need the full doc build)

I agree that this can help. I think we could probably standardize the way we do this across repos so that we always follow the same logic, e.g. in the main repo we have linting done in parallel with the actual tests: https://github.com/rucio/rucio/blob/master/.github/workflows/autotest.yml - maybe in this context it would make sense to sequentially run linting before the tests, so that we don't even begin the test setup in case one of the quicker lint checks fails.

But within quality checks, parallel vs sequential is worth measuring rather than assuming.
Want to benchmark both approaches to see what actually performs better?

Sure, we can. https://github.com/rucio/documentation/actions/metrics/performance this is the performance of the current setup, it takes 5m30s on average, most of it comes from the build step

[mgajek-cern]

I agree that this can help. I think we could probably standardize the way we do this across repos so that we always follow the same logic, e.g. in the main repo we have linting done in parallel with the actual tests: https://github.com/rucio/rucio/blob/master/.github/workflows/autotest.yml - maybe in this context it would make sense to sequentially run linting before the tests, so that we don't even begin the test setup in case one of the quicker lint checks fails.

Good point. Sequential linting with early failure makes sense - if markdown linting fails, no point running the expensive build. Follows the fail-fast principle and aligns with the main repo pattern.

---

Sure, we can. https://github.com/rucio/documentation/actions/metrics/performance this is the performance of the current setup, it takes 5m30s on average, most of it comes from the build step

I'll try to address this issue the next time and set up a minimalistic refactoring of the existing workflow for comparison in a feature branch in the forked repository. We can run both approaches in parallel on a feature branch and gather actual performance data to see which works better in practice. Maybe attach images/screenshots of the timing results as evidence.

[mgajek-cern]

Benchmark Results: Parallel vs Sequential for quality checks only

Tested both approaches across multiple runs:

• Sequential average: ~32.5s
• Parallel average: ~22.7s (NOTE: Commented build and deploy jobs)

Parallel is ~10 seconds faster, though the difference is relatively negligible for these lightweight checks.

• https://github.com/mgajek-cern/documentation/actions/runs/18274555395
• https://github.com/mgajek-cern/documentation/actions/runs/18274555402

---

Real Issue: PRs trigger the expensive build job unnecessarily. While the build doesn't deploy on PRs (deploy only runs on main), PRs still waste time and resources running Docker builds, API generation, and yarn builds that are never used.

documentation/.github/workflows/update_documentation.yml

Lines 96 to 99 in 8819210

	deploy:
	name: Deploy
	needs: build
	if: github.ref == 'refs/heads/main' || github.event_name == 'schedule'

Recommendation:

• Keep quality checks parallel (4 separate runners as-is)
• Separate build/deploy into its own workflow that only runs on push to main and schedule (skip PRs entirely)
• Fix duplicate actions/checkout@v5 in the build job

This eliminates wasteful documentation builds on PRs while keeping fast quality check feedback.

---

Based on these findings, the main value is separating quality checks from build/deploy rather than changing parallel to sequential. Should we proceed with splitting the workflows to skip PR builds or is there a specific reason PRs need to build documentation even though it's not deployed?

@rdimaio, thoughts?