Skip to content

🌱 Add broken link check: pr and scheduled #966

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions .github/workflows/pr-link-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
name: Check Links In Pull Requests

on:
pull_request:
types: [opened, edited, reopened, synchronize, ready_for_review]
workflow_call:

permissions: {}

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
check-links-pr:
runs-on: ubuntu-latest

steps:
- name: Clone repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
fetch-depth: 0
ref: ${{github.event.pull_request.head.ref}}
repository: ${{github.event.pull_request.head.repo.full_name}}

- name: Checkout base branch
run: git checkout ${{ github.event.pull_request.base.ref }}

- name: Get list of changed Markdown files
id: changed-files
run: |
git diff --name-only origin/${{ github.event.pull_request.base.ref }}...${{ github.head_ref }} -- '*.md' > changed-files.txt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
git diff --name-only origin/${{ github.event.pull_request.base.ref }}...${{ github.head_ref }} -- '*.md' > changed-files.txt
git diff --name-only "origin/${{ github.event.pull_request.base.ref }}...${{ github.head_ref }}" -- "*.md" > changed-files.txt

cat changed-files.txt
if [ -s changed-files.txt ]; then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if [ -s changed-files.txt ]; then
if [[ -s "changed-files.txt" ]]; then

echo "Changed md files found"
echo "foundFiles=true" >> $GITHUB_ENV
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
echo "foundFiles=true" >> $GITHUB_ENV
echo "foundFiles=true" >> "${GITHUB_ENV}"

fi

- name: Switch to PR branch
run: git checkout ${{ github.head_ref }}

- name: Check links in changed files
if: env.foundFiles == 'true'
uses: lycheeverse/lychee-action@f796c8b7d468feb9b8c0a46da3fac0af6874d374 # v2.2.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
failIfEmpty: false
args: |
--no-progress
--github-token
$(cat changed-files.txt | tr '\n' ' ')

- name: Provide helpful failure message
if: failure()
run: |
echo "::error::Link check failed! Please review the broken links reported above."
echo ""
echo "If certain links are valid but fail due to:"
echo "- CAPTCHA challenges"
echo "- IP blocking"
echo "- Authentication requirements"
echo "- Rate limiting"
echo ""
echo "Consider adding them to .lycheeignore to bypass future checks."
echo "Format: Add one URL pattern per line"
exit 1
47 changes: 47 additions & 0 deletions .github/workflows/scheduled-link-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: Scheduled Check Links

on:
workflow_dispatch:
schedule:
- cron: "0 0 1 * *"
repository_dispatch:
# run manually
types: [check-links]
workflow_call:

permissions: {}

jobs:
check-links:
runs-on: ubuntu-latest

permissions:
contents: read
issues: write

steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

- name: Link Checker
id: linkcheck
uses: lycheeverse/lychee-action@f796c8b7d468feb9b8c0a46da3fac0af6874d374 # v2.2.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
args: |
--user-agent "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0"
--root-dir "$(pwd)/"
--fallback-extensions "md"
--github-token
"./**/*.md"
output: /tmp/lychee_output.md
fail: false

- name: Create Issue From File
if: steps.linkcheck.outputs.exit_code != 0
uses: peter-evans/create-issue-from-file@e8ef132d6df98ed982188e460ebb3b5d4ef3a9cd # v5.0.1
with:
title: Link Checker Report
content-filepath: /tmp/lychee_output.md
labels: |
kind/bug
3 changes: 3 additions & 0 deletions .lycheeignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
https://jenkins.nordix.org/log/GHPRB/
https://jenkins.nordix.org/ghprbhook/
https://prow.apps.test.metal3.io/hook
17 changes: 8 additions & 9 deletions jenkins/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ easier in certain cases:
commenting:
* **/cherry-pick release-x.y** where `release-x.y` is any supported release
branch, where the backport PR will be opened by the
[cherry picker bot](https://github.com/metal3-cherrypick-bot).
[metal3-io-bot](https://github.com/metal3-io-bot).
1. `transfer-issue` plugin can be used for transferring issues across
repositories in the same GitHub organization by commenting:
* **/transfer-issue target-repo-name** on the issue. One simple example
Expand All @@ -72,7 +72,7 @@ branch to be able to open a PR against it in the future.
## Cloud Resources cleanup

There is a Jenkins
[main job](https://jenkins.nordix.org/view/Metal3%20Periodic/job/metal3_daily_main_integration_tests_cleanup/)
[main job](https://jenkins.nordix.org/view/Metal3%20Periodic/job/metal3_periodic_resource_cleanup/)
that cleans up all the leftover VMs from [CityCloud](https://www.citycloud.com/)
every 6 hours which has failed to be deleted at the end of v1alphaX/v1betaX
integration tests.
Expand All @@ -91,7 +91,7 @@ trusted.
Pods, CRDs logs are collected at the end of each Jenkins job run and
archived so that they can be later used for debugging purposes. You can
find the archived logs under the "*Build artifacts*" section of the
[job](https://jenkins.nordix.org/view/Metal3/job/metal3_bmo_main_integration_test_ubuntu/).
[job](https://jenkins.nordix.org/view/Metal3%20Periodic/job/metal3-periodic-centos-e2e-integration-test-main/).
Please note that the logs will be removed 30 days after creation or after 100
subsequent job runs, whichever occurs first.

Expand Down Expand Up @@ -132,15 +132,14 @@ stored in
[metal3-io/project-infra](https://github.com/metal3-io/project-infra/tree/main/jenkins/jobs)
repository. In a nutshell, pipelines defines sequence of steps to be executed.
Each step can run a script or perform something else. For example,
[integration_tests.pipeline](https://github.com/metal3-io/project-infra/blob/main/jenkins/jobs/integration_tests.pipeline)
[dev_env_integration_tests.pipeline](https://github.com/metal3-io/project-infra/blob/main/jenkins/jobs/dev_env_integration_tests.pipeline)
executes following scripts:

1. [clones](https://github.com/metal3-io/project-infra/blob/0a6cc3f9f8592914a316c27ea2411ccb48aba7c3/jenkins/jobs/integration_tests.pipeline#L65)
1. [clones](https://github.com/metal3-io/project-infra/blob/1e8d2111a33dc95134b6f546d76f19be4a637e3d/jenkins/jobs/dev_env_integration_tests.pipeline#L56)
git repository
1. [jenkins/scripts/integration_test.sh](https://github.com/metal3-io/project-infra/blob/main/jenkins/scripts/integration_test.sh)
1. [jenkins/scripts/fetch_logs.sh](https://github.com/metal3-io/project-infra/blob/main/jenkins/scripts/fetch_logs.sh)
1. [jenkins/scripts/integration_test_clean.sh](https://github.com/metal3-io/project-infra/blob/main/jenkins/scripts/integration_test_clean.sh)
1. [jenkins/scripts/integration_delete.sh](https://github.com/metal3-io/project-infra/blob/main/jenkins/scripts/integration_delete.sh)
1. [jenkins/scripts/dynamic_worker_workflow/dev_env_integration_tests.sh](https://github.com/metal3-io/project-infra/blob/main/jenkins/scripts/dynamic_worker_workflow/dev_env_integration_tests.sh)
1. [jenkins/scripts/dynamic_worker_workflow/fetch_logs.sh](https://github.com/metal3-io/project-infra/blob/main/jenkins/scripts/dynamic_worker_workflow/fetch_logs.sh)
1. [jenkins/scripts/dynamic_worker_workflow/run_clean.sh](https://github.com/metal3-io/project-infra/blob/main/jenkins/scripts/dynamic_worker_workflow/run_clean.sh)

### GitHub Pull Request Builder, a.k.a. ghprb

Expand Down
2 changes: 1 addition & 1 deletion jenkins/scripts/bare_metal_lab/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,6 @@ the same host and interfere with each other. This means that if there is already
one build job running in the BML, a new one will not start before the first has
finished. Github won't show the usual *Details* link for this specific run but
build status can be checked from the
[Jenkins dashboard](https://jenkins.nordix.org/job/metal3_metal3io_project_infra_bml_integration_tests_centos/)
[Jenkins dashboard](https://jenkins.nordix.org/job/metal3-bml-integration-test-centos/)
where the build will be scheduled and stay in pending at this time.
Once the build starts, the status will be updated with a link.
8 changes: 3 additions & 5 deletions prow/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,11 +49,9 @@ instance:
Prow.

In addition to this, we rely on a GitHub bot account
([metal3-io-bot](https://github.com/metal3-io-bot), owner of the GitHub token)
and a separate GitHub bot
[metal3-cherrypick-bot](https://github.com/metal3-cherrypick-bot), for cherry
picking pull requests. A webhook must be configured in GitHub to send events to
Prow and a DNS record must be configured so that
([metal3-io-bot](https://github.com/metal3-io-bot), owner of the GitHub token).
A webhook must be configured in GitHub to
send events to Prow and a DNS record must be configured so that
<https://prow.apps.test.metal3.io> points to the IP where Prow can be accessed.

The DNS records are managed by CNCF for us. Any changes to them can be done
Expand Down