Skip benchmarks in ci when running in fork repositories by Cycloctane · Pull Request #11737 · aio-libs/aiohttp

Cycloctane · 2025-10-29T06:23:54Z

What do these changes do?

Forks cannot use codspeed account, which can make benchmark job in ci fail if aiohttp contributors want to run test workflow on their forks. This pr make unnecessary steps skip if triggered workflow is not in the main repository.

Are there changes in behavior for the user?

aiohttp contributors can now run github action workflows to test changes in their forks without unnecessary errors.

Is it a substantial burden for the maintainers to support this?

Related issue number

Checklist

I think the code is well written
Unit tests for the changes exist
Documentation reflects the changes
If you provide code modification, please add yourself to CONTRIBUTORS.txt
- The format is <Name> <Surname>.
- Please keep alphabetical order, the file is sorted by names.
Add a new news fragment into the CHANGES/ folder

codspeed-hq · 2025-10-29T06:35:22Z

CodSpeed Performance Report

Merging #11737 will not alter performance

_{Comparing Cycloctane:skip-workflows-in-forks (6dc6b3e) with master (72fadb8)}

Summary

✅ 59 untouched

codecov · 2025-10-29T06:57:03Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.74%. Comparing base (72fadb8) to head (6dc6b3e).
⚠️ Report is 180 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #11737   +/-   ##
=======================================
  Coverage   98.74%   98.74%           
=======================================
  Files         127      127           
  Lines       43754    43754           
  Branches     2328     2328           
=======================================
  Hits        43205    43205           
  Misses        389      389           
  Partials      160      160

Flag	Coverage Δ
CI-GHA	`98.61% <ø> (ø)`
OS-Linux	`98.35% <ø> (-0.01%)`	⬇️
OS-Windows	`96.68% <ø> (-0.01%)`	⬇️
OS-macOS	`97.57% <ø> (ø)`
Py-3.10.11	`97.12% <ø> (-0.01%)`	⬇️
Py-3.10.19	`97.62% <ø> (ø)`
Py-3.11.14	`97.81% <ø> (-0.01%)`	⬇️
Py-3.11.9	`97.32% <ø> (-0.01%)`	⬇️
Py-3.12.10	`97.42% <ø> (-0.01%)`	⬇️
Py-3.12.12	`97.92% <ø> (-0.01%)`	⬇️
Py-3.13.9	`98.17% <ø> (-0.01%)`	⬇️
Py-3.14.0	`98.12% <ø> (-0.01%)`	⬇️
Py-3.14.0t	`97.19% <ø> (-0.01%)`	⬇️
Py-pypy3.11.13-7.3.20	`97.42% <ø> (-0.02%)`	⬇️
VM-macos	`97.57% <ø> (ø)`
VM-ubuntu	`98.35% <ø> (-0.01%)`	⬇️
VM-windows	`96.68% <ø> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

webknjaz · 2025-10-29T07:42:09Z

The preferred solution is storing the plain text token fore Codecov. As for codspeed, I haven't seen it failing. Do you have any examples?

FWIW, the repo checks should be done through repo IDs.

.github/workflows/ci-cd.yml

webknjaz

I wonder, though, if benchmarking could still run but just skip uploading the results, conditionally. Could you check?

CHANGES/11737.contrib.rst

.github/workflows/ci-cd.yml

webknjaz · 2025-11-05T12:47:40Z

I wonder, though, if benchmarking could still run but just skip uploading the results, conditionally. Could you check?

@Cycloctane have you been able to check if this is possible?

Cycloctane · 2025-11-16T21:06:10Z

Sorry for the delay. I've checked codspeed docs. Benchmarking can be done separately with pytest-codespeed. But codspeed's github action does not to support skipping uploading.

Codspeed token seems not necessary for public repository. We can safely remove it.

Cycloctane · 2025-11-16T21:28:49Z

pypy test is failing. Because pyo3 has dropped support for pypy3.10.

.github/workflows/ci-cd.yml

webknjaz · 2025-11-21T16:43:59Z

Benchmarking can be done separately with pytest-codespeed. But codspeed's github action does not to support skipping uploading.

Alright, this is blocked on CodSpeedHQ/action#146, then.

Codspeed token seems not necessary for public repository. We can safely remove it.

Sounds good.

webknjaz · 2025-11-21T16:51:59Z

.github/workflows/ci-cd.yml

+    needs:
+    - build-tarball
+    - build-wheels
+    - pre-setup  # transitive, for accessing settings
    runs-on: ubuntu-latest
+    if: >-
+      needs.pre-setup.outputs.upstream-repository-id == github.repository_id


~~Could you move this to a separate PR?~~

Ah, never mind. It's kinda related but the PR title+description+change note should reflect this.

webknjaz · 2025-11-21T16:53:52Z

.github/workflows/ci-cd.yml

      uses: CodSpeedHQ/action@v4
      with:
        mode: instrumentation
-        token: ${{ secrets.CODSPEED_TOKEN }}


Aparrently, they've implemented OIDC two days ago. Perhaps, we could configure it as a part of replacing this token: https://codspeed.io/docs/integrations/ci/github-actions/configuration#oidc-recommended

Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) <wk.cvs.github@sydorenko.org.ua>

webknjaz

I don't remember what I was waiting for. Let's just merge this. Thanks!

patchback · 2026-01-19T17:12:30Z

Backport to 3.13: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply f30f43e on top of patchback/backports/3.13/f30f43e61a015daa81db5220eb52c8201c71ffc9/pr-11737

Backporting merged PR #11737 into master

Ensure you have a local repo clone of your fork. Unless you cloned it
from the upstream, this would be your origin remote.
Make sure you have an upstream repo added as a remote too. In these
instructions you'll refer to it by the name upstream. If you don't
have it, here's how you can add it:
```
$ git remote add upstream https://github.com/aio-libs/aiohttp.git
```

Ensure you have the latest copy of upstream and prepare a branch
that will hold the backported code:

$ git fetch upstream
$ git checkout -b patchback/backports/3.13/f30f43e61a015daa81db5220eb52c8201c71ffc9/pr-11737 upstream/3.13

Now, cherry-pick PR Skip benchmarks in ci when running in fork repositories #11737 contents into that branch:
```
$ git cherry-pick -x f30f43e61a015daa81db5220eb52c8201c71ffc9
```
If it'll yell at you with something like fatal: Commit f30f43e61a015daa81db5220eb52c8201c71ffc9 is a merge but no -m option was given., add -m 1 as follows instead:
```
$ git cherry-pick -m1 -x f30f43e61a015daa81db5220eb52c8201c71ffc9
```
At this point, you'll probably encounter some merge conflicts. You must
resolve them in to preserve the patch from PR Skip benchmarks in ci when running in fork repositories #11737 as close to the
original as possible.

Push this branch to your fork on GitHub:

$ git push origin patchback/backports/3.13/f30f43e61a015daa81db5220eb52c8201c71ffc9/pr-11737

Create a PR, ensure that the CI is green. If it's not — update it so that
the tests and any other checks pass. This is it!
Now relax and wait for the maintainers to process your pull request
when they have some cycles to do reviews. Don't worry — they'll tell you if
any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

patchback · 2026-01-19T17:12:38Z

Backport to 3.14: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply f30f43e on top of patchback/backports/3.14/f30f43e61a015daa81db5220eb52c8201c71ffc9/pr-11737

Backporting merged PR #11737 into master

Ensure you have a local repo clone of your fork. Unless you cloned it
from the upstream, this would be your origin remote.
Make sure you have an upstream repo added as a remote too. In these
instructions you'll refer to it by the name upstream. If you don't
have it, here's how you can add it:
```
$ git remote add upstream https://github.com/aio-libs/aiohttp.git
```

Ensure you have the latest copy of upstream and prepare a branch
that will hold the backported code:

$ git fetch upstream
$ git checkout -b patchback/backports/3.14/f30f43e61a015daa81db5220eb52c8201c71ffc9/pr-11737 upstream/3.14

Now, cherry-pick PR Skip benchmarks in ci when running in fork repositories #11737 contents into that branch:
```
$ git cherry-pick -x f30f43e61a015daa81db5220eb52c8201c71ffc9
```
If it'll yell at you with something like fatal: Commit f30f43e61a015daa81db5220eb52c8201c71ffc9 is a merge but no -m option was given., add -m 1 as follows instead:
```
$ git cherry-pick -m1 -x f30f43e61a015daa81db5220eb52c8201c71ffc9
```
At this point, you'll probably encounter some merge conflicts. You must
resolve them in to preserve the patch from PR Skip benchmarks in ci when running in fork repositories #11737 as close to the
original as possible.

Push this branch to your fork on GitHub:

$ git push origin patchback/backports/3.14/f30f43e61a015daa81db5220eb52c8201c71ffc9/pr-11737

Create a PR, ensure that the CI is green. If it's not — update it so that
the tests and any other checks pass. This is it!
Now relax and wait for the maintainers to process your pull request
when they have some cycles to do reviews. Don't worry — they'll tell you if
any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

webknjaz · 2026-01-19T17:13:34Z

@Cycloctane would you be able to handle the backports, please?

webknjaz · 2026-01-23T13:53:52Z

@rodrigobnogueira perhaps you'd want to help out?

rodrigobnogueira · 2026-01-24T02:21:00Z

@rodrigobnogueira perhaps you'd want to help out?

Let's try 🙏

Forks cannot use codspeed account, which can make benchmark job in ci fail if aiohttp contributors want to run test workflow on their forks. This pr make unnecessary steps skip if triggered workflow is not in the main repository. aiohttp contributors can now run github action workflows to test changes in their forks without unnecessary errors. Co-authored-by: 🇺🇦 Sviatoslav Sydorenko <webknjaz@redhat.com> (cherry picked from commit f30f43e)

rodrigobnogueira · 2026-01-24T04:09:08Z

Done. Having problems running test test_import_time (fixed in PR #11992)

…g in fork repositories (#11991) Co-authored-by: Rui Xi <Cycloctane@outlook.com> Co-authored-by: 🇺🇦 Sviatoslav Sydorenko <webknjaz@redhat.com>

…g in fork repositories (#11990) Co-authored-by: Rui Xi <Cycloctane@outlook.com> Co-authored-by: 🇺🇦 Sviatoslav Sydorenko <webknjaz@redhat.com> Co-authored-by: rodrigo.nogueira <rodrigo.nogueira@prf.gov.br>

Cycloctane requested a review from webknjaz as a code owner October 29, 2025 06:23

Cycloctane requested a review from asvetlov as a code owner October 29, 2025 06:49

psf-chronographer bot added the bot:chronographer:provided There is a change note present in this PR label Oct 29, 2025

Cycloctane changed the title ~~Skip codecov uploads and benchmarks in ci when running in forks~~ Skip codecov uploads and benchmarks in ci when running in fork repositories Oct 29, 2025

Cycloctane changed the title ~~Skip codecov uploads and benchmarks in ci when running in fork repositories~~ Skip benchmarks in ci when running in fork repositories Oct 29, 2025

bdraco reviewed Oct 29, 2025

View reviewed changes