MAINT - CI improvements (security and maintenance) #2077

trallard · 2024-12-12T15:28:00Z

I thought I would do some winter/summer cleaning (depending on your location) ⛄🌞.

This PR adds several improvements/updates to our CI with a focus on improving the contributor experience and security
Details below:

🔒 Security focused

Use SHA for third-party actions and our internal action for development setup
Replace potentially dangerous trigger workflow_run for workflow_call and use as a reusable workflow
Create and use a dedicated environment for releases:
- Created a pst-release environment in the repo (restricted to main only)
- Use pst-release for our release-PST step in publish.yml
- Add pst-release as the default env in PyPI
Add a zizmor.yml workflow to run static analysis on our GH workflows
Add explicit persist-credentials: false to relevant actions (where we do not need further git operations)

👩🏽‍🎤 Contributor experience

Prevent the pre-release.yml workflow from running in repos not under the pydata org (forks)
Our CI workflow has grown significantly with the various tests and checks. This PR splits it into:
CI.yml: pytest, a11y-tests, profiling, coverage
docs.yml: docs-related checks like building across OSes and Python and Sphinx versions, check for broken links (new, note that I had to fix some broken links to get this in 🙈 and there seem to be still some others to fix)
Add tox run -e docs-linkcheck to check for broken links in our docs
Add workflow to add a link to the RTD docs preview in PRs.

🧰 Maintenance

Adds Python 3.13 to our testing matrices (3.12 is left as the default until we are confident all is ok with 3.13)
Add an explicit ubuntu-22.04 target as ubuntu-latest will soon be 24.04 (being rolled out right now) -> I think I might actually have explicit versions on both and only change to latest (or not) when the rollout is completed

Questions / notes

@drammock, we have "sphinx-theme-builder @ https://github.com/pradyunsg/sphinx-theme-builder/archive/87214d0671c943992c05e3db01dca997e156e8d6.zip", in our project. tool and tox.ini. I do not believe this pin is needed anymore, so I would like to remove it, too. WDYT?
@drammock did you create the token for Anaconda.org? I would like to make this an environment secret (vs a repository secret as it is right now)
Also, while adding a new environment, I noticed a github-pages environment that I do not think we are using, so I'd like to delete it.
Finally, I deleted a leftover PYPI_TOKEN, which should have been removed when we changed to trusted publishers.

Closes #2095

github-advanced-security · 2024-12-12T15:28:26Z

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.

github-actions · 2024-12-12T15:38:28Z

Coverage report

This PR does not seem to contain any modification to coverable code.

.github/workflows/publish.yml

docs/.jupyterlite.doit.db-journal

sethmlarson · 2024-12-12T15:53:27Z

.github/workflows/publish.yml

    runs-on: ubuntu-latest
    needs: [build-package]
    permissions:
      id-token: write # needed for PyPI upload
+    environment:


Double-checking that this GitHub Environment is required on the Trusted Publisher in PyPI?

I added it to my checklist so I remember to add it

Double-checking that this GitHub Environment is required on the Trusted Publisher in PyPI?

did this get done @trallard? If so please resolve the conversation.

Nope, since I did not know when we would merge this I did not want to enforce the env just in case we needed to do a release before this was merged (so the release would have failed).
Will do this change when we merge this PR.

.github/workflows/publish.yml

drammock · 2024-12-12T16:03:29Z

@drammock, we have "sphinx-theme-builder @ https://github.com/pradyunsg/sphinx-theme-builder/archive/87214d0671c943992c05e3db01dca997e156e8d6.zip", in our project. tool and tox.ini. I do not believe this pin is needed anymore, so I would like to remove it, too. WDYT?

my fixes to STB were done in May and August of 2023, and the most recent release was March 2023. So we do still need the pin (or to install from tip of main would be fine too --- or convince Pradyun to cut a release 😄).
https://github.com/pradyunsg/sphinx-theme-builder/commits/main/

@drammock did you create the token for Anaconda.org? I would like to make this an environment secret (vs a repository secret as it is right now)

I don't remember creating it. But it might have been me. Are we pushing nightlies to the scientific python channel or something, is that what it's used for?

trallard · 2024-12-12T16:16:14Z

@drammock yes we are using the Scientific Python channel so I am pretty confident it was you then.

my fixes to STB were done in May and August of 2023, and the most recent release was March 2023. So we do still need the pin (or to install from tip of main would be fine too --- or convince Pradyun to cut a release 😄).
pradyunsg/sphinx-theme-builder@main (commits)

ha! I can do both, use main for now and bug Pradyun (I am pretty good at the latter now)

trallard · 2025-02-28T11:16:23Z

Ok I have removed Windows from the a11y CI as there were some failures that I am not sure are true accessibility fails but something else. Since this PR has been opened for a while it might be best to investigate these separately and re-add them, but for now we have a better matrix for a11y tests.

@drammock could you give this PR a last check and see if we can merge?

trallard · 2025-02-28T18:14:05Z

Note that the linkcheck is failing with community/inspiration.md:46: [broken] https://www.docker.com/wp-content/uploads/2022/03/vertical-logo-monochromatic.png: 404 Client Error: Not Found for url: https://www.docker.com/wp-content/uploads/2022/03/vertical-logo-monochromatic.png as it seems this has moved permanently.

I could not find a 1:1 alternative so I am not sure if the best option is to download and add the logo or something else.

drammock · 2025-02-28T19:56:07Z

Note that the linkcheck is failing with community/inspiration.md:46: [broken] https://www.docker.com/wp-content/uploads/2022/03/vertical-logo-monochromatic.png: 404 Client Error: Not Found for url: https://www.docker.com/wp-content/uploads/2022/03/vertical-logo-monochromatic.png as it seems this has moved permanently.

I could not find a 1:1 alternative so I am not sure if the best option is to download and add the logo or something else.

Fixed in #2141. According to Docker docs, the plain logo without wordmark is intended to replace the previous "stacked" logo+wordmark.

.github/workflows/CI.yml

.github/workflows/a11y.yml

.github/workflows/prerelease.yml

.github/workflows/publish.yml

drammock · 2025-02-28T20:30:18Z

.github/workflows/publish.yml

    runs-on: ubuntu-latest
    needs: [build-package]
    permissions:
      id-token: write # needed for PyPI upload
+    environment:


Double-checking that this GitHub Environment is required on the Trusted Publisher in PyPI?

did this get done @trallard? If so please resolve the conversation.

docs/community/topics/accessibility.md

tox.ini

Co-authored-by: Daniel McCloy <[email protected]>

- addresses the missing Docker logo discovered in #2077 (comment) (stores SVG in our repo) - updates GitBook logo (stores SVG in our repo) - updates metaflow and mkdocs material URLs - adds some basic CSS to that gallery grid (otherwise docker logo has no padding around it, violates their brand guidelines) Co-authored-by: Tania Allard <[email protected]>

.github/workflows/prerelease.yml

trallard

I believe I have now addressed all the comments by @drammock and the last CI run was all green after merging the PR to fix the Docker logo link.

.github/workflows/CI.yml

trallard · 2025-03-04T17:33:25Z

.github/workflows/prerelease.yml

+      - name: "Get PST version"
+        run: |
+          python -Im pip install -e .
+          # Get the version of the package
+          PST_VERSION=$(python -c "import pydata_sphinx_theme; print(pydata_sphinx_theme.__version__)")
+          echo "PST_VERSION=$PST_VERSION" >> $GITHUB_ENV
+
+      # Run tests on the built package (which will be later uploaded to PyPI)
+      - name: "Install PST from wheel and test"
+        env:
+          BAIPP_DIST: ${{ steps.baipp.outputs.dist }}
+          PST_VERSION: ${{ env.PST_VERSION }}
+        run: |
+          # calling with --installpkg so we can use the already built package
+          tox run -e py312-tests-no-cov \
+          --installpkg "${BAIPP_DIST}"/pydata_sphinx_theme-"${PST_VERSION}"-py3-none-any.whl \
+          -- --deselect tests/test_build.py::test_translations


Added this extra test run. This would have caught the issues reported in #2090 for example

trallard · 2025-03-04T17:47:42Z

I have noticed some flaky accessibility tests re tab traps (notebooks), I wonder if https://github.com/pydata/pydata-sphinx-theme/actions/runs/13659445744/job/38186908516#step:4:672 is related to #2139 (comment) at all @gabalafou

drammock · 2025-03-04T18:00:11Z

.github/workflows/publish.yml

    # needed for the coverage action
    permissions:
      contents: write
      pull-requests: write
-  # calls our docs workflow (build docs, check broken links, lighthouse)
+      # calls our docs workflow (build docs, check broken links, lighthouse)


this indentation looks wrong, but I'll assume it was a linter autofix and not bother trying to put it back. Intent is pretty clear either way.

trallard · 2025-03-06T19:44:35Z

Final update I just updated the environment in PyPI so this will be the one used when doing the release (via trusted publisher)

This follows #2077 as our actions and workflows needed re-pinning after merging this branch. Included: - **:pushpin: Pin our actions to a concrete SHA** - **:construction_worker: Add test to publish workflow**

- addresses the missing Docker logo discovered in pydata#2077 (comment) (stores SVG in our repo) - updates GitBook logo (stores SVG in our repo) - updates metaflow and mkdocs material URLs - adds some basic CSS to that gallery grid (otherwise docker logo has no padding around it, violates their brand guidelines) Co-authored-by: Tania Allard <[email protected]>

@drammock

I thought I would do some winter/summer cleaning (depending on your location) ⛄🌞. This PR adds several improvements/updates to our CI with a focus on improving the contributor experience and security Details below: ## 🔒 Security focused - Use SHA for third-party actions and our internal action for development setup - Replace potentially [dangerous trigger `workflow_run`](https://woodruffw.github.io/zizmor/audits/#dangerous-triggers) for `workflow_call` and use as a reusable workflow - Create and use a dedicated environment for releases: - [x] Created a `pst-release` environment in the repo (restricted to `main` only) - [x] Use `pst-release` for our `release-PST` step in `publish.yml` - [ ] Add `pst-release` as the default env in PyPI - Add a `zizmor.yml` workflow to run static analysis on our GH workflows - Add explicit `persist-credentials: false` to relevant actions (where we do not need further git operations) ## 👩🏽‍🎤 Contributor experience - Prevent the `pre-release.yml` workflow from running in repos not under the `pydata` org (forks) - Our CI workflow has grown significantly with the various tests and checks. This PR splits it into: - `CI.yml`: pytest, a11y-tests, profiling, coverage - `docs.yml`: docs-related checks like building across OSes and Python and Sphinx versions, check for broken links (new, note that I had to fix some broken links to get this in 🙈 and there seem to be still some others to fix) - Add `tox run -e docs-linkcheck` to check for broken links in our docs - Add workflow to add a link to the RTD docs preview in PRs. ## 🧰 Maintenance - Adds Python `3.13` to our testing matrices (`3.12` is left as the `default` until we are confident all is ok with `3.13`) - Add an explicit `ubuntu-22.04` target as `ubuntu-latest` will soon be `24.04` (being rolled out right now) -> I think I might actually have explicit versions on both and only change to latest (or not) when the rollout is completed ---- ## Questions / notes - @drammock, we have `"sphinx-theme-builder @ https://github.com/pradyunsg/sphinx-theme-builder/archive/87214d0671c943992c05e3db01dca997e156e8d6.zip",` in our `project. tool` and `tox.ini`. I do not believe this pin is needed anymore, so I would like to remove it, too. WDYT? - @drammock did you create the token for Anaconda.org? I would like to make this an environment secret (vs a repository secret as it is right now) - Also, while adding a new environment, I noticed a `github-pages` environment that I do not think we are using, so I'd like to delete it. - Finally, I deleted a leftover `PYPI_TOKEN,` which should have been removed when we changed to trusted publishers. Closes pydata#2095 --------- Co-authored-by: Daniel McCloy <[email protected]>

This follows pydata#2077 as our actions and workflows needed re-pinning after merging this branch. Included: - **:pushpin: Pin our actions to a concrete SHA** - **:construction_worker: Add test to publish workflow**

This is a follow-up to #2077. This PR fixes action permissions to call reusable workflows in our release workflow. I also pinned the SHA for a workflow I missed in a previous review.

trallard added 18 commits November 21, 2024 12:28

👷 Update release workflow

890ee6f

👷 Add check for docs links

e89a490

🔧 Add linkcheck for docs

865158a

👷 Separate docs CI into another workflow

e93cf6c

🔧 Add ignores and redirects to conf

0095423

✏️ Fix URLs in the docs

e70a1d3

🔧 Update tox command for linkchecks

18e2198

Remove needs - CI workflow

53dafed

Merge branch 'main' into trallard/maintenance-ql-improvements

74f4a1a

🔐 Change workflow trigger to workflow_call

76b95ca

🔒 Ensure proper variable escaping in docs workflow

6e7851c

⬆️ Upgrade runners OS and Python versions

3e0899d

Add coverage required permissions

d61186f

Add actions scope to permissions:

e9bd826

👷 Add GitHub attestations to release workflow

21d7a24

🔒 Add static analysis for GH actions with Zizmor

e560340

👷 Pin our reusable action to a hash

9b31a14

Add environment and pinned actions to release workflow

560ecfe

trallard added kind: maintenance Improving maintainability and reducing technical debt tag: CI Pull requests that update GitHub Actions code labels Dec 12, 2024

Add more comments to workflows

2a7fca4

trallard commented Dec 12, 2024

View reviewed changes

.github/workflows/publish.yml Outdated Show resolved Hide resolved

trallard commented Dec 12, 2024

View reviewed changes

.github/workflows/publish.yml Outdated Show resolved Hide resolved

sethmlarson reviewed Dec 12, 2024

View reviewed changes

trallard added 2 commits December 12, 2024 16:16

🔥 Remove accidentally commited file

8ecc548

Remove setup-dev action

3864d38

Remove Windows for now

b074f7d

trallard added 2 commits February 28, 2025 18:08

Merge branch 'main' into trallard/maintenance-ql-improvements

a199273

Fix links

4228bcb

drammock mentioned this pull request Feb 28, 2025

cleanup UX inspiration page #2141

Merged

drammock reviewed Feb 28, 2025

View reviewed changes

trallard and others added 2 commits March 3, 2025 12:02

Apply suggestions from code review

a3db2d2

Co-authored-by: Daniel McCloy <[email protected]>

Add SHA for upload action

417d831

trallard added 3 commits March 4, 2025 16:20

git commit -m 📝 Add notes about updating actions SHA

eb9b1e7

Merge branch 'main' into trallard/maintenance-ql-improvements

fd88aa7

Add step to run tests on built package

0a37edf

github-advanced-security bot found potential problems Mar 4, 2025

View reviewed changes

.github/workflows/prerelease.yml Fixed Show fixed Hide fixed

Ensure we do not use var expansion

38171d0

trallard commented Mar 4, 2025

View reviewed changes

drammock approved these changes Mar 4, 2025

View reviewed changes

drammock merged commit 4a1e789 into pydata:main Mar 4, 2025
30 of 31 checks passed

trallard deleted the trallard/maintenance-ql-improvements branch March 5, 2025 10:06

trallard mentioned this pull request Mar 5, 2025

MAINT - Update actions/workflows SHAs #2151

Merged

drammock mentioned this pull request Mar 6, 2025

flaky accessibility tests: tab traps (notebooks) #2158

Closed

trallard mentioned this pull request Mar 11, 2025

BUG - Update release workflow permissions #2162

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MAINT - CI improvements (security and maintenance) #2077

MAINT - CI improvements (security and maintenance) #2077

trallard commented Dec 12, 2024 •

edited

Loading

github-advanced-security bot commented Dec 12, 2024

github-actions bot commented Dec 12, 2024 •

edited

Loading

sethmlarson Dec 12, 2024

trallard Dec 12, 2024

drammock Feb 28, 2025

trallard Mar 3, 2025

drammock commented Dec 12, 2024

trallard commented Dec 12, 2024

trallard commented Feb 28, 2025

trallard commented Feb 28, 2025

drammock commented Feb 28, 2025

drammock Feb 28, 2025

trallard left a comment

trallard Mar 4, 2025

trallard commented Mar 4, 2025

drammock Mar 4, 2025

trallard commented Mar 6, 2025

MAINT - CI improvements (security and maintenance) #2077

MAINT - CI improvements (security and maintenance) #2077

Conversation

trallard commented Dec 12, 2024 • edited Loading

🔒 Security focused

👩🏽‍🎤 Contributor experience

🧰 Maintenance

Questions / notes

github-advanced-security bot commented Dec 12, 2024

github-actions bot commented Dec 12, 2024 • edited Loading

Coverage report

sethmlarson Dec 12, 2024

Choose a reason for hiding this comment

trallard Dec 12, 2024

Choose a reason for hiding this comment

drammock Feb 28, 2025

Choose a reason for hiding this comment

trallard Mar 3, 2025

Choose a reason for hiding this comment

drammock commented Dec 12, 2024

trallard commented Dec 12, 2024

trallard commented Feb 28, 2025

trallard commented Feb 28, 2025

drammock commented Feb 28, 2025

drammock Feb 28, 2025

Choose a reason for hiding this comment

trallard left a comment

Choose a reason for hiding this comment

trallard Mar 4, 2025

Choose a reason for hiding this comment

trallard commented Mar 4, 2025

drammock Mar 4, 2025

Choose a reason for hiding this comment

trallard commented Mar 6, 2025

trallard commented Dec 12, 2024 •

edited

Loading

github-actions bot commented Dec 12, 2024 •

edited

Loading