Skip to content

Add support and testing for Python 3.12 and 3.13#4101

Merged
luckyh merged 4 commits intoavocado-framework:masterfrom
pevogam:python3-updates
Sep 4, 2025
Merged

Add support and testing for Python 3.12 and 3.13#4101
luckyh merged 4 commits intoavocado-framework:masterfrom
pevogam:python3-updates

Conversation

@pevogam
Copy link
Copy Markdown
Contributor

@pevogam pevogam commented Apr 7, 2025

Summary by CodeRabbit

  • Chores
    • Expanded CI Python compatibility to include 3.12 and 3.13.
    • Centralized the Python version matrix for reuse across CI jobs.
    • Improved Python setup and enabled dependency caching to speed CI runs.
    • Added packaging dependencies for more reliable builds.
    • Restricted legacy egg builds to older Python versions (3.9–3.11); skipped for 3.12/3.13.
    • No functional/product behavior changes.

@YvanY0
Copy link
Copy Markdown
Contributor

YvanY0 commented Apr 9, 2025

Hello @pevogam, distutils was deprecated, if we plan to enable the CI check for python3.12 and python3.13, we should consider this.

I had a PR for distutils, but it has some risks, what's your opinion of the distutils module?

@pevogam
Copy link
Copy Markdown
Contributor Author

pevogam commented Apr 9, 2025

distutils

Hi @PaulYuuu I think we should move forward from deprecated packages like this and comply with PEP 632.

Can you add a reference/link to the distutils pull request you proposed?

@YvanY0
Copy link
Copy Markdown
Contributor

YvanY0 commented Apr 9, 2025

distutils

Hi @PaulYuuu I think we should move forward from deprecated packages like this and comply with PEP 632.

Can you add a reference/link to the distutils pull request you proposed?

ah sorry, should be this one: #3920

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Aug 25, 2025

📝 Walkthrough

Walkthrough

Centralized Python versions in the CI workflow using a YAML anchor (adds 3.12 and 3.13), reused the anchor in job matrices, enabled check-latest and pip caching in setup steps, expanded build dependencies to include setuptools and wheel, and gated a legacy eggs build to run only on Python 3.9–3.11.

Changes

Cohort / File(s) Change Summary
CI workflow updates
.github/workflows/ci.yml
- Added YAML anchor &py_versions (now includes Python 3.12 and 3.13) and reused it for travis-check and package-build matrices
- travis-check now references the shared anchor and updates actions/setup-python with check-latest: true, cache: 'pip', and cache-dependency-path: requirements-travis.txt
- package-build matrix now uses *py_versions; actions/setup-python also has check-latest: true and dependency install changed to pip install build setuptools wheel (was build only)
- Renamed Build eggs to Build eggs (legacy) and added if: ${{ contains(fromJSON('[\"3.9\",\"3.10\",\"3.11\"]'), matrix.python-version) }} so it runs only for Python 3.9–3.11 (skipped for 3.12/3.13)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20–30 minutes

Tip

🔌 Remote MCP (Model Context Protocol) integration is now available!

Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbit in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbit in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbit gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbit read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbit help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbit ignore or @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbit summary or @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbit or @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@pevogam
Copy link
Copy Markdown
Contributor Author

pevogam commented Aug 25, 2025

Hi @PaulYuuu, the PR that we thought is a blocker for this is merged but I still get some problems here like

Run python setup.py bdist_egg
  python setup.py bdist_egg
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.12.11/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.12.11/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.11/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.11/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.11/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.12.11/x64/lib
Traceback (most recent call last):
  File "/home/runner/work/avocado-vt/avocado-vt/setup.py", line 17, in <module>
    from setuptools import find_packages, setup
ModuleNotFoundError: No module named 'setuptools'
Error: Process completed with exit code 1.

So I assume we have to deprecate the setuptools as well.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.github/workflows/ci.yml (1)

100-101: Remove deprecated egg build step and update distutils usage

The CI workflow’s “Build eggs” step uses bdist_egg, which is deprecated and will fail on Python 3.12+ (where distutils has been removed). Additionally, the repository contains a direct import of distutils.version.LooseVersion, which must be replaced to ensure compatibility.

• In .github/workflows/ci.yml (around lines 100–101):

     - name: Build package
       run: python -m build
-    - name: Build eggs
-      run: python setup.py bdist_egg

If you still need to produce eggs for legacy consumers, gate this step to Python < 3.12 only:

- name: Build eggs (legacy)
  if: startsWith(matrix.python-version, '3.8') || startsWith(matrix.python-version, '3.9') || startsWith(matrix.python-version, '3.10') || startsWith(matrix.python-version, '3.11')
  run: python setup.py bdist_egg

• In virttest/shared/scripts/virtio_console_guest.py (line 24):

- from distutils.version import LooseVersion  # pylint: disable=W0611,E0611
+ from packaging.version import parse as parse_version- if LooseVersion(version) >= LooseVersion("1.0"):
+ if parse_version(version) >= parse_version("1.0"):

Make sure to add packaging as a dependency if it isn’t already. This change removes reliance on the removed distutils module in Python 3.12+.

🧹 Nitpick comments (5)
.github/workflows/ci.yml (5)

38-38: Avoid patch pin in matrix; use minor-only for stability and consistency

Mixing a patch pin (“3.10.1”) with minor-only versions (“3.11”, “3.12”, “3.13”) is inconsistent and can break when that exact patch isn’t available on the runner. Prefer “3.10” here, like in package-build.

Apply this diff:

-        python-version: ["3.8", "3.9", "3.10.1", "3.11", "3.12", "3.13"]
+        python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]

Optionally, ensure latest patch of each minor is used:

# under: uses: actions/setup-python@v5
with:
  python-version: ${{ matrix.python-version }}
  check-latest: true

85-85: De-duplicate version lists across jobs to prevent drift

Both jobs target the same minors. Consider centralizing via a YAML anchor so future updates touch one place.

Example:

jobs:
  travis-check:
    strategy:
      matrix:
        python-version: &py_versions ["3.8","3.9","3.10","3.11","3.12","3.13"]

  package-build:
    strategy:
      matrix:
        python-version: *py_versions

51-52: Upgrade pip before installing requirements to avoid resolver/ABI issues on 3.12/3.13

Older pip/setuptools combos often choke on newest Python minors. Upgrade first, then install.

-      - name: Install dependencies
-        run: pip install -r requirements-travis.txt
+      - name: Upgrade pip and install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements-travis.txt

25-26: Apply the same pip upgrade in static-checks job

Static checks run on the runner’s default Python (Ubuntu 24.04 → Python 3.12). Upgrade pip there too for consistency.

-      - name: Install dependencies
-        run: pip install -r requirements-travis.txt
+      - name: Upgrade pip and install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements-travis.txt

83-99: Optional: cache pip downloads to speed up CI

actions/setup-python@v5 supports pip caching; it materially speeds up multi-version matrices.

Example:

- uses: actions/setup-python@v5
  with:
    python-version: ${{ matrix.python-version }}
    cache: 'pip'
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled
  • Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 78d9d0d and 22c3106.

📒 Files selected for processing (1)
  • .github/workflows/ci.yml (2 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
  • GitHub Check: fedora_40 AVOCADO_SRC:avocado-framework<104.0 SETUP:-m pip install . VT_TYPE:libvirt
  • GitHub Check: fedora_40 AVOCADO_SRC:avocado-framework<104.0 SETUP:-m pip install PYPI_UPLOAD/*.whl VT_TYPE:libvirt
  • GitHub Check: fedora_40 AVOCADO_SRC: SETUP:-m pip install PYPI_UPLOAD/*.whl VT_TYPE:qemu
  • GitHub Check: fedora_40 AVOCADO_SRC:avocado-framework<104.0 SETUP:-m pip install . VT_TYPE:qemu
  • GitHub Check: fedora_40 AVOCADO_SRC: SETUP:-m pip install PYPI_UPLOAD/*.whl VT_TYPE:libvirt
  • GitHub Check: fedora_40 AVOCADO_SRC:avocado-framework<104.0 SETUP:-m pip install PYPI_UPLOAD/*.whl VT_TYPE:qemu
  • GitHub Check: fedora_40 AVOCADO_SRC: SETUP:-m pip install . VT_TYPE:libvirt
  • GitHub Check: fedora_40 AVOCADO_SRC: SETUP:-m pip install . VT_TYPE:qemu
  • GitHub Check: Static checks

@YvanY0
Copy link
Copy Markdown
Contributor

YvanY0 commented Aug 25, 2025

Hi @PaulYuuu, the PR that we thought is a blocker for this is merged but I still get some problems here like

Run python setup.py bdist_egg
  python setup.py bdist_egg
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.12.11/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.12.11/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.11/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.11/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.11/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.12.11/x64/lib
Traceback (most recent call last):
  File "/home/runner/work/avocado-vt/avocado-vt/setup.py", line 17, in <module>
    from setuptools import find_packages, setup
ModuleNotFoundError: No module named 'setuptools'
Error: Process completed with exit code 1.

So I assume we have to deprecate the setuptools as well.

Hello @pevogam, Python >= 3.12 will not pre-install the setuptools module, see https://docs.python.org/3/whatsnew/3.12.html
But avocado-vt uses setuptools as the build backend system.
However python setup.py will not install required packages before building the wheel/egg. In the future when we switch to use python -m build, the issue should be gone.

uv run python -m build
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - setuptools >= 40.8.0
* Getting build dependencies for sdist...
running egg_info
creating avocado_framework_plugin_vt.egg-info
writing avocado_framework_plugin_vt.egg-info/PKG-INFO

In short, to fix the setuptools problem in this PR, please consider adding setuptools;python_version >= '3.12' in the requirements-travis.txt file.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
requirements-travis.txt (2)

1-1: Tiny readability nit + optional context comment

Purely cosmetic, but adding a space after the semicolon and a short comment helps future readers understand why this line exists.

- setuptools;python_version >= '3.12'
+ # Python ≥ 3.12 does not preinstall setuptools; needed for setup.py workflows
+ setuptools; python_version >= '3.12'

1-1: Consider setting a conservative lower bound and planning the move to PEP 517 builds

For long‑term stability on 3.12/3.13, consider pinning a minimum setuptools version known to support those interpreters (e.g., >= 68/69+). Also, when you switch CI from “python setup.py …” to “python -m build”, you’ll want build tooling available.

Please verify the minimum setuptools version you want to require and whether “bdist_egg” is still needed in CI. If useful, here’s a minimal change that keeps current behavior but lays groundwork for the future:

- setuptools; python_version >= '3.12'
+ # runtime build backend for Python ≥ 3.12
+ setuptools>=68; python_version >= '3.12'
+ # optional: when migrating CI to `python -m build`
+ build>=1.2; python_version >= '3.12'
+ wheel>=0.41; python_version >= '3.12'

If you’d like, I can open a follow‑up PR to migrate the workflow to “python -m build” and drop egg builds.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled
  • Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 72d191d and 59d615e.

📒 Files selected for processing (2)
  • .github/workflows/ci.yml (2 hunks)
  • requirements-travis.txt (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/ci.yml
🧰 Additional context used
🪛 LanguageTool
requirements-travis.txt

[grammar] ~1-~1: There might be a mistake here.
Context: setuptools;python_version >= '3.12' Sphinx==1.3b1 # inspektor (static and st...

(QB_NEW_EN)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Static checks
🔇 Additional comments (2)
requirements-travis.txt (2)

1-1: LGTM: correct PEP 508 environment marker for setuptools on Python ≥ 3.12

This is a sensible stop‑gap to fix the CI failure caused by setuptools no longer being bundled with Python ≥ 3.12. Nice and targeted.


2-11: Ensure compatibility of pinned tools with Python 3.12/3.13
Our local sanity-check on Python 3.11 failed to build the C extension for netifaces==0.11.0 (missing GCC/sysheaders) and couldn’t even create a virtualenv (ensurepip wasn’t available). This highlights that several of our legacy pins are brittle on modern interpreters—even if CI provides build tools, they’re at high risk of breakage on 3.12/3.13.

• Gate legacy pins to older Python versions via environment markers and add up-to-date equivalents for ≥3.12.
• Alternatively, split requirements into per-purpose files (e.g., docs/requirements.txt, lint/requirements.txt, test/requirements.txt) so you can modernize each slice without impacting the others.
• In your .github/workflows/ci.yml, confirm that:

  • the requirements-travis.txt install step runs before any python setup.py … or project imports,
  • and that the CI runner includes python3.x-venv, gcc, and other build dependencies needed for C extensions.

Please manually verify that all matrix jobs for Python 3.12 and 3.13 install cleanly and pass in CI.

@YvanY0
Copy link
Copy Markdown
Contributor

YvanY0 commented Aug 25, 2025

@pevogam , sorry I was in the wrong context, name: Build Package for Python ${{ matrix.python-version }} will not install requirements-travis.txt, so have to updte its

      run: |
        python -m pip install --upgrade pip
        pip install build

to

      run: |
        python -m pip install --upgrade pip
        pip install build setuptools

For CI env, I don't think it's a problem that install setuptools even if it's installed(python<=3.11). But let's listen the voice from @richtja and @luckyh

@pevogam
Copy link
Copy Markdown
Contributor Author

pevogam commented Aug 25, 2025

Thanks for the quick feedback, I was just about to investigate this! Alright, pushed with the other suggested change, let's see.

@pevogam
Copy link
Copy Markdown
Contributor Author

pevogam commented Aug 25, 2025

Everything passes now, @richtja, @luckyh, let us know if you can review these changes too.

@pevogam
Copy link
Copy Markdown
Contributor Author

pevogam commented Aug 25, 2025

We could also do what the rabbit recommends:

image

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (3)
requirements.txt (1)

1-1: Avoid hard-pinning packaging and add a Python version marker to prevent breakage on older/guest environments.

  • PEP 632 replacement in the guest script is good, but pinning packaging==25.0 globally may (a) increase resolver conflicts, and (b) try to install packaging on Python 2 guests where this script may run, which will fail.
  • Prefer a compatible range and gate it to Python 3 only. If we implement a no-deps fallback in the guest script (see my other comment), we can even drop this dependency entirely.

Apply this diff:

-packaging==25.0
+packaging>=23,<26; python_version >= '3.9'

If you decide to keep Python 3.8 support anywhere, adjust the marker accordingly.

Would you like me to open a follow-up to audit whether requirements.txt is consumed on guests vs host-only? That determines if we can safely remove this line altogether.

.github/workflows/ci.yml (2)

87-87: Mirror the fromJSON-based matrix here as well.

This alias is what triggers actionlint’s “alias node” and “unknown matrix property” diagnostics downstream.

Apply this diff:

-      matrix:
-        python-version: *py_versions
+      matrix:
+        python-version: ${{ fromJSON(env.PY_VERSIONS) }}

102-104: Gate the legacy eggs step more simply and ensure setuptools is present.

  • The current condition is verbose and includes 3.8, which isn’t in the matrix. Use an allowlist with fromJSON for clarity.
  • bdist_egg requires setuptools; ensure it’s installed in this job to avoid the earlier 3.12+ missing-setuptools pitfall resurfacing in other images.

Apply this diff to the condition:

-    - name: Build eggs (legacy)
-      if: startsWith(matrix.python-version, '3.8') || startsWith(matrix.python-version, '3.9') || startsWith(matrix.python-version, '3.10') || startsWith(matrix.python-version, '3.11')
+    - name: Build eggs (legacy)
+      if: ${{ contains(fromJSON('["3.9","3.10","3.11"]'), matrix["python-version"]) }}
       run: python setup.py bdist_egg

And (outside this hunk) update the install step to include setuptools:

     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        pip install build
+        pip install build setuptools

If you want to be extra safe, we can also drop the legacy step entirely for 3.12/3.13 and keep only python -m build, which already produces wheels and sdists in an isolated env.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled
  • Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 59ece03 and 2f26346.

📒 Files selected for processing (3)
  • .github/workflows/ci.yml (4 hunks)
  • requirements.txt (1 hunks)
  • virttest/shared/scripts/virtio_console_guest.py (2 hunks)
🧰 Additional context used
🪛 actionlint (1.7.7)
.github/workflows/ci.yml

87-87: "matrix values" section must be sequence node but got alias node with "" tag

(syntax-check)


103-103: property "python-version" is not defined in object type {}

(expression)


103-103: property "python-version" is not defined in object type {}

(expression)


103-103: property "python-version" is not defined in object type {}

(expression)


103-103: property "python-version" is not defined in object type {}

(expression)

🔇 Additional comments (2)
.github/workflows/ci.yml (2)

51-52: Good: check-latest + pip cache.

These options help keep the CI stable on patch releases and speed up installs. No change requested.


38-38: It looks like actionlint treated “never” as a file path because the flag needs two dashes. Please re-run the lint with the correct option and confirm there are no matrix-alias warnings:

# Install actionlint if you haven’t already
# Then from your repo root:
actionlint --color never .github/workflows/ci.yml

Once you’ve switched to a fromJSON-based matrix (as suggested), verify that this produces no warnings.

Comment thread virttest/shared/scripts/virtio_console_guest.py Outdated
Comment thread virttest/shared/scripts/virtio_console_guest.py Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (3)
.github/workflows/ci.yml (3)

38-38: Anchor is defined but not reused; prefer reusing across jobs or remove.

You introduce &py_versions here but don’t reference it later. Since the package-build job tries to source versions elsewhere, either reuse this anchor there or drop it to avoid confusion.

Apply this to reuse the anchor in package-build:

-        python-version: ${{ fromJSON(env.PY_VERSIONS) }}
+        python-version: *py_versions

51-52: Pip cache is good; consider pinning dependency paths for better cache hits.

check-latest is fine; to improve cache keying, include the requirements file so cache busts when it changes.

       with:
         python-version: ${{ matrix.python-version }}
         check-latest: true
         cache: 'pip'
+        cache-dependency-path: requirements-travis.txt

99-99: Include wheel to avoid build-tool gaps.

Some PEP 517 builds still expect wheel present. It’s lightweight and avoids edge-case failures.

-        pip install build setuptools
+        pip install build setuptools wheel
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled
  • Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 2f26346 and 5461a9b.

📒 Files selected for processing (1)
  • .github/workflows/ci.yml (4 hunks)
🧰 Additional context used
🪛 YAMLlint (1.37.1)
.github/workflows/ci.yml

[error] 104-104: duplication of key "if" in mapping

(key-duplicates)

Comment thread .github/workflows/ci.yml Outdated
Comment thread .github/workflows/ci.yml Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (1)
.github/workflows/ci.yml (1)

88-88: Correct reuse of the YAML anchor; resolves the earlier matrix expansion issue.

Switching to *py_versions here fixes the previously noted fromJSON/env variable problem. This aligns both jobs to the same version set.

🧹 Nitpick comments (4)
.github/workflows/ci.yml (4)

51-53: Setup-python caching looks good; minor tweaks can improve cache efficacy.

  • pip cache key is derived from cache-dependency-path; any deps installed outside requirements-travis.txt (e.g., setuptools-rust, editable installs) won’t influence the cache key and may reduce hit rate.
  • Optional: include additional lock/manifest files if available, or add a secondary actions/cache around pip’s cache after checking out avocado-libs.

Example adjustments (only if such files exist in-repo):

       with:
         python-version: ${{ matrix.python-version }}
         check-latest: true
         cache: 'pip'
-        cache-dependency-path: requirements-travis.txt
+        cache-dependency-path: |
+          requirements-travis.txt
+          pyproject.toml
+          setup.cfg

If you want higher determinism, consider dropping check-latest: true to reduce flakiness from latest patch bumps, though it’s fine to keep for catching regressions early.


54-56: Upgrade pip earlier to avoid resolver/build quirks on 3.12/3.13.

You upgrade pip later inside the avocado-libs step, but the first pip install -r requirements-travis.txt still uses the runner’s default pip. Upgrading pip before this install improves reliability, especially with PEP 517/518 wheels on 3.13.

       - name: Install dependencies
-        run: pip install -r requirements-travis.txt
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements-travis.txt

93-100: Consider enabling pip caching in the package-build job as well.

The build matrix will repeatedly download wheels across versions. Adding cache: 'pip' here can trim minutes off cold runs.

     - name: Set up Python
       uses: actions/setup-python@v5
       with:
         python-version: ${{ matrix.python-version }}
+        check-latest: true
+        cache: 'pip'
+        cache-dependency-path: |
+          pyproject.toml
+          setup.cfg

103-105: Eggs step gating is correct and fixes the duplicate-if lint error noted earlier.

Limiting legacy bdist_egg to 3.9–3.11 is reasonable. One caution: newer setuptools versions may deprecate or disable bdist_egg. If CI breaks in the future, consider pinning setuptools<70 for this step or dropping eggs entirely.

Targeted pin (only for the eggs step), if needed:

     - name: Build eggs (legacy)
       if: ${{ contains(fromJSON('["3.9","3.10","3.11"]'), matrix.python-version) }}
-      run: python setup.py bdist_egg
+      run: |
+        python -m pip install "setuptools<70"
+        python setup.py bdist_egg
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled
  • Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 5461a9b and 9e710df.

📒 Files selected for processing (1)
  • .github/workflows/ci.yml (4 hunks)
🔇 Additional comments (2)
.github/workflows/ci.yml (2)

38-38: Good consolidation of Python versions via YAML anchor.

Defining the matrix list once with &py_versions keeps jobs in sync and avoids env binding pitfalls. Nice cleanup.


100-100: Adding setuptools and wheel to build deps is the right call for 3.12+.

This addresses the lack of preinstalled setuptools on 3.12+ and ensures wheels are generated. Thanks for incorporating the guidance from the discussion.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (2)
.github/workflows/ci.yml (2)

38-38: Good call: central YAML anchor for Python versions (and reuse).

Defining &py_versions here and reusing it across jobs simplifies future updates and fixes the earlier undefined env.PY_VERSIONS issue. Nicely done.


88-88: Matrix now reuses the anchor — resolves expansion errors.

Switching to *py_versions here eliminates the prior matrix expansion failure and keeps both jobs in lockstep on supported versions.

🧹 Nitpick comments (3)
.github/workflows/ci.yml (3)

51-53: setup-python caching is configured correctly; minor consistency nit.

check-latest: true, cache: 'pip', and cache-dependency-path: requirements-travis.txt are all appropriate. As a small consistency tweak, prefer python -m pip everywhere in this job to avoid PATH discrepancies.

Apply within the existing “Install dependencies” step in this job:

-      - name: Install dependencies
-        run: pip install -r requirements-travis.txt
+      - name: Install dependencies
+        run: python -m pip install -r requirements-travis.txt

97-99: Pip cache here likely won’t be effective; either add a dep file or drop cache.

This job installs tools via an inline command (pip install build setuptools wheel) with no requirements/lock file. Without cache-dependency-path, the cache key won’t reflect changes and may not provide value. Two options:

  • Simpler (recommended): drop the cache for this job.
  • Or: add a tiny requirements file and point the cache to it.

Option A — remove cache keys:

       with:
         python-version: ${{ matrix.python-version }}
-        check-latest: true
-        cache: 'pip'
+        check-latest: true

Option B — introduce a build requirements file:

-    - name: Install dependencies
-      run: |
-        python -m pip install --upgrade pip
-        pip install build setuptools wheel
+    - name: Prepare build requirements
+      run: printf "build\nsetuptools\nwheel\n" > requirements-build.txt
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        python -m pip install -r requirements-build.txt
-      with:
-        python-version: ${{ matrix.python-version }}
-        check-latest: true
-        cache: 'pip'
+      with:
+        python-version: ${{ matrix.python-version }}
+        check-latest: true
+        cache: 'pip'
+        cache-dependency-path: requirements-build.txt

105-106: Legacy eggs step is correctly gated; consider future removal or matrix-level gating.

Single if: fixes the duplicate-key issue noted earlier. If eggs are truly legacy, consider moving the constraint into strategy (exclude/include) or planning deprecation to simplify the workflow.

If eggs are still required on 3.8, adjust the list accordingly:

-      if: ${{ contains(fromJSON('["3.9","3.10","3.11"]'), matrix.python-version) }}
+      if: ${{ contains(fromJSON('["3.8","3.9","3.10","3.11"]'), matrix.python-version) }}
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled
  • Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 9e710df and 9f92faf.

📒 Files selected for processing (1)
  • .github/workflows/ci.yml (4 hunks)
🔇 Additional comments (1)
.github/workflows/ci.yml (1)

102-102: Build tooling install looks correct for 3.12/3.13.

Installing build setuptools wheel covers both wheel builds and the legacy bdist_egg step that follows (setuptools). No issues here.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
.github/workflows/ci.yml (3)

51-53: Cache is good; broaden cache-dependency-path for robust invalidation.

Including only requirements-travis.txt won’t invalidate the pip cache when build inputs change (e.g., pyproject.toml, setup.cfg, setup.py). Recommend listing them explicitly.

       with:
         python-version: ${{ matrix.python-version }}
         check-latest: true
-        cache: 'pip'
-        cache-dependency-path: requirements-travis.txt
+        cache: 'pip'
+        cache-dependency-path: |
+          requirements-travis.txt
+          pyproject.toml
+          setup.cfg
+          setup.py

97-97: check-latest enabled — OK, just note reproducibility trade-off.

This pulls the newest patch for each minor on every run. If you ever need fully reproducible CI, consider pinning patch versions in the matrix. Otherwise, this is fine.


101-101: Prefer invoking pip via the interpreter.

Minor but safer across environments; keeps installs tied to the selected Python.

-        pip install build setuptools wheel
+        python -m pip install build setuptools wheel
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled
  • Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 9f92faf and d0b6fa8.

📒 Files selected for processing (1)
  • .github/workflows/ci.yml (4 hunks)
🧰 Additional context used
🪛 actionlint (1.7.7)
.github/workflows/ci.yml

106-106: could not parse as YAML: yaml: line 106: mapping values are not allowed in this context

(syntax-check)

🪛 YAMLlint (1.37.1)
.github/workflows/ci.yml

[error] 108-108: no new line character at the end of file

(new-line-at-end-of-file)


[error] 106-106: syntax error: mapping values are not allowed here

(syntax)

🔇 Additional comments (2)
.github/workflows/ci.yml (2)

38-38: Centralized Python versions via YAML anchor — nice.

Defining &py_versions once and reusing it improves readability and avoids drift across jobs.


88-88: *Reusing py_versions in package-build matrix — aligned and DRY.

Referencing the earlier anchor here resolves the prior “undefined env.PY_VERSIONS” concern and keeps both jobs in sync.

Comment thread .github/workflows/ci.yml Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
.github/workflows/ci.yml (1)

104-106: Legacy eggs gating is correct; duplicate-if issue resolved

The step now has a single, valid if: predicate and is correctly limited to 3.9–3.11, avoiding Python 3.12+ where setuptools is not preinstalled and distutils is removed. Looks good.

🧹 Nitpick comments (3)
.github/workflows/ci.yml (3)

97-101: Mirror pip caching in package-build for consistency and speed

Optional, but aligning this job with the travis-check caching will cut repeated downloads across the matrix. Also safe to keep check-latest here.

Suggested adjustments:

-    - name: Set up Python
+    - name: Set up Python
       uses: actions/setup-python@v5
       with:
         python-version: ${{ matrix.python-version }}
         check-latest: true
+        cache: 'pip'
+        cache-dependency-path: |
+          requirements-travis.txt
+          **/pyproject.toml
+          **/setup.cfg

Note: the globbed files are optional; if they don’t exist, caching still works using requirements-travis.txt.


56-71: Double-check avocado-libs build deps on 3.13

Minor verification: the pinned setuptools-rust==1.1.2 plus editable install of avocado-libs should work on 3.12/3.13, but if upstream bumps minimum versions, this may break first in 3.13. Keep an eye on this in CI; if it flakes, consider loosening the pin or moving to pyproject-managed build deps.


104-106: Optional: derive eggs gating from a single source to avoid drift

If you adopt env.PY_VERSIONS, you can also centralize the eggs gate to avoid forgetting to update both lists. For example, define an env for eggable versions:

Add (outside this range):

env:
  PY_EGG_VERSIONS: '["3.9","3.10","3.11"]'

Then adjust the condition:

-      if: ${{ contains(fromJSON('["3.9","3.10","3.11"]'), matrix.python-version) }}
+      if: ${{ contains(fromJSON(env.PY_EGG_VERSIONS), matrix.python-version) }}
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled
  • Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between d0b6fa8 and 193c3c5.

📒 Files selected for processing (1)
  • .github/workflows/ci.yml (4 hunks)
🧰 Additional context used
🪛 actionlint (1.7.7)
.github/workflows/ci.yml

88-88: "matrix values" section must be sequence node but got alias node with "" tag

(syntax-check)


105-105: property "python-version" is not defined in object type {}

(expression)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: Static checks
  • GitHub Check: fedora_40 AVOCADO_SRC: SETUP:-m pip install PYPI_UPLOAD/*.whl VT_TYPE:qemu
🔇 Additional comments (1)
.github/workflows/ci.yml (1)

51-53: Good use of setup-python caching and check-latest

check-latest plus pip cache and dependency path on requirements-travis.txt is a solid speedup for the travis-check job. No issues.

Comment thread .github/workflows/ci.yml
@pevogam
Copy link
Copy Markdown
Contributor Author

pevogam commented Aug 25, 2025

Alright, @PaulYuuu I think now I made both you and the rabbit bot happy (the latter happy enough as it cannot be easily satiated) so let me know if the final commits and changes look good enough to you.

Copy link
Copy Markdown
Contributor

@luckyh luckyh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pevogam thank you for the enhancement!

LGTM

@YvanY0
Copy link
Copy Markdown
Contributor

YvanY0 commented Aug 28, 2025

Oh, @pevogam, please drop the python 3.8 commit, the current master branch already removed it.

@pevogam
Copy link
Copy Markdown
Contributor Author

pevogam commented Aug 28, 2025

Hi @PaulYuuu, I just fetched and rebased but I don't see it being removed, can you point me to exact commit hash?

@YvanY0
Copy link
Copy Markdown
Contributor

YvanY0 commented Aug 28, 2025

Hi @PaulYuuu, I just fetched and rebased but I don't see it being removed, can you point me to exact commit hash?

Should be b6ba9b7, and you can check the file from web to view.

Signed-off-by: Plamen Dimitrov <plamen.dimitrov@intra2net.com>
Signed-off-by: Plamen Dimitrov <plamen.dimitrov@intra2net.com>
Signed-off-by: Plamen Dimitrov <plamen.dimitrov@intra2net.com>
Limit the egg build to earlier python versions only.

Also add wheel as it is lightweight and can handle edge-case
failures.

Signed-off-by: Plamen Dimitrov <plamen.dimitrov@intra2net.com>
@pevogam
Copy link
Copy Markdown
Contributor Author

pevogam commented Aug 28, 2025

I see, the time the pull request added 3.12 and 3.13 was before the 3.8 was dropped on master so this change ended up reintroducing the 3.8. Nice catch, must be fixed now.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (3)
.github/workflows/ci.yml (3)

99-101: Use python -m pip consistently.

Minor consistency nit: prefer python -m pip for all pip invocations.

Apply:

-        python -m pip install --upgrade pip
-        pip install build setuptools wheel
+        python -m pip install --upgrade pip
+        python -m pip install build setuptools wheel

94-98: Optional: add pip cache to package-build for speed.

Mirroring travis-check, caching pip here speeds builds and is safe.

     - name: Set up Python
       uses: actions/setup-python@v5
       with:
         python-version: ${{ matrix.python-version }}
-        check-latest: true
+        check-latest: true
+        cache: 'pip'
+        # Cache invalidates when build tooling changes
+        cache-dependency-path: |
+          pyproject.toml
+          setup.cfg
+          setup.py
+          requirements*.txt

88-88: Heads-up: actionlint 1.7.7 errors on YAML anchors in matrix.python-version

actionlint reports on line 88:
.github/workflows/ci.yml:88:25: "matrix values" section must be sequence node but got alias node with "" tag [syntax-check]

Options (pick one):

  • Inline the Python versions list in this job instead of using the *py_versions anchor.
  • Upgrade actionlint to a version that tolerates aliases in matrix values.
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled
  • Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 193c3c5 and f55c8da.

📒 Files selected for processing (1)
  • .github/workflows/ci.yml (4 hunks)
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: pevogam
PR: avocado-framework/avocado-vt#4101
File: .github/workflows/ci.yml:36-39
Timestamp: 2025-08-25T15:08:02.212Z
Learning: For GitHub Actions workflows in avocado-vt, pevogam prefers using YAML anchors (&py_versions and *py_versions) over fromJSON(env.PY_VERSIONS) for centralizing Python version matrices. Do not repeatedly suggest the fromJSON approach when YAML anchors are being used.
📚 Learning: 2025-08-25T15:08:02.212Z
Learnt from: pevogam
PR: avocado-framework/avocado-vt#4101
File: .github/workflows/ci.yml:36-39
Timestamp: 2025-08-25T15:08:02.212Z
Learning: For GitHub Actions workflows in avocado-vt, pevogam prefers using YAML anchors (&py_versions and *py_versions) over fromJSON(env.PY_VERSIONS) for centralizing Python version matrices. Do not repeatedly suggest the fromJSON approach when YAML anchors are being used.

Applied to files:

  • .github/workflows/ci.yml
🪛 actionlint (1.7.7)
.github/workflows/ci.yml

88-88: "matrix values" section must be sequence node but got alias node with "" tag

(syntax-check)


105-105: property "python-version" is not defined in object type {}

(expression)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Static checks
🔇 Additional comments (4)
.github/workflows/ci.yml (4)

38-38: Good centralization of Python versions with a YAML anchor.

Matches the repo preference and keeps both matrices in sync.


51-53: Setup-python caching and check-latest look good.

cache-dependency-path points to requirements-travis.txt, which should give deterministic cache busting here.


97-97: check-latest on package-build is fine.

Consistent with the other job.


104-106: Eggs step gating verified
Step triggers only for Python 3.9–3.11 as intended. No changes required.

Copy link
Copy Markdown
Contributor

@richtja richtja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pevogam LGTM, thank you

@richtja
Copy link
Copy Markdown
Contributor

richtja commented Sep 2, 2025

@pevogam , sorry I was in the wrong context, name: Build Package for Python ${{ matrix.python-version }} will not install requirements-travis.txt, so have to updte its

      run: |
        python -m pip install --upgrade pip
        pip install build

to

      run: |
        python -m pip install --upgrade pip
        pip install build setuptools

For CI env, I don't think it's a problem that install setuptools even if it's installed(python<=3.11). But let's listen the voice from @richtja and @luckyh

Hi @pevogam and @PaulYuuu, FYI we do the same on avocado side, see.

@luckyh
Copy link
Copy Markdown
Contributor

luckyh commented Sep 4, 2025

Merging this, thanks to all!

@luckyh luckyh merged commit a3ff20b into avocado-framework:master Sep 4, 2025
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants