Nightly upload

This is a GitHub Action that uploads nightly builds to the scientific-python nightly channel, as recommended in SPEC4 — Using and Creating Nightly Wheels.

In a GitHub Actions workflow (.github/workflows/*.yaml), use the following snippet on a Linux, macOS, or x86 Windows runner to upload built wheels to the channel:

jobs:
  steps:
    ...
    - name: Upload wheel
      uses: scientific-python/upload-nightly-action@e76cfec8a4611fd02808a801b0ff5a7d7c1b2d99 # 0.6.4
      with:
        artifacts_path: dist
        anaconda_nightly_upload_token: ${{secrets.UPLOAD_TOKEN}}

Note that we recommend pinning the action against a specific SHA (rather than a tag), to guard against the unlikely event of upstream being compromised.

Updating the action

You can use Dependabot to keep the GitHub Action up to date, with a .github/dependabot.yml config file similar to:

version: 2
updates:
  # Maintain dependencies for GitHub Actions
  - package-ecosystem: "github-actions"
    directory: "/"
    schedule:
      interval: "weekly"

Access to the scientific-python-nightly-wheels channel

To request access to the wheel channel, please open an issue on the upload action's repository. You can then generate a token at https://anaconda.org/<anaconda.org user name>/settings/access with permissions to Allow write access to the API site and Allow uploads to Standard Python repositories, and add the token as a secret to your GitHub repository.

Using a different channel

This Github Action can upload your nightly builds to a different channel. To do so, define the anaconda_nightly_upload_organization variable. Furthermore, you can add labels for organizing your artifacts using anaconda_nightly_upload_labels optional parameter. See below:

jobs:
  steps:
    ...
    - name: Upload wheel
      uses: scientific-python/upload-nightly-action@e76cfec8a4611fd02808a801b0ff5a7d7c1b2d99 # 0.6.4
      with:
        artifacts_path: dist
        anaconda_nightly_upload_organization: my-alternative-organization
        anaconda_nightly_upload_token: ${{secrets.UPLOAD_TOKEN}}
        anaconda_nightly_upload_labels: dev

Artifact cleanup-policy at the scientific-python-nightly-wheels channel

To avoid hosting outdated development versions, as well as to clean up space, we do have a default retention policy of:

• Latest 5 versions
• Artifacts newer than 30 days

Any versions beyond these are automatically removed as part of a daily cron job run from this repository. Projects may have reasons to request to be added to the list exempt from this automated cleanup, however in that case the responsibility of cleaning-up old, unused versions fall back on the individual project.

If you are a low traffic project this policy might catch you out, as the last wheel will be removed from this channel 30 days after it was uploaded. A low activity project might not receive updates often enough to produce a new wheel every 30 days. We recommend that you upload a nightly wheel on a regular cadence even if there are no changes.

In addition cron jobs in GitHub repositories will be disabled after a certain amount of inactivity. We are not aware of a solution other than regular commit activity to prevent the deactivation of scheduled cron jobs.

Stale wheel reminders

So that the removal above does not come as a surprise, a daily job in this repository checks how long ago each package was last uploaded. If it has been more than 15 days, the job opens an issue on that project's own issue tracker naming the date its wheels will be removed, comments once more at 25 days, and closes the issue automatically once a new wheel is uploaded. Issues are opened by @scientific-python-bot; the repository to report to is taken from the project's PyPI metadata (Project-URL), so keeping those URLs pointed at your GitHub repository is enough to be reachable.

The check runs daily from tools/check_stale_wheels.py in this repository, an hour before the cleanup job that does the deleting. If your project's PyPI metadata carries no GitHub URL we can follow, we have no way to reach you: the run records that in an issue here instead, and the fix is either to add a Project-URL upstream or to ask us to map the package by hand. Maintainers of this repository: the workflow posts with ISSUE_OPENER_TOKEN, a token belonging to the bot account with the public_repo scope, and falls back to the workflow's own GITHUB_TOKEN when reporting here.

Reporting a failing scheduled workflow

A cron job that quietly stops working can go unnoticed for months, which is the same problem the reminders above exist to solve. The scheduled workflows here each call report-failure.yml, a small reusable workflow that opens one issue when a run fails and rewrites that issue on later failures rather than adding a comment a day.

Other projects are welcome to call it too:

jobs:
  nightly:
    ...

  report-failure:
    needs: [nightly]
    if: failure() && github.event_name == 'schedule'
    permissions:
      issues: write
    uses: scientific-python/upload-nightly-action/.github/workflows/report-failure.yml@main
    with:
      title: 'The nightly wheel build is failing'

The issue is opened in the calling repository by github-actions[bot], using that workflow's own GITHUB_TOKEN, so no secret is needed. The title doubles as the identity of the issue, so keep it stable. As with the action itself, we recommend pinning to a specific SHA rather than to main.

Using nightly builds in CI

To test against nightly builds, you can use the following command to install from the nightly channel:

python -m pip install \
  --upgrade \
  --pre \
  --index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple \
  --extra-index-url https://pypi.org/simple \
  matplotlib

Note that --index-url takes priority over --extra-index-url, so that packages, and their dependencies, with versions available in the nightly channel will be installed before falling back to the Python Package Index.

To install nightly builds within a conda environment, specify an extra index in your environment.yml:

name: test
dependencies:
  - pip
  - pip:
    - --pre --index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple --extra-index-url https://pypi.org/simple
    - matplotlib