Skip to content

doc changes for Github Artifacts #593

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 19 commits into from
Closed

doc changes for Github Artifacts #593

Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
096bc1e
doc changes for Github Artifacts
VenkateshJaya Apr 14, 2025
662b728
removed trailing white space
VenkateshJaya Apr 14, 2025
03e0441
changed -e to --env
VenkateshJaya Apr 14, 2025
6e30ef3
updated images
VenkateshJaya Apr 14, 2025
89b3afe
fixed grammar
VenkateshJaya Apr 14, 2025
4140053
Update resources/github-actions.md
VenkateshJaya Apr 15, 2025
035961c
Update resources/github-actions.md
VenkateshJaya Apr 15, 2025
ec14bf7
Update resources/github-actions.md
VenkateshJaya Apr 15, 2025
82aa001
Update resources/github-actions.md
VenkateshJaya Apr 15, 2025
d47cabc
Update resources/github-actions.md
VenkateshJaya Apr 15, 2025
813e4b6
Update resources/github-actions.md
VenkateshJaya Apr 15, 2025
ab5c0f8
Merge branch 'main' into main
VenkateshJaya Apr 15, 2025
45dc235
updated other references to S3
VenkateshJaya Apr 22, 2025
81c462a
Merge remote-tracking branch 'origin/main'
VenkateshJaya Apr 22, 2025
ae9fa0e
Merge branch 'main' into main
VenkateshJaya Apr 22, 2025
9e8706a
Merge branch 'main' into main
VenkateshJaya Apr 28, 2025
fa95ca1
Merge pull request #1 from rapidsai/main
VenkateshJaya May 2, 2025
1eaf8ee
updated links and gh auth notes
VenkateshJaya May 2, 2025
6b883ca
removed trailing whitespace
VenkateshJaya May 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added assets/images/downloads-github.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/images/workflow-ui.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
74 changes: 44 additions & 30 deletions resources/github-actions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,45 +123,59 @@ As an example, `rapids-print-env` is used to print common environment informatio

## Downloading CI Artifacts

For NVIDIA employees with VPN access, artifacts from both pull-requests and branch builds can be accessed on [https://downloads.rapids.ai/](https://downloads.rapids.ai/).
Artifacts from both pull-requests and branch builds can be accessed on the GitHub UI for the specific workflow run, found in the Actions tab of the repository.

There is a link provided at the end of every C++ and Python build job where the build artifacts for that particular workflow run can be accessed.
![](/assets/images/workflow-ui.png)

![](/assets/images/downloads.png)
There is also a link provided in the `Run actions/upload-artifact@v4` step of every C++ and Python build job where the build artifacts for that particular job can be accessed.

![](/assets/images/downloads-github.png)

This link downloads the required artifact as a zip file.

This can also be done using the following `gh` CLI command to download the artifact. This command also unzips the artifact to the destination location:

```sh
gh run download <workflow-run-id> --repo <repo-name> --name <artifact-name> --dir <destination-directory>
```

For example, to download the artifact `rmm_conda_python_cuda11_py311_x86_64` from the workflow run ID `14437867406` on the `rmm` repository into the `/artifacts` directory, you can use this command:

```sh
gh run download 14437867406 --repo rapidsai/rmm --name rmm_conda_python_cuda11_py311_x86_64 --dir /artifacts
```

## Using Conda CI Artifacts Locally

The artifacts that result from running `conda build` are conda channels. RAPIDS' CI system then compresses these conda channels into tarballs and uploads them to [https://downloads.rapids.ai/](https://downloads.rapids.ai/).
The artifacts that result from running `conda build` are conda channels. RAPIDS' CI system uploads these conda channels to GitHub Artifacts as zip files.

The packages in the conda channel can be used by extracting the tarball to your local filesystem and using the resulting path in your conda commands.
The packages in the conda channel can be used by downloading the artifact to your local filesystem using `gh` CLI and using the resulting path in your conda commands.

For example, the following snippet will download a pull request artifact for `librmm` and install it into the active conda environment:
For example, the following snippet will download an artifact for `librmm` from workflow run ID `14437867406` and install it into the active conda environment:

```sh
wget https://downloads.rapids.ai/ci/rmm/pull-request/1376/5124d43/rmm_conda_cpp_cuda11_x86_64.tar.gz
mkdir local_channel
tar xzf rmm_conda_cpp_cuda11_x86_64.tar.gz -C local_channel/
# Download and extract the artifact
gh run download 14437867406 --repo rapidsai/rmm -name rmm_conda_cpp_cuda11_x86_64 --dir local_channel
mamba install --channel file://local_channel --channel rapidsai-nightly --channel conda-forge --channel nvidia librmm
```

Note that CI artifacts can only be downloaded while connected to the NVIDIA VPN.
Note: Make sure you have the GitHub CLI (`gh`) installed and authenticated on your host machine to download artifacts from GitHub Artifacts. To download artifacts made for a specific PR, replace the workflow run ID with the ID of the run triggered by the PR branch.

## Using Wheel CI Artifacts Locally

RAPIDS' CI system compresses the wheels that it builds into tarballs and uploads them to [https://downloads.rapids.ai/](https://downloads.rapids.ai/).
RAPIDS' CI system compresses the wheels that it builds into zip files and uploads them to GitHub Artifacts.

The wheels can be used by extracting the tarball to your local filesystem and using the resulting path in your pip commands.
The wheels can be used by downloading the artifact to your local filesystem using `gh` CLI and using the resulting path in your pip commands.

For example, the following snippet will download a pull request artifact for `librmm` and install it into the active conda environment:
For example, the following snippet will download an artifact for `librmm` and install it into the active conda environment:

```sh
wget https://downloads.rapids.ai/ci/rmm/pull-request/1376/5124d43/rmm_wheel_python_rmm_cu12_39_x86_64.tar.gz
mkdir wheels
tar xzf rmm_wheel_python_rmm_cu12_39_x86_64.tar.gz -C wheels/
pip install wheels/rmm_cu12-24.2.0a1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
# Download and extract the artifact
gh run download 14437867406 --repo rapidsai/rmm --name rmm_wheel_python_rmm_cu12_py312_x86_64 --dir wheels
pip install wheels/rmm_cu12-25.6.0a23-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl
```

Note that CI artifacts can only be downloaded while connected to the NVIDIA VPN.
Note: Make sure you have the GitHub CLI (`gh`) installed and authenticated on your host machine to download artifacts from GitHub Artifacts. To download artifacts made for a specific PR, replace the workflow run ID with the ID of the run triggered by the PR branch.

## Using Conda CI Artifacts in Other PRs

Expand All @@ -184,8 +198,8 @@ Add a new file called `ci/use_conda_packages_from_prs.sh`.
# ci/use_conda_packages_from_prs.sh

# download CI artifacts
LIBRAFT_CHANNEL=$(rapids-get-pr-conda-artifact raft 1388 cpp)
LIBRMM_CHANNEL=$(rapids-get-pr-conda-artifact rmm 1095 cpp)
LIBRAFT_CHANNEL=$(rapids-get-pr-conda-artifact-github raft 1388 cpp)
LIBRMM_CHANNEL=$(rapids-get-pr-conda-artifact-github rmm 1095 cpp)

# make sure they can be found locally
conda config --system --add channels "${LIBRAFT_CHANNEL}"
Expand All @@ -207,9 +221,9 @@ So, for example, Python testing jobs that use the `rmm` Python package also need
# ci/use_conda_packages_from_prs.sh

# download CI artifacts
LIBKVIKIO_CHANNEL=$(rapids-get-pr-conda-artifact kvikio 224 cpp)
LIBRMM_CHANNEL=$(rapids-get-pr-conda-artifact rmm 1223 cpp)
RMM_CHANNEL=$(rapids-get-pr-conda-artifact rmm 1223 python)
LIBKVIKIO_CHANNEL=$(rapids-get-pr-conda-artifact-github kvikio 224 cpp)
LIBRMM_CHANNEL=$(rapids-get-pr-conda-artifact-github rmm 1223 cpp)
RMM_CHANNEL=$(rapids-get-pr-conda-artifact-github rmm 1223 python)

# make sure they can be found locally
conda config --system --add channels "${LIBKVIKIO_CHANNEL}"
Expand All @@ -223,7 +237,7 @@ Then copy the following into every script in the `ci/` directory that is doing `
source ./ci/use_conda_packages_from_prs.sh
```

**Note:** By default `rapids-get-pr-conda-artifact` uses the most recent commit from the specified PR.
**Note:** By default `rapids-get-pr-conda-artifact-github` uses the most recent commit from the specified PR.
A commit hash from the dependent PR can be added as an optional 4th argument to pin testing to a specific commit.

## Using Wheel CI Artifacts in Other PRs
Expand All @@ -246,10 +260,10 @@ RAPIDS_PY_CUDA_SUFFIX=$(rapids-wheel-ctk-name-gen "${RAPIDS_CUDA_VERSION}")

# download wheels, store the directories holding them in variables
LIBRMM_WHEELHOUSE=$(
RAPIDS_PY_WHEEL_NAME="librmm_${RAPIDS_PY_CUDA_SUFFIX}" rapids-get-pr-wheel-artifact rmm 1678 cpp
RAPIDS_PY_WHEEL_NAME="librmm_${RAPIDS_PY_CUDA_SUFFIX}" rapids-get-pr-wheel-artifact-github rmm 1678 cpp
)
LIBRAFT_WHEELHOUSE=$(
RAPIDS_PY_WHEEL_NAME="libraft_${RAPIDS_PY_CUDA_SUFFIX}" rapids-get-pr-wheel-artifact raft 2433 cpp
RAPIDS_PY_WHEEL_NAME="libraft_${RAPIDS_PY_CUDA_SUFFIX}" rapids-get-pr-wheel-artifact-github raft 2433 cpp
)

# write a pip constraints file saying e.g. "whenever you encounter a requirement for 'librmm-cu12', use this wheel"
Expand Down Expand Up @@ -283,13 +297,13 @@ RAPIDS_PY_CUDA_SUFFIX=$(rapids-wheel-ctk-name-gen "${RAPIDS_CUDA_VERSION}")

# download wheels, store the directories holding them in variables
LIBKVIKIO_WHEELHOUSE=$(
RAPIDS_PY_WHEEL_NAME="libkvikio_${RAPIDS_PY_CUDA_SUFFIX}" rapids-get-pr-wheel-artifact kvikio 510 cpp
RAPIDS_PY_WHEEL_NAME="libkvikio_${RAPIDS_PY_CUDA_SUFFIX}" rapids-get-pr-wheel-artifact-github kvikio 510 cpp
)
LIBRMM_WHEELHOUSE=$(
RAPIDS_PY_WHEEL_NAME="librmm_${RAPIDS_PY_CUDA_SUFFIX}" rapids-get-pr-wheel-artifact rmm 1678 cpp
RAPIDS_PY_WHEEL_NAME="librmm_${RAPIDS_PY_CUDA_SUFFIX}" rapids-get-pr-wheel-artifact-github rmm 1678 cpp
)
RMM_WHEELHOUSE=$(
RAPIDS_PY_WHEEL_NAME="rmm_${RAPIDS_PY_CUDA_SUFFIX}" rapids-get-pr-wheel-artifact rmm 1678 python
RAPIDS_PY_WHEEL_NAME="rmm_${RAPIDS_PY_CUDA_SUFFIX}" rapids-get-pr-wheel-artifact-github rmm 1678 python
)

# write a pip constraints file saying e.g. "whenever you encounter a requirement for 'librmm-cu12', use this wheel"
Expand All @@ -311,7 +325,7 @@ source ./ci/use_wheels_from_prs.sh
As above, if any of the other CI scripts are already setting the environment variable `PIP_CONSTRAINT`, you may need to
modify them slightly to ensure they **append to**, instead of **overwriting**, the constraints set up by `use_wheels_from_prs.sh`.

**Note:** By default `rapids-get-pr-wheel-artifact` uses the most recent commit from the specified PR.
**Note:** By default `rapids-get-pr-wheel-artifact-github` uses the most recent commit from the specified PR.
A commit hash from the dependent PR can be added as an optional 4th argument to pin testing to a specific commit.

## Skipping CI for Commits
Expand Down
25 changes: 13 additions & 12 deletions resources/reproducing-ci.md
View file
Open in desktop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Limitations section on this page has a few S3 references that should be updated as well.

Copy link
Author

@VenkateshJaya VenkateshJaya Apr 15, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The references in that section primarily refer to downloads. Considering we didn't switch over downloads just yet, I did not change those sections of the documentation. Let me know if you want me to preemptively change them.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can preemptively make those changes. I assume we won't merge any of these doc updates until after all of the repository PRs are completely merged.

Original file line number Diff line number Diff line change
Expand Up @@ -51,12 +51,14 @@ docker run \
-it \
--gpus all \
--pull=always \
--network=host \
--env GITHUB_TOKEN=$(gh auth token) \
--volume $PWD:/repo \
--workdir /repo \
rapidsai/ci-conda:cuda11.8.0-ubuntu22.04-py3.10
```

Note: The `--env GITHUB_TOKEN=$(gh auth token)` flag is required to authenticate with GitHub for accessing artifacts. This token is automatically generated from your GitHub CLI authentication and passed to the container. Make sure you have the GitHub CLI (`gh`) installed and authenticated on your host machine.
Copy link
Member

@pentschev pentschev Apr 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if I'm working on a remote machine where gh isn't installed? Can I pass a custom GH token? What permissions do I need at a minimum to prevent the need for a token with too broad scope (and thus reduce security concerns)? Could we document that?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll get all of this into docs in the replacement PR (#601), but to answer here:

What if I'm working on a remote machine where gh isn't installed?

The CI images come with the gh CLI installed already and the relevant gha-tools tools will now prompt you to login interactively if it looks like you haven't authenticated with GitHub.

See rapidsai/gha-tools#172

That interactive auth will last for as long as the container stays up, and won't leave any credentials behind on the system you ran docker run from.

Can I pass a custom GH token? What permissions do I need at a minimum to prevent the need for a token with too broad scope (and thus reduce security concerns)?

Yes, you can provide a GitHub personal access token in the GH_TOKEN environment variable for any processes using the gh CLI to pull these artifacts.

It only needs the repo scope (see "How I tested this" in the description of rapidsai/gha-tools#172)

Could we document that?

Yes, I'll explain these topics in #601


Once the container has started, you can run any of the CI scripts inside of it:

```sh
Expand All @@ -83,11 +85,11 @@ The `docker` command above makes the follow assumptions:

- Your current directory is the repository that you wish to troubleshoot
- Your current directory has the same commit checked out as the pull-request whose jobs you are trying to debug
- You have the GitHub CLI (`gh`) installed and authenticated on your host machine

A few notes about the `docker` command flags:

- Most of the RAPIDS conda builds occur on machines without GPUs. Only the tests require GPUs. Therefore, you can omit the `--gpus` flag when running local conda builds
- The `--network` flag ensures that the container has access to the VPN connection on your host machine. VPN connectivity is required for test jobs since they need access to [downloads.rapids.ai](https://downloads.rapids.ai) for downloading build artifacts from a particular pull-request. This flag can be omitted for build jobs


## Additional Considerations
Expand All @@ -108,13 +110,14 @@ In RAPIDS CI workflows, the builds and tests occur on different machines.

Machines without GPUs are used for builds, while the tests occur on machines with GPUs.

Due to this process, the artifacts from the build jobs must be downloaded from [downloads.rapids.ai](https://downloads.rapids.ai) in order for the test jobs to run.
Due to this process, the artifacts from the build jobs must be downloaded from GitHub Artifacts in order for the test jobs to run.

In CI, this process happens transparently.

Local builds lack the context provided by the CI environment and therefore will require input from the user in order to ensure that the correct artifacts are downloaded.

Any time the `rapids-download-conda-from-s3` command (e.g. [here](https://github.com/rapidsai/cugraph/blob/b50850f0498e163e56b0374c1c64e551a5898f26/ci/test_python.sh#L22-L23)) is encountered in a local test run, the user will be prompted for any necessary environment variables that are missing.
Any time the `rapids-download-conda-from-github` command (e.g. [here](https://github.com/rapidsai/cugraph/blob/b50850f0498e163e56b0374c1c64e551a5898f26/ci/test_python.sh#L22-L23)) is encountered in a local test run, the user will be prompted for any necessary environment variables that are missing.
<!-- The reference to the file needs to be changed once the downloading from GH PRs are merged -->

The screenshot below shows an example.

Expand All @@ -140,21 +143,21 @@ There are a few limitations to keep in mind when running CI scripts locally.

### Local Artifacts Cannot Be Uploaded

Build artifacts from local jobs cannot be uploaded to [downloads.rapids.ai](https://downloads.rapids.ai).
Build artifacts from local jobs cannot be uploaded to GitHub Artifacts.

If builds are failing in CI, developers should fix the problem locally and then push their changes to a pull-request.

Then CI jobs can run and the fixed build artifacts will be made available for the test job to download and use.

To attempt a complete build and test workflow locally, you can manually update any instances of `CPP_CHANNEL` and `PYTHON_CHANNEL` that use `rapids-download-conda-from-s3` (e.g. [1](https://github.com/rapidsai/cuml/blob/dc38afc584154ebe7332d43f69e3913492f7a273/ci/build_python.sh#L14),[2](https://github.com/rapidsai/cuml/blob/dc38afc584154ebe7332d43f69e3913492f7a273/ci/test_python_common.sh#L22-L23)) with the value of the `RAPIDS_CONDA_BLD_OUTPUT_DIR` environment variable that is [set in our CI images](https://github.com/rapidsai/ci-imgs/blob/d048ffa6bfd672fa72f31aeb7cc5cf2363aff6d9/Dockerfile#L105).
To attempt a complete build and test workflow locally, you can manually update any instances of `CPP_CHANNEL` and `PYTHON_CHANNEL` that use `rapids-download-conda-from-github` (e.g. [1](https://github.com/rapidsai/cuml/blob/dc38afc584154ebe7332d43f69e3913492f7a273/ci/build_python.sh#L14),[2](https://github.com/rapidsai/cuml/blob/dc38afc584154ebe7332d43f69e3913492f7a273/ci/test_python_common.sh#L22-L23)) with the value of the `RAPIDS_CONDA_BLD_OUTPUT_DIR` environment variable that is [set in our CI images](https://github.com/rapidsai/ci-imgs/blob/d048ffa6bfd672fa72f31aeb7cc5cf2363aff6d9/Dockerfile#L105).

This value is used to set the `output_folder` of the `.condarc` file used in our CI images (see [docs](https://conda.io/projects/conda/en/latest/user-guide/configuration/use-condarc.html#specify-conda-build-build-folder-conda-build-3-16-3-output-folder)). Therefore, any locally built packages will end up in this directory.

For example:

```sh
# Replace all local uses of `rapids-download-conda-from-s3`
sed -ri '/rapids-download-conda-from-s3/ s/_CHANNEL=.*/_CHANNEL=${RAPIDS_CONDA_BLD_OUTPUT_DIR}/' ci/*.sh
# Replace all local uses of `rapids-download-conda-from-github`
sed -ri '/rapids-download-conda-from-github/ s/_CHANNEL=.*/_CHANNEL=${RAPIDS_CONDA_BLD_OUTPUT_DIR}/' ci/*.sh

# Run the sequence of build/test scripts
./ci/build_cpp.sh
Expand All @@ -165,11 +168,9 @@ sed -ri '/rapids-download-conda-from-s3/ s/_CHANNEL=.*/_CHANNEL=${RAPIDS_CONDA_B
./ci/build_docs.sh
```

### VPN Access

Currently, [downloads.rapids.ai](https://downloads.rapids.ai) is only available via the NVIDIA VPN.
Similarly, you can manually update any instance of the environment variable `RAPIDS_WHEEL_BLD_OUTPUT_DIR` to set the destination location for newly built wheels. (e.g. [1](https://github.com/rapidsai/cudf/blob/05646df0b8dd4b69f7cfed6fbf9e882df795210c/ci/build_wheel_cudf.sh#L29),[2](https://github.com/rapidsai/cuml/blob/e02797cabbd54a328fb22a48b5f1f02c4b6c81ee/ci/build_wheel_cuml.sh#L38)).

If you want to run any test jobs locally, you'll need to be connected to the VPN to download CI build artifacts.
This value is set to `/tmp/wheelhouse` in our [CI images](https://github.com/rapidsai/ci-imgs/blob/adc9f61a0c9d37b21b9ce0a978681e406a00bc64/ci-wheel.Dockerfile#L27)

### Some Builds Rely on Versioning Information in Git Tags

Expand Down