Skip to content

Allow running sanity and unit tests with antsibull-nox; impove testing documentation #10104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 17, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 66 additions & 7 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,49 @@ If you want to test a PR locally, refer to [our testing guide](https://github.co

If you find any inconsistencies or places in this document which can be improved, feel free to raise an issue or pull request to fix it.

## Run sanity, unit or integration tests locally
## Run sanity or unit locally (with antsibull-nox)

The easiest way to run sanity and unit tests locally is to use [antsibull-nox](https://ansible.readthedocs.io/projects/antsibull-nox/).
(If you have [nox](https://nox.thea.codes/en/stable/) installed, it will automatically install antsibull-nox in a virtual environment for you.)

### Sanity tests

The following commands show how to run ansible-test sanity tests:

```.bash
# Run basic sanity tests for all files in the collection:
nox -Re ansible-test-sanity-devel

# Run basic sanity tests for the given files and directories:
nox -Re ansible-test-sanity-devel -- plugins/modules/system/pids.py tests/integration/targets/pids/

# Run all other sanity tests for all files in the collection:
nox -R
```

If you replace `-Re` with `-e`, respectively. leave `-R` away, then the virtual environments will be re-created. The `-R` re-uses them (if they already exist).

### Unit tests

The following commands show how to run unit tests:

```.bash
# Run all unit tests:
nox -Re ansible-test-units-devel

# Run all unit tests for one Python version (a lot faster):
nox -Re ansible-test-units-devel -- --python 3.13

# Run a specific unit test (for the nmcli module) for one Python version:
nox -Re ansible-test-units-devel -- --python 3.13 tests/unit/plugins/modules/net_tools/test_nmcli.py
```

If you replace `-Re` with `-e`, then the virtual environments will be re-created. The `-R` re-uses them (if they already exist).

## Run basic sanity, unit or integration tests locally (with ansible-test)

Instead of using antsibull-nox, you can also run sanity and unit tests with ansible-test directly.
This also allows you to run integration tests.

You have to check out the repository into a specific path structure to be able to run `ansible-test`. The path to the git checkout must end with `.../ansible_collections/community/general`. Please see [our testing guide](https://github.com/ansible/community-docs/blob/main/test_pr_locally_guide.rst) for instructions on how to check out the repository into a correct path structure. The short version of these instructions is:

Expand All @@ -56,20 +98,27 @@ cd ~/dev/ansible_collections/community/general

Then you can run `ansible-test` (which is a part of [ansible-core](https://pypi.org/project/ansible-core/)) inside the checkout. The following example commands expect that you have installed Docker or Podman. Note that Podman has only been supported by more recent ansible-core releases. If you are using Docker, the following will work with Ansible 2.9+.

### Sanity tests
### Basic sanity tests

The following commands show how to run sanity tests:
The following commands show how to run basic sanity tests:

```.bash
# Run sanity tests for all files in the collection:
# Run basic sanity tests for all files in the collection:
ansible-test sanity --docker -v

# Run sanity tests for the given files and directories:
# Run basic sanity tests for the given files and directories:
ansible-test sanity --docker -v plugins/modules/system/pids.py tests/integration/targets/pids/
```

### Unit tests

Note that for running unit tests, you need to install required collections in the same folder structure that `community.general` is checked out in.
Right now, you need to install [`community.internal_test_tools`](https://github.com/ansible-collections/community.internal_test_tools).
If you want to use the latest version from GitHub, you can run:
```
git clone https://github.com/ansible-collections/community.internal_test_tools.git ~/dev/ansible_collections/community/internal_test_tools
```

The following commands show how to run unit tests:

```.bash
Expand All @@ -85,15 +134,25 @@ ansible-test units --docker -v --python 3.8 tests/unit/plugins/modules/net_tools

### Integration tests

Note that for running integration tests, you need to install required collections in the same folder structure that `community.general` is checked out in.
Right now, depending on the test, you need to install [`ansible.posix`](https://github.com/ansible-collections/ansible.posix), [`community.crypto`](https://github.com/ansible-collections/community.crypto), and [`community.docker`](https://github.com/ansible-collections/community.docker):
If you want to use the latest versions from GitHub, you can run:
```
mkdir -p ~/dev/ansible_collections/ansible
git clone https://github.com/ansible-collections/ansible.posix.git ~/dev/ansible_collections/ansible/posix
git clone https://github.com/ansible-collections/community.crypto.git ~/dev/ansible_collections/community/crypto
git clone https://github.com/ansible-collections/community.docker.git ~/dev/ansible_collections/community/docker
```

The following commands show how to run integration tests:

#### In Docker

Integration tests on Docker have the following parameters:
- `image_name` (required): The name of the Docker image. To get the list of supported Docker images, run
`ansible-test integration --help` and look for _target docker images_.
- `test_name` (optional): The name of the integration test.
For modules, this equals the short name of the module; for example, `pacman` in case of `community.general.pacman`.
- `test_name` (optional): The name of the integration test.
For modules, this equals the short name of the module; for example, `pacman` in case of `community.general.pacman`.
For plugins, the plugin type is added before the plugin's short name, for example `callback_yaml` for the `community.general.yaml` callback.
```.bash
# Test all plugins/modules on fedora40
Expand Down
6 changes: 6 additions & 0 deletions antsibull-nox.toml
Original file line number Diff line number Diff line change
Expand Up @@ -46,3 +46,9 @@ doc_fragment = "community.general.proxmox.actiongroup_proxmox"

[sessions.build_import_check]
run_galaxy_importer = true

[sessions.ansible_test_sanity]
include_devel = true

[sessions.ansible_test_units]
include_devel = true