Skip to content

GETTING_STARTED.md

Latest commit

 

History

History
183 lines (117 loc) · 4.86 KB

GETTING_STARTED.md

File metadata and controls

183 lines (117 loc) · 4.86 KB

Getting started

Installation

Install uv

Tests cluster

These tests can be executed against arbitrary cluster with ODH / RHOAI installed.

You can log in into such cluster via:

oc login -u user -p password

Or by setting KUBECONFIG variable:

KUBECONFIG=<kubeconfig file>

or by saving the kubeconfig file under ~/.kube/config

OpenShift CLI (oc) Binary

By default, the test framework automatically downloads the OpenShift CLI binary from the target cluster's console CLI download service. This ensures compatibility between the client and cluster versions.

Using a Local oc Binary

If you already have the oc binary installed locally, you can avoid the download by setting the OC_BINARY_PATH environment variable:

export OC_BINARY_PATH=/usr/local/bin/oc

Or run tests with the variable:

OC_BINARY_PATH=/usr/local/bin/oc uv run pytest

Note: Ensure your local oc binary is executable and compatible with your target cluster version.

Must gather

In order to collect must-gather on failure point one may use --collect-must-gather to the pytest command. e.g.

uv run pytest tests/<your component> --collect-must-gather

By default, the collected must-gather would be archived. To skip archiving, please set environment variable ARCHIVE_MUST_GATHER to any value other than "true". e.g.

export ARCHIVE_MUST_GATHER="false"

Benefits of Using Local Binary

  • Faster test startup (no download time)
  • Consistent tooling across different test runs
  • Useful in air-gapped environments or when internet access is limited

Running the tests

Basic run of all tests

uv run pytest

To see optional CLI arguments run:

uv run pytest --help

Using CLI arguments

CLI arguments can be passed to pytest by setting them in pytest.ini.
You can either use the default pytest.ini file and pass CLI arguments or create a custom one.
For example, add the below under the addopts section:

    --ci-s3-bucket-name=name
    --ci-s3-bucket-endpoint=endpoint-path
    --ci-s3-bucket-region=region

Then pass the path to the custom pytest.ini file to pytest:

uv run pytest -c custom-pytest.ini

Turning off console logging

By default, pytest will output logging reports in the console. You can disable this behavior with -o log_cli=false

uv run pytest -o log_cli=false

Running specific tests

uv run pytest -k test_name

Running component smoke

uv run pytest tests/<component_name> -m "smoke and not sanity and not tier1"

LlamaStack Integration Tests

For more information about LlamaStack integration tests, see /tests/llama_stack/README.md.

Running on different distributions

By default, RHOAI distribution is set.
To run on ODH, pass --tc=distribution:upstream to pytest.

Skip cluster sanity checks

By default, cluster sanity checks are run to make cluster ready for tests. To skip cluster sanity checks, pass --cluster-sanity-skip-check to skip all tests. To skip RHOAI/ODH-related tests (for example when running in upstream), pass --cluster-sanity-skip-rhoai-check.

Running tests with admin client instead of unprivileged client

To run tests with admin client only, pass --tc=use_unprivileged_client:False to pytest.

Skipping tests for known Jira issues

To skip a test that is affected by a known Jira bug, use pytest.mark.xfail with is_jira_issue_open as the condition:

from utilities.jira import is_jira_issue_open

@pytest.mark.xfail(condition=is_jira_issue_open(jira_id="RHOAIENG-12345"), reason="RHOAIENG-12345", run=False)
def test_example(self):
    ...
  • condition=is_jira_issue_open(...) checks the Jira issue status at collection time. If the issue is open, the test is marked as xfail.
  • run=False skips execution entirely while the issue is open. Once the issue is closed/resolved, the test runs normally.
  • If Jira is unreachable, the issue is assumed open and the test is skipped.

The following environment variables must be set for Jira connectivity:

export PYTEST_JIRA_URL=<your_jira_url>
export PYTEST_JIRA_USERNAME=<username>
export PYTEST_JIRA_TOKEN=<token>

Running containerized tests

Save kubeconfig file to a local directory, for example: $HOME/kubeconfig To run tests in containerized environment:

podman run  -v $HOME:/mnt/host:Z  -e KUBECONFIG=/mnt/host/kubeconfig quay.io/opendatahub/opendatahub-tests

Debugging test failures

Log output

Console output uses structlog's ConsoleRenderer (human-readable, colorized). The log file (pytest-tests.log) uses structlog's JSONRenderer (structured JSON).

To convert the JSON log file to a readable format:

jq -r '"\(.timestamp) [\(.level)] \(.logger): \(.event)"' pytest-tests.log