Merge branch 'main' into add_providerlist

Author: kpunwatk

CONSTITUTION.md

@@ -79,6 +79,7 @@ All code MUST consider security implications.
 - Avoid running destructive commands without explicit user confirmation
 - Use detect-secrets and gitleaks pre-commit hooks to prevent secret leakage
 - Test code MUST NOT introduce vulnerabilities into the tested systems
+- Use `utilities.path_utils.resolve_repo_path` to resolve and validate any user-supplied or parameterized file paths, preventing path-traversal and symlink-escape outside the repository root
 - JIRA ticket links are allowed in PRs and commit messages (our Jira is public)
 - Do NOT reference internal-only resources (Jenkins, Confluence, Slack threads) in code, PRs, or commit messages
 - Do NOT link embargoed or security-restricted (RH-employee-only) tickets

pyproject.toml

@@ -6,7 +6,7 @@ output-format = "grouped"
 extend-exclude = ["utilities/manifests"]
 
 [tool.ruff.lint]
-external = ["E501"]
+external = ["E501", "FCN001"]
 
 [tool.ruff.format]
 exclude = [".git", ".venv", ".mypy_cache", ".tox", "__pycache__", "utilities/manifests"]

pytest.ini

@@ -4,8 +4,6 @@ testpaths = tests
 
 markers =
     # General
-    polarion: Store polarion test ID
-
     skip_on_disconnected: Mark tests that can only be run in deployments with Internet access i.e. not on disconnected clusters.
     parallel: marks tests that can run in parallel along with pytest-xdist
 

tests/cluster_health/README.md

@@ -0,0 +1,56 @@
+# Cluster Health Tests
+
+This directory contains foundational health check tests for OpenDataHub/RHOAI clusters. These tests serve as prerequisites to ensure the cluster and operators are in a healthy state before running more complex integration tests.
+
+## Directory Structure
+
+```text
+cluster_health/
+├── test_cluster_health.py      # Cluster node health validation
+└── test_operator_health.py     # Operator and pod health validation
+```
+
+### Current Test Suites
+
+- **`test_cluster_health.py`** - Validates that all cluster nodes are healthy and schedulable
+- **`test_operator_health.py`** - Validates that DSCInitialization, DataScienceCluster resources are ready, and all pods in operator/application namespaces are running
+
+## Test Markers
+
+Tests use the following markers defined in `pytest.ini`:
+
+- `@pytest.mark.cluster_health` - Tests that verify the cluster is healthy to begin testing
+- `@pytest.mark.operator_health` - Tests that verify OpenDataHub/RHOAI operators are healthy and functioning correctly
+
+## Test Details
+
+### Cluster Node Health (`test_cluster_health.py`)
+
+- **`test_cluster_node_healthy`** - Asserts all cluster nodes have `KubeletReady: True` condition and are schedulable (not cordoned)
+
+### Operator Health (`test_operator_health.py`)
+
+- **`test_data_science_cluster_initialization_healthy`** - Validates the DSCInitialization resource reaches `READY` status (120s timeout)
+- **`test_data_science_cluster_healthy`** - Validates the DataScienceCluster resource reaches `READY` status (120s timeout)
+- **`test_pods_cluster_healthy`** - Validates all pods in operator and application namespaces reach Running/Completed state (180s timeout). Parametrized across `operator_namespace` and `applications_namespace` from global config
+
+## Running Tests
+
+### Run All Cluster Health Tests
+
+```bash
+uv run pytest tests/cluster_health/
+```
+
+### Run by Marker
+
+```bash
+# Run cluster node health tests
+uv run pytest -m cluster_health
+
+# Run operator health tests
+uv run pytest -m operator_health
+
+# Run both
+uv run pytest -m "cluster_health or operator_health"
+```

tests/fixtures/README.md

@@ -0,0 +1,74 @@
+# Shared Test Fixtures
+
+This directory contains shared pytest fixtures that are used across multiple test modules. These fixtures are automatically loaded via pytest's plugin mechanism, registered in `/tests/conftest.py`.
+
+## Directory Structure
+
+```text
+fixtures/
+├── files.py           # File storage provider fixtures
+├── guardrails.py      # Guardrails orchestrator infrastructure fixtures
+├── inference.py       # Inference service and serving runtime fixtures
+├── trustyai.py        # TrustyAI operator and DSC configuration fixtures
+└── vector_io.py       # Vector database provider deployment fixtures
+```
+
+### Fixture Modules
+
+- **`files.py`** - Factory fixture for configuring file storage providers (local, S3/MinIO)
+- **`guardrails.py`** - Fixtures for deploying and configuring the Guardrails Orchestrator, including pods, routes, health checks, and gateway configuration
+- **`inference.py`** - Fixtures for vLLM CPU serving runtimes, InferenceServices (Qwen), LLM-d inference simulator, and KServe controller configuration
+- **`trustyai.py`** - Fixtures for TrustyAI operator deployment and DataScienceCluster LMEval configuration
+- **`vector_io.py`** - Factory fixture for deploying vector database providers (Milvus, Faiss, PGVector, Qdrant) with their backing services and configuration
+
+## Registration
+
+All fixture modules are registered as pytest plugins in `/tests/conftest.py`:
+
+```python
+pytest_plugins = [
+    "tests.fixtures.inference",
+    "tests.fixtures.guardrails",
+    "tests.fixtures.trustyai",
+    "tests.fixtures.vector_io",
+    "tests.fixtures.files",
+]
+```
+
+## Usage
+
+Fixtures are automatically available to all tests. Factory fixtures accept parameters via `pytest.mark.parametrize` with `indirect=True`.
+
+### Vector I/O Provider Example
+
+```python
+@pytest.mark.parametrize(
+    "vector_io_provider_deployment_config_factory",
+    ["milvus", "pgvector", "qdrant-remote"],
+    indirect=True,
+)
+def test_with_vector_db(vector_io_provider_deployment_config_factory):
+    # Fixture deploys the provider and returns env var configuration
+    ...
+```
+
+### Supported Vector I/O Providers
+
+| Provider        | Type   | Description                                 |
+| --------------- | ------ | ------------------------------------------- |
+| `milvus`        | Local  | In-memory Milvus (no external dependencies) |
+| `milvus-remote` | Remote | Milvus standalone with etcd backend         |
+| `faiss`         | Local  | Facebook AI Similarity Search (in-memory)   |
+| `pgvector`      | Local  | PostgreSQL with pgvector extension          |
+| `qdrant-remote` | Remote | Qdrant vector database                      |
+
+### Supported File Providers
+
+| Provider | Description                        |
+| -------- | ---------------------------------- |
+| `local`  | Local filesystem storage (default) |
+| `s3`     | S3/MinIO remote object storage     |
+
+## Adding New Fixtures
+
+When adding shared fixtures, place them in the appropriate module file (or create a new one), and register the new module in `/tests/conftest.py` under `pytest_plugins`. Follow the project's fixture conventions: use noun-based names, narrowest appropriate scope, and context managers for resource lifecycle.

tests/llama_stack/README.md

@@ -88,23 +88,23 @@ LLS_FILES_S3_AUTO_CREATE_BUCKET=true             # Optional
 To run all tests in the `/tests/llama_stack` directory:
 
 ```bash
-pytest tests/llama_stack/
+uv run pytest tests/llama_stack/
 ```
 
 ### Run Tests by Component/Team
 
 To run tests for a specific team (e.g. rag):
 
 ```bash
-pytest -m rag tests/llama_stack/
+uv run pytest -m rag tests/llama_stack/
 ```
 
 ### Run Tests for a llama-stack API
 
 To run tests for a specific API (e.g., vector_io):
 
 ```bash
-pytest tests/llama_stack/vector_io
+uv run pytest tests/llama_stack/vector_io
 ```
 
 ### Run Tests with Additional Markers
@@ -113,10 +113,10 @@ You can combine team markers with other pytest markers:
 
 ```bash
 # Run only smoke tests for rag
-pytest -m "rag and smoke" tests/llama_stack/
+uv run pytest -m "rag and smoke" tests/llama_stack/
 
 # Run all rag tests except the ones requiring a GPU
-pytest -m "rag and not gpu" tests/llama_stack/
+uv run pytest -m "rag and not gpu" tests/llama_stack/
 ```
 
 ## Related Testing Repositories
@@ -145,5 +145,3 @@ For information about the APIs and Providers available in the Red Hat LlamaStack
 ## Additional Resources
 
 - [Llama Stack Documentation](https://llamastack.github.io/docs/)
-- [OpenDataHub Documentation](https://opendatahub.io/docs)
-- [OpenShift AI Documentation](https://docs.redhat.com/en/documentation/red_hat_openshift_ai_self-managed)

tests/llama_stack/conftest.py

@@ -1,6 +1,5 @@
 import os
 from collections.abc import Callable, Generator
-from pathlib import Path
 from typing import Any
 
 import httpx
@@ -810,7 +809,6 @@ def vector_store(
             try:
                 vector_store_upload_doc_sources(
                     doc_sources=doc_sources,
-                    repo_root=Path(request.config.rootdir).resolve(),
                     llama_stack_client=unprivileged_llama_stack_client,
                     vector_store=vector_store,
                     vector_io_provider=vector_io_provider,

tests/llama_stack/utils.py

@@ -20,6 +20,7 @@
     LLS_CORE_POD_FILTER,
 )
 from utilities.exceptions import UnexpectedResourceCountError
+from utilities.path_utils import resolve_repo_path
 from utilities.resources.llama_stack_distribution import LlamaStackDistribution
 
 LOGGER = get_logger(name=__name__)
@@ -279,38 +280,32 @@ def vector_store_create_file_from_path(
 
 
 def vector_store_upload_doc_sources(
-    doc_sources: Any,
-    repo_root: Path,
+    doc_sources: list[str],
     llama_stack_client: LlamaStackClient,
     vector_store: Any,
     vector_io_provider: str,
 ) -> None:
     """Upload parametrized document sources (URLs and repo-local paths) to a vector store.
 
-    Resolves each local path under ``repo_root`` and re-resolves directory entries to avoid
-    symlink escape outside the repository.
+    Resolves each local path via ``resolve_repo_path`` and re-resolves directory entries
+    to avoid symlink escape outside the repository.
 
     Args:
         doc_sources: List of URL or path strings (repo-relative or absolute under repo root).
-        repo_root: Resolved repository root; local paths must resolve under this directory.
         llama_stack_client: Client used for file and vector store APIs.
         vector_store: Target vector store (must expose ``id``).
         vector_io_provider: Provider id for log context only.
 
     Raises:
-        TypeError: If ``doc_sources`` is not a list.
-        ValueError: If a local path resolves outside ``repo_root``.
+        ValueError: If a local path resolves outside the repo root.
         FileNotFoundError: If a file or non-empty directory source is missing.
     """
-    if not isinstance(doc_sources, list):
-        raise TypeError(f"doc_sources must be a list[str], got {type(doc_sources).__name__}")
     LOGGER.info(
         "Uploading doc_sources to vector_store (provider_id=%s, id=%s): %s",
         vector_io_provider,
         vector_store.id,
         doc_sources,
     )
-    repo_root_resolved = repo_root.resolve()
     for source in doc_sources:
         if source.startswith(("http://", "https://")):
             vector_store_create_file_from_url(
@@ -319,25 +314,14 @@ def vector_store_upload_doc_sources(
                 vector_store=vector_store,
             )
             continue
-        raw_path = Path(source)  # noqa: FCN001
-        resolved_source = raw_path.resolve() if raw_path.is_absolute() else (repo_root_resolved / raw_path).resolve()
-        if not resolved_source.is_relative_to(repo_root_resolved):
-            raise ValueError(
-                f"doc_sources path must be under repo root ({repo_root_resolved}): {source!r}",
-            )
-        source_path = resolved_source
+        source_path = resolve_repo_path(source=source)
 
         if source_path.is_dir():
             files = sorted(source_path.iterdir())
             if not files:
                 raise FileNotFoundError(f"No files found in directory: {source_path}")
             for file_path in files:
-                file_path_resolved = file_path.resolve(strict=True)
-                if not file_path_resolved.is_relative_to(repo_root_resolved):
-                    raise ValueError(
-                        f"doc_sources directory entry must resolve under repo root "
-                        f"({repo_root_resolved}): {file_path!r} -> {file_path_resolved!r}",
-                    )
+                file_path_resolved = resolve_repo_path(source=file_path)
                 if not file_path_resolved.is_file():
                     continue
                 vector_store_create_file_from_path(