Github Actions | Tests (#10)

* Add uv

* Update lock check

* Fix requirements installation

* Fix workdir in uv-build

* Fix python version and deps version

* Fix tools and uv lock

* Fix bandit

* Fix pyprojct

* Fix codestyle tools

* Fix flake8 config

* remove verbosity of bandit

* Dependency Updates

* Mypy fixes

* Build matrix

* Add uv to readme

* Store scenarios to file

* Do not store hash if no workdir

* argparse

* args handling and cli implementation

* Improve cli args handling

* CLI improvements

* Add tests & fix cli

* Tests gh action

* add pytest coverage

* Fix artifact naming

* Add badge in readme

* rabbit suggestion

Author: yuval-qf

.github/actions/run-tests/action.yml

@@ -0,0 +1,27 @@
+name: "Run Tests"
+description: "Runs pytest"
+
+
+inputs:
+  python-version:
+    description: "Python version used for the test run. This is only used for output file naming to avoid conflicts."
+    required: true
+
+runs:
+  using: "composite"
+  steps:
+    - name: "Run tests"
+      shell: bash
+      run: uv run --dev pytest --junitxml=test-results.xml --cov=rogue --cov-report=xml
+
+    - name: "Upload Test Results"
+      uses: actions/upload-artifact@v4
+      with:
+        name: junit-results-${{ inputs.python-version }}
+        path: test-results.xml
+
+    - name: "Upload Coverage Results"
+      uses: actions/upload-artifact@v4
+      with:
+        name: coverage-report-${{ inputs.python-version }}
+        path: coverage.xml

.github/workflows/build.yml

@@ -1,20 +1,15 @@
-name: Build Matrix
+name: Build
 
 on:
   pull_request:
+  push:
+    branches:
+      - main
   workflow_dispatch:
 
 jobs:
   build:
     runs-on: ubuntu-latest
-    strategy:
-      matrix:
-        python-version:
-          - "3.10"
-          - "3.11"
-          - "3.12"
-          - "3.13"
-
     steps:
       - name: Checkout Repository
         uses: actions/checkout@v4
@@ -25,11 +20,7 @@ jobs:
       - name: Setup Python
         uses: actions/setup-python@v5
         with:
-          python-version: ${{ matrix.python-version }}
+          python-version-file: ".python-version"
 
       - name: Run uv build
         uses: ./.github/actions/uv-build
-
-      # TODO:
-      # - name: Run tests
-      #   run: uv run pytest

.github/workflows/test.yml

@@ -0,0 +1,36 @@
+name: Test
+
+on:
+  pull_request:
+  push:
+    branches:
+      - main
+  workflow_dispatch:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version:
+          - "3.10"
+          - "3.11"
+          - "3.12"
+          - "3.13"
+
+    steps:
+      - name: Checkout Repository
+        uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Run tests
+        uses: ./.github/actions/run-tests
+        with:
+          python-version: ${{ matrix.python-version }}

README.md

@@ -1,6 +1,7 @@
 # Rogue - The AI Agent Evaluator
 
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+![Tests](https://github.com/qualifire-dev/rogue-private/actions/workflows/test.yml/badge.svg?branch=main)
 
 Rogue is a powerful, tool designed to evaluate the performance, compliance, and reliability of AI agents. It pits a dynamic `EvaluatorAgent` against your agent using Google's A2A protocol, testing it with a range of scenarios to ensure it behaves exactly as intended.
 

pyproject.toml

@@ -23,5 +23,6 @@ dev = [
     "flake8==7.3.0",
     "mypy==1.16.1",
     "pytest>=8.4.1",
+    "pytest-cov>=6.2.1",
     "pytest-mock>=3.14.1",
 ]