Add mypy type checking, test fixtures, and build docs

- Add mypy config to pyproject.toml with baseline error suppressions
- Add mypy step to lint CI workflow
- Add make typecheck and make check (lint + typecheck + test) targets
- Create tests/fixtures/ with sample garak config, intents models, report JSONL
- Document dependency extras and lockfile workflow in CONTRIBUTING.md

Made-with: Cursor

Author: saichandrapandraju

.github/workflows/lint.yml

@@ -20,11 +20,14 @@ jobs:
         with:
           python-version: '3.12'
 
-      - name: Install ruff
-        run: pip install ruff
+      - name: Install tools
+        run: pip install ruff mypy
 
       - name: Ruff check
         run: ruff check src/ tests/
 
       - name: Ruff format check
         run: ruff format --check src/ tests/
+
+      - name: Mypy type check
+        run: mypy src/

CONTRIBUTING.md

@@ -19,6 +19,27 @@ pip install -e ".[dev]"
 pre-commit install
 ```
 
+## Dependency Management
+
+Dependencies are declared in `pyproject.toml` with optional extras:
+
+| Extra | Install Command | What You Get |
+|-------|----------------|-------------|
+| (none) | `pip install -e .` | Core provider (Llama Stack remote mode) |
+| `[inline]` | `pip install -e ".[inline]"` | Core + garak for local scans |
+| `[dev]` | `pip install -e ".[dev]"` | Tests + ruff + pre-commit |
+| `[server]` | `pip install -e ".[server]"` | Llama Stack server |
+
+**Lockfile**: `requirements.txt` is a pinned lockfile generated from the
+[RH AI PyPI index](https://console.redhat.com/api/pypi/public-rhai/rhoai/3.4/cpu-ubi9-test/simple/)
+via `uv pip compile`. It's used by downstream production builds for hermetic
+dependency pre-fetching. Regenerate with `make lock`.
+
+**Container image**: The dev `Containerfile` installs from standard PyPI with
+garak from the midstream git branch. Production images use the AIPCC base image
+with the RH AI index. See `Containerfile` for the dev build and
+`Dockerfile.konflux` (downstream) for production.
+
 ## Development Workflow
 
 ### Running Tests

Makefile

@@ -1,4 +1,4 @@
-.PHONY: test coverage lint format build lock install install-dev
+.PHONY: test coverage lint format typecheck check build lock install install-dev
 
 test:
 	pytest tests -v
@@ -12,6 +12,11 @@ lint:
 format:
 	ruff format src/ tests/
 
+typecheck:
+	mypy src/
+
+check: lint typecheck test
+
 build:
 	docker build -f Containerfile -t trustyai-garak:dev .
 

pyproject.toml

@@ -82,3 +82,23 @@ ignore = [
 "tests/*" = ["F401", "F841"]
 "src/*/inline/garak_eval.py" = ["F841"]
 "src/*/remote/garak_remote_eval.py" = ["F841"]
+
+[tool.mypy]
+python_version = "3.12"
+warn_unused_configs = true
+ignore_missing_imports = true
+check_untyped_defs = false
+disable_error_code = [
+    "union-attr",
+    "assignment",
+    "no-any-return",
+    "arg-type",
+    "attr-defined",
+    "valid-type",
+    "no-redef",
+    "call-arg",
+    "var-annotated",
+    "return-value",
+    "index",
+    "override",
+]

tests/fixtures/README.md

@@ -0,0 +1,9 @@
+# Test Fixtures
+
+Sample data files used by the test suite.
+
+| File | Purpose |
+|------|---------|
+| `sample_garak_config.yaml` | Example garak config matching `GarakCommandConfig` schema |
+| `sample_intents_models.json` | Intents model endpoint configurations (single-role and all-roles) |
+| `sample_report.jsonl` | Minimal garak report JSONL output for result parsing tests |

tests/fixtures/sample_garak_config.yaml

@@ -0,0 +1,31 @@
+# Sample garak config used in tests.
+# Mirrors the structure of GarakCommandConfig.
+system:
+  verbose: false
+  parallel_requests: 0
+  parallel_attempts: 1
+
+run:
+  seed: 42
+  eval_threshold: 0.5
+  generations: 5
+  deprefix: true
+  extended_detectors: false
+
+plugins:
+  model_type: openai.OpenAICompatible
+  model_name: target-llm
+
+  probe_spec:
+    - encoding.InjectBase64
+    - promptinject.HijackHateHumans
+
+  extended_detectors: false
+
+  probes: {}
+  detectors: {}
+  buffs: {}
+
+reporting:
+  taxonomy: owasp
+  report_prefix: scan

tests/fixtures/sample_report.jsonl

@@ -0,0 +1,3 @@
+{"entry_type": "config", "model_name": "target-llm", "model_type": "openai.OpenAICompatible"}
+{"entry_type": "attempt", "probe": "encoding.InjectBase64", "detector": "always.Pass", "passed": true, "prompt": "test prompt", "output": "test output", "trigger": null}
+{"entry_type": "eval", "probe": "encoding.InjectBase64", "detector": "always.Pass", "passed": 1, "total": 1, "rate": 1.0}