WARNING: Proof-of-Concept / Rapid Prototype
This project is in an early, experimental state. APIs, CLI flags, gRPC contracts, configuration formats, and internal architecture are all unstable and subject to breaking changes without notice. There are no stability guarantees, no migration paths between versions, and no backward-compatibility commitments at this time.
Do not depend on any interface or behavior remaining the same between commits. If you are evaluating this project, expect things to move fast and break often.
Ansible Policy & Modernization Engine — a multi-validator static analysis platform for Ansible content. It parses playbooks, roles, collections, and task files into a structured hierarchy, then fans validation out in parallel across four independent backends (OPA/Rego, native Python, Ansible-runtime, and Gitleaks) to produce a single, unified list of violations.
APME is a static and semi-static analysis tool for Ansible content. It reads your YAML, reasons about structure and module usage, and reports what it finds — without running tasks or executing against target hosts.
It answers questions like:
- Will this playbook parse on ansible-core 2.19?
- Are any modules I use removed, deprecated, or redirected?
- Do my module arguments match the argspec for the version I'm targeting?
- Does my code follow organizational style and security policies?
- Are there hardcoded credentials in my project?
- What migration work is required to move from one ansible-core version to another?
APME is not a test framework, a deployment tool, or a runtime verification system.
It cannot tell you whether a playbook will achieve its desired outcome on your infrastructure. A playbook's intended outcome — packages installed, services configured, files in the right state — depends on target host state, inventory variables, network reachability, external APIs, and runtime facts that only exist during execution. No static analysis tool can evaluate those.
APME also cannot guarantee a playbook will run successfully, even if it reports zero violations. A clean APME scan means no known incompatibilities were detected — not that every runtime path will succeed.
APME provides a compatibility and quality floor. It catches the preventable mistakes — the removed module, the broken include:, the wrong argument name, the committed secret — before they reach staging or production.
| When discovered | Cost |
|---|---|
| APME scan in CI | Developer fixes it in their branch |
| Syntax check in staging | Deployment blocked, team context-switches |
| Production run fails | Outage, incident response, postmortem |
For organizations managing hundreds of roles and collections across ansible-core version upgrades, this shift-left is the difference between a planned migration and an emergency one.
What still requires execution-time tools:
| Concern | Tool |
|---|---|
| Does the playbook produce the correct end state? | Molecule, integration tests |
| Is the playbook idempotent (safe to run twice)? | --check mode, Molecule converge+idempotence |
| Do templates render correctly with real variables? | Integration tests against test inventory |
| Does the playbook handle failure paths gracefully? | Molecule verify, side-effect testing |
| Does it work with my specific inventory and vault? | Staging environment dry-run |
APME and these tools are complementary. APME runs in seconds without infrastructure and catches structural and compatibility issues early. Execution-time tools validate behavior and correctness against real systems. Use both.
┌─────────┐ gRPC ┌────────────┐ gRPC (parallel) ┌────────────┐
│ CLI │ ──────────────► │ Primary │ ──────────────────────► │ Native │ :50055
│ (on-the │ ScanRequest │ (orchestr) │ ValidateRequest │ (Python) │
│ -fly) │ chunked fs │ │ ┌─────────────────────► ├────────────┤
└─────────┘ │ Engine │ │ │ OPA │ :50054
▲ │ ┌──────┐ │ │ ┌──────────────────► │ (Rego) │
│ ScanResponse │ │parse │ │ │ │ ├────────────┤
│ violations │ │annot.│ │ │ │ ┌───────────────► │ Ansible │ :50053
└──────────────────────│ │hier. │ ├─┘ │ │ │ (runtime) │
│ └──────┘ ├────┘ │ ├────────────┤
└────────────┘───────┘ │ Gitleaks │ :50056
│ │ (secrets) │
┌────┴────┐ └────────────┘
│ Galaxy │ :8765
│ Proxy │ (PEP 503)
└─────────┘
Six app containers, one pod. All inter-service communication is gRPC. The Galaxy Proxy serves Ansible collections as Python wheels (PEP 503). The CLI is run on-the-fly with the project directory mounted. See docs/architecture/ for the full design.
- Single parse, multiple validators — the engine parses Ansible content once and produces a hierarchy payload + scandata; validators consume it independently.
- Parallel fan-out — Primary calls Native, OPA, Ansible, and Gitleaks validators concurrently via
asyncio.gather(); total latency = max(validators), not sum. - Unified gRPC contract — every validator implements the same
Validatorservice (validate.proto); adding a new validator means implementing one RPC. - 100+ rules across four backends: OPA Rego (L003–L025, L061–L072, M006/M008/M009/M011, R118), native Python (L026–L105, M005/M010, R101–R501), Ansible runtime (L057–L059, M001–M004), Gitleaks (SEC:* — 800+ secret patterns).
- Secret scanning — Gitleaks binary wrapped in gRPC; scans all project files for hardcoded credentials, API keys, private keys. Vault-encrypted files and Jinja2 expressions are automatically filtered.
- Multi ansible-core version support — the Primary orchestrator builds session-scoped venvs per ansible-core version (UV-cached); argspec and deprecation checks run against the requested version. Venvs are shared read-only with validators via a
/sessionsvolume. - Structured diagnostics — every validator reports per-rule timing data via the gRPC contract; use
-vfor summaries or-vvfor full per-rule breakdowns. - Galaxy Proxy — converts Ansible Galaxy collection tarballs into pip-installable Python wheels (PEP 503/427); collections are
uv pip install'd into session venvs, leveraging standard Python caching and dependency resolution. - YAML formatter — normalize indentation, key ordering, Jinja spacing, and tab removal with comment preservation. Idempotent by design; runs as a pre-pass before semantic fixes.
- Colocated tests — every rule has a
*_test.py(native),*_test.rego(OPA), or.mddoc with violation/pass examples usable as integration tests.
# Install tox (one-time)
uv tool install tox --with tox-uv
# Run a check (user-facing); engine runs the internal scan pipeline
apme check /path/to/playbook-or-project
# JSON output
apme check --json .
# Diagnostics: summary + top 10 slowest rules
apme check -v .
# Diagnostics: full per-rule breakdown
apme check -vv .
# Format YAML files (show diff)
apme format /path/to/project
# Format and apply changes in place
apme format --apply /path/to/project
# CI check mode (exit 1 if changes needed)
apme format --check /path/to/project
# Full remediate pipeline: format → idempotency check → re-scan → modernize
apme remediate /path/to/playbook-or-project
# AI-assisted remediation (requires Abbenay daemon)
apme remediate --ai /path/to/playbook-or-project
# AI with auto-approve (no interactive review)
apme remediate --ai --auto-approve /path/to/playbook-or-projectThe full pod runs 9 containers: engine services (Primary, Native, OPA, Ansible,
Gitleaks, Galaxy Proxy), Gateway (REST API + persistence), UI (React dashboard),
and Abbenay (AI provider). Pod management scripts (build.sh, up.sh,
down.sh, wait-for-pod.sh) run from the repo root. The CLI helper
run-cli.sh runs from the directory you want to scan.
Container Port Role
─────────────────────────────────────────────────────
primary 50051 Engine orchestrator (gRPC)
native 50055 Python rule validator (gRPC)
opa 50054 Rego rule validator (gRPC)
ansible 50053 Ansible runtime validator (gRPC)
gitleaks 50056 Secret scanner (gRPC)
galaxy-proxy 8765 Collection → wheel proxy (PEP 503)
gateway 8080/50060 REST API + WebSocket + persistence
ui 8081 React dashboard (nginx)
abbenay 50057 AI provider (gRPC)
tox -e up # build all images and start the pod
tox -e up -- --no-cache # rebuild from scratch
tox -e up-clean # wipe state + rebuild + start (clean slate)This builds a shared base image, nine service images, pulls the Abbenay AI
image, then starts all containers. Cache defaults to
${XDG_CACHE_HOME:-$HOME/.cache}/apme (override with APME_CACHE_HOST_PATH).
APME Pod Architecture
┌─────────────────────────────────────────────────────────────────────────────────┐
│ apme-pod │
│ │
│ ┌─────────────────────────────────────────────────────────────────────────┐ │
│ │ User Entry Points │ │
│ │ │ │
│ │ Browser ──► http://localhost:8081 ──► UI (React/nginx) │ │
│ │ │ │ │
│ │ Terminal ──► tox -e cli ────────────────┐ │ REST/WebSocket │ │
│ │ │ │ ▼ │ │
│ │ │ ┌──────────────┐ │ │
│ │ │ │ Gateway │ ◄── SQLite DB │ │
│ │ │ │ :8080/:50060│ /data/apme.db │ │
│ │ │ └──────┬───────┘ │ │
│ │ │ │ gRPC │ │
│ └──────────────────────┼─────────────────────┼────────────────────────────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────────────────────────────────────────────────────────────────┐ │
│ │ Engine Layer │ │
│ │ │ │
│ │ ┌────────────────────────────────────────────────────────────────┐ │ │
│ │ │ Primary :50051 │ │ │
│ │ │ • Orchestrator: parse → annotate → fan-out → aggregate │ │ │
│ │ │ • Session venv management (ansible-core versions) │ │ │
│ │ └────────────────────────────┬───────────────────────────────────┘ │ │
│ │ │ │ │
│ │ gRPC parallel fan-out (asyncio.gather) │ │
│ │ ┌────────┬──────┴──────┬─────────┐ │ │
│ │ ▼ ▼ ▼ ▼ │ │
│ │ ┌─────────┐ ┌─────────┐ ┌──────────┐ ┌─────────────┐ │ │
│ │ │ Native │ │ OPA │ │ Ansible │ │ Gitleaks │ │ │
│ │ │ :50055 │ │ :50054 │ │ :50053 │ │ :50056 │ │ │
│ │ │ │ │ │ │ │ │ │ │ │
│ │ │ Python │ │ Rego │ │ Runtime │ │ Secrets │ │ │
│ │ │ rules │ │ rules │ │ checks │ │ scan │ │ │
│ │ │ L026+ │ │ L003+ │ │ L057-59 │ │ SEC:* │ │ │
│ │ │ R101+ │ │ R118 │ │ M001-04 │ │ 800+ rules │ │ │
│ │ └─────────┘ └─────────┘ └──────────┘ └─────────────┘ │ │
│ │ │ │
│ └──────────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌──────────────────────────────────────────────────────────────────────────┐ │
│ │ Supporting Services │ │
│ │ │ │
│ │ ┌────────────────────┐ ┌─────────────────────┐ │ │
│ │ │ Galaxy Proxy │ │ Abbenay │ │ │
│ │ │ :8765 │ │ :50057 │ │ │
│ │ │ │ │ │ │ │
│ │ │ Collections → PEP │ │ AI Provider for │ │ │
│ │ │ 503 Python wheels │ │ Tier 2 remediation│ │ │
│ │ └────────────────────┘ └─────────────────────┘ │ │
│ │ │ │
│ └──────────────────────────────────────────────────────────────────────────┘ │
│ │
│ Shared Volumes: │
│ • /sessions ─ Session venvs (Primary ↔ Ansible validator) │
│ • /cache ─ UV cache + Galaxy collections (~/.cache/apme on host) │
│ • /data ─ Gateway SQLite database │
│ │
└─────────────────────────────────────────────────────────────────────────────────┘
CLI Container (ephemeral, joins pod network on each invocation)
┌─────────────────────────────────────────────────────────────────────────────────┐
│ tox -e cli -- [command] │
│ • Mounts $(pwd) → /workspace │
│ • Connects to Primary at 127.0.0.1:50051 │
│ • Commands: check, remediate, format, health-check │
└─────────────────────────────────────────────────────────────────────────────────┘
Data flow:
- Check/Scan: CLI or UI → Gateway → Primary → validators (parallel) → aggregated violations
- Remediate: Check + Tier 1 transforms in Primary + Tier 2 AI proposals via Abbenay
- Collections: Primary/Ansible → Galaxy Proxy → galaxy.ansible.com → wheels
Once the pod is running, open http://localhost:8081 in your browser. The UI proxies API calls to the Gateway on port 8080. No authentication is required for local development.
The dashboard provides project management, live scan/remediate operations with real-time progress, interactive AI proposal review, dependency tracking, and cross-project analytics. See .sdlc/research/ui-capabilities-assessment.md for a full capabilities inventory.
The CLI container is not part of the pod — it joins the pod network
on each invocation with your current directory mounted at /workspace.
tox -e cli # default: check .
tox -e cli -- check --json . # JSON output
tox -e cli -- remediate . # Tier 1 deterministic fixes
tox -e cli -- remediate --ai . # include AI proposals (Tier 2)
tox -e cli -- format --check . # YAML format dry-run
tox -e cli -- health-check # health check all servicesTo enable AI-assisted remediation, create containers/abbenay/.env from
the example and add your API key:
cp containers/abbenay/.env.example containers/abbenay/.env
# Edit .env: set OPENROUTER_API_KEY=your-keyThe up.sh script sources this file automatically. The Abbenay container
starts on port 50057 and Primary connects to it for Tier 2 AI proposals.
tox -e down # stop and remove pod
tox -e wipe # also delete database + session cacheapme health-check # local daemon
tox -e cli -- health-check # from podAPME can escalate Tier 2 violations (no deterministic transform) to an AI provider for proposed fixes. This requires the Abbenay daemon.
pip install apme-engine[ai]
# Consumer auth token (required for inline policy)
export APME_ABBENAY_TOKEN="your-token"# Start Abbenay daemon (auto-discovers socket at $XDG_RUNTIME_DIR/abbenay/)
abbenay daemon start
# Or from a Sea binary:
./abbenay-daemon-linux-x64 start
# Set consumer auth token for inline policy (required)
export APME_ABBENAY_TOKEN="your-token"
# Remediate with AI
apme remediate --ai /path/to/playbook-or-projectSee the Abbenay container documentation for full container setup instructions.
# Pull the pre-built multi-arch image (amd64 + arm64)
podman pull ghcr.io/redhat-developer/abbenay:latest
# Standalone: Abbenay defaults to gRPC on :50051.
# When running inside the APME pod, use -p 50057:50051 to avoid
# colliding with APME Primary (also :50051).
podman run -d --name abbenay \
-v ./config.yaml:/home/abbenay/.config/abbenay/config.yaml:ro \
-e OPENROUTER_API_KEY="$OPENROUTER_API_KEY" \
-p 8787:8787 -p 50057:50051 \
ghcr.io/redhat-developer/abbenay:latest
# Point APME at the Abbenay container via gRPC TCP
APME_ABBENAY_ADDR=localhost:50057 apme remediate --ai .| Flag | Description |
|---|---|
--ai |
Enable AI escalation (opt-in) |
--auto-approve |
Approve all AI proposals without prompting (CI mode) |
--max-passes N |
Max convergence passes (default: 5) |
--json |
Output structured data payloads as JSON |
--session ID |
Explicit session ID for venv reuse (default: auto-derived from project root) |
- Tier 1 (deterministic): convergence loop applies transforms until stable
- Tier 2 (AI): remaining violations are sent to the AI provider one at a time; each proposal is re-validated, cleaned with Tier 1 transforms, and retried with feedback if needed
- Interactive review: accepted proposals are applied (or previewed with
check --diff) - Tier 3 (manual): violations that neither transforms nor AI can fix are reported for human review
Scale pods, not services within a pod. Each pod is a self-contained stack that can process check and remediate workloads end-to-end. For more throughput, run multiple pods behind a load balancer. See docs/architecture/17-scaling-and-deployment.md.
# Install tox (one-time setup)
uv tool install tox --with tox-uv
# Unit tests with coverage
tox -e unit
# Integration tests (requires OPA binary)
tox -e integration
# All default environments (lint + unit + integration + ai + ui)
toxSee docs/guides/DEVELOPMENT.md for the full tox environment reference.
proto/apme/v1/ gRPC service definitions (.proto)
src/apme/v1/ generated Python gRPC stubs
src/apme_engine/
├── engine/ project loader (parse, annotate, hierarchy)
│ └── annotators/ per-module risk annotators
├── validators/
│ ├── base.py Validator protocol + ScanContext
│ ├── native/ Python rules (L026–L105, M005/M010, R101–R501)
│ ├── opa/ Rego bundle (L003–L025, L061–L072, M006/M008/M009/M011, R118)
│ ├── ansible/ Ansible-runtime rules (L057–L059, M001–M004)
│ └── gitleaks/ Gitleaks wrapper (SEC:* — secret detection)
├── daemon/ gRPC server implementations
├── venv_manager/ Session-scoped venv lifecycle (VenvSessionManager)
├── remediation/ Tier 1 transforms + AI escalation
├── formatter.py YAML formatter (phase 1 remediation)
├── cli/ CLI entry point (check, format, remediate, health-check)
└── runner.py scan orchestration (engine-internal pipeline)
src/apme_gateway/ API gateway (FastAPI, REST/WebSocket, SQLite)
src/galaxy_proxy/ Galaxy → PEP 503 wheel proxy
frontend/ React operator UI (Vite + TypeScript)
containers/ Containerfiles + Podman pod config
docs/ architecture, design, rule mapping
tests/ unit, integration, rule doc coverage
| Document | Description |
|---|---|
| Architecture series | Pipeline walkthrough, container topology, gRPC contracts, data flow, scaling model |
| Design docs | Remediation engine, AI escalation, validator abstraction — design rationale |
| Guides | Development setup, deployment, troubleshooting |
| Rule reference | Rule catalog, ID mapping, doc format, ansible-lint coverage |
| ADRs | Architecture Decision Records — key design decisions with context, alternatives, and rationale |
format subcommand with --diff/--apply/--check modes, idempotency guarantees, gRPC Format RPC.
remediatesubcommand: format → idempotency gate → re-scan → semantic transforms.is_finding_resolvable()partition: each rule declares afixableattribute; the remediate pipeline splits findings into auto-fixable vs manual/AI.- Multi-pass convergence loop: remediate applies transforms and re-runs the internal scan pipeline until stable or oscillation detected (max N passes).
module_metadata.json: machine-readable module lifecycle data (introduced, deprecated, removed, parameter renames) generated fromansible-docacross core versions. M-series rules become data-driven lookups instead of per-rule hardcoded logic.
- Secret scanning (done) — Gitleaks validator: 800+ patterns for credentials, API keys, private keys via dedicated container + gRPC wrapper. Vault and Jinja filtering built in.
- EE compatibility rules (R505–R507): undeclared collections, system path assumptions, undeclared Python deps. Requires static
ee_baseline.jsonfromee-supported-rhel9inspection. - Version auto-detection: infer source Ansible version from playbook signals (short-form module names → ≤2.9,
include:→ ≤2.7,tower_*→ ≤2.13). Auto-scope M-rules without an explicit--ansible-core-versionflag.
- Abbenay daemon as the AI backend via gRPC:
pip install apme-engine[ai]. - AIProvider Protocol (
ADR-024): pluggable abstraction for LLM providers;AbbenayProvideris the default. - Hybrid validation loop: AI proposals are re-scanned through APME validators, cleaned up with Tier 1 transforms, and retried with feedback if issues persist (max 2 attempts).
- Interactive review (
--aiflag): per-fix diff review (y/n/skip) likegit add -p, or--auto-approvefor automatic application. - Structured best practices: curated Ansible guidelines injected into prompts for higher-quality fixes.
- Preflight checks: auto-discover Abbenay daemon socket, health check before AI calls.
- See DESIGN_AI_ESCALATION.md for the full design.
Operator UI for check and remediate sessions, Activity (history), health monitoring, and findings management. API gateway (FastAPI), REST/WebSocket API (/api/v1/activity, etc.), persistence (SQLite), and React frontend. Operation streaming uses FixSession gRPC (ADR-039). See docs/architecture/13-gateway-and-persistence.md and 14-ui-integration.md.
Apache-2.0