Add database smoke checks

marcofortina · marcofortina · commit f0e260b948c6 · 2026-06-01T18:47:25.000+02:00
diff --git a/README.md b/README.md
@@ -309,7 +309,7 @@ devices:       device.udev.*
 notifications: notify.mail.send
 ```
 
-See `docs/plugins/` for detailed examples of every builtin plugin. Use `scripts/release-check.sh` before tagging a release to verify generated plugin docs, runbook indexes, policy/risk preflight, plugin coverage gates, tests, documentation and packages together.
+See `docs/plugins/` for detailed examples of every builtin plugin. Use `scripts/release-check.sh` before tagging a release to verify generated plugin docs, runbook indexes, policy/risk preflight, SQLite database smoke, plugin coverage gates, tests, documentation and packages together.
 
 ## Variables and secrets
 
diff --git a/docs/guides/database-smoke.md b/docs/guides/database-smoke.md
@@ -0,0 +1,73 @@
+<!--
+Copyright (C) 2026 Marco Fortina
+SPDX-License-Identifier: AGPL-3.0-or-later
+-->
+
+# Database Smoke Checks
+
+`scripts/database-smoke.sh` runs controller-side database health checks through the Automax database check plugins.
+
+SQLite is always executed because it requires no external service. PostgreSQL, MySQL/MariaDB and Oracle are optional and run only when their environment variables are provided.
+
+## Commands
+
+```bash
+scripts/database-smoke.sh --list
+scripts/database-smoke.sh --engine sqlite
+scripts/database-smoke.sh --engine all
+```
+
+The default output directory is:
+
+```text
+dist/database-smoke/
+```
+
+Each run writes:
+
+```text
+summary.txt
+summary.json
+sqlite.log
+postgres.log
+mysql.log
+oracle.log
+```
+
+Only the selected engines create logs.
+
+## Environment variables
+
+PostgreSQL:
+
+```bash
+export AUTOMAX_POSTGRES_DSN='postgresql://automax:automax@127.0.0.1:5432/automax'
+```
+
+MySQL/MariaDB:
+
+```bash
+export AUTOMAX_MYSQL_HOST=127.0.0.1
+export AUTOMAX_MYSQL_PORT=3306
+export AUTOMAX_MYSQL_USER=automax
+export AUTOMAX_MYSQL_PASSWORD=automax
+export AUTOMAX_MYSQL_DATABASE=automax
+```
+
+Oracle:
+
+```bash
+export AUTOMAX_ORACLE_DSN='db01.example.com/FREEPDB1'
+export AUTOMAX_ORACLE_USER=automax
+export AUTOMAX_ORACLE_PASSWORD=automax
+```
+
+Optional engines require their corresponding Python extras and database client setup.
+
+## Runtime validation packs
+
+Runtime validation packs include the `database-smoke` step. Without optional database environment variables, the script records PostgreSQL, MySQL/MariaDB and Oracle as skipped while SQLite still runs.
+
+## Release check
+
+`scripts/release-check.sh` runs SQLite database smoke as a release guardrail because it is local and deterministic.
diff --git a/docs/guides/runtime-validation-packs.md b/docs/guides/runtime-validation-packs.md
@@ -81,3 +81,7 @@ AUTOMAX_RUNTIME_EVIDENCE_RETAIN=10 scripts/runtime-validation-packs.sh --pack pa
 
 Use `scripts/runtime-evidence-index.sh --verify` to rebuild the index and verify
 all discovered checksums without running a new pack.
+
+## Database smoke step
+
+The `database-smoke` step runs `scripts/database-smoke.sh`. SQLite is local and deterministic; PostgreSQL, MySQL/MariaDB and Oracle run only when their environment variables are configured.
diff --git a/docs/plugins/database.md b/docs/plugins/database.md
@@ -130,8 +130,20 @@ register the canonical plugin in `build_builtin_registry()` or load it through a
 external plugin path. The shared base already handles statement validation,
 output shaping and common result metadata.
 
+## Database smoke script
+
+Use the controller-side smoke script for release or lab evidence:
+
+```bash
+scripts/database-smoke.sh --engine sqlite
+scripts/database-smoke.sh --engine all
+```
+
+SQLite runs locally. PostgreSQL, MySQL/MariaDB and Oracle run when their environment variables are provided.
+
 ## Database health checks
 
+
 `database.sqlite.check` is a read-only controller-side health check plugin for the existing
 SQLite, PostgreSQL, MySQL/MariaDB and Oracle database families. It is intended
 for precheck and postcheck gates, not for application queries.
diff --git a/docs/reference/release.md b/docs/reference/release.md
@@ -37,6 +37,7 @@ The script verifies:
 - generated plugin reference freshness;
 - plugin smoke runbook index freshness;
 - policy/risk preflight for committed examples;
+- SQLite database smoke through `scripts/database-smoke.sh`;
 - Python source compilation;
 - Python 3.9 compatibility guardrails;
 - Ruff checks;
@@ -80,6 +81,7 @@ Keep the following with the release record:
 - JSONL run event stream exported with `automax runs events`;
 - runtime validation pack output from `scripts/runtime-validation-packs.sh`;
 - policy/risk preflight output from `scripts/policy-risk-preflight.sh`;
+- database smoke summaries when database behavior changed;
 - approval gate files for release operations that require sign-off;
 - `automax plugins coverage --strict` output for builtin and release-loaded
   plugins;
diff --git a/examples/runtime-packs/debian-like.env b/examples/runtime-packs/debian-like.env
@@ -2,5 +2,5 @@
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 AUTOMAX_RUNTIME_PACK_DESCRIPTION="Debian-like disposable VM validation pack"
-AUTOMAX_RUNTIME_PACK_STEPS="release-metadata package-smoke plugin-checks ssh-evidence"
+AUTOMAX_RUNTIME_PACK_STEPS="release-metadata package-smoke database-smoke plugin-checks ssh-evidence"
 AUTOMAX_RUNTIME_OS_FAMILY="debian-like"
diff --git a/examples/runtime-packs/package-only.env b/examples/runtime-packs/package-only.env
@@ -2,4 +2,4 @@
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 AUTOMAX_RUNTIME_PACK_DESCRIPTION="Release metadata and package smoke validation"
-AUTOMAX_RUNTIME_PACK_STEPS="release-metadata package-smoke"
+AUTOMAX_RUNTIME_PACK_STEPS="release-metadata package-smoke database-smoke"
diff --git a/examples/runtime-packs/privileged-vm.env b/examples/runtime-packs/privileged-vm.env
@@ -2,5 +2,5 @@
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 AUTOMAX_RUNTIME_PACK_DESCRIPTION="Privileged disposable VM validation pack"
-AUTOMAX_RUNTIME_PACK_STEPS="release-metadata package-smoke plugin-checks ssh-evidence"
+AUTOMAX_RUNTIME_PACK_STEPS="release-metadata package-smoke database-smoke plugin-checks ssh-evidence"
 AUTOMAX_RUNTIME_PRIVILEGED="1"
diff --git a/examples/runtime-packs/redhat-like.env b/examples/runtime-packs/redhat-like.env
@@ -2,5 +2,5 @@
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 AUTOMAX_RUNTIME_PACK_DESCRIPTION="Red Hat-like disposable VM validation pack"
-AUTOMAX_RUNTIME_PACK_STEPS="release-metadata package-smoke plugin-checks ssh-evidence"
+AUTOMAX_RUNTIME_PACK_STEPS="release-metadata package-smoke database-smoke plugin-checks ssh-evidence"
 AUTOMAX_RUNTIME_OS_FAMILY="redhat-like"
diff --git a/mkdocs.yml b/mkdocs.yml
@@ -70,6 +70,7 @@ nav:
     - Extended SSH Smoke: guides/ssh-smoke.md
     - Runtime Evidence Suite: guides/runtime-evidence.md
     - Runtime Validation Packs: guides/runtime-validation-packs.md
+    - Database Smoke: guides/database-smoke.md
     - Policy and Risk Preflight: guides/policy-risk-preflight.md
     - Approval Gates: guides/approval-gates.md
     - Runtime Evidence Index: guides/runtime-evidence-index.md
diff --git a/scripts/database-smoke.sh b/scripts/database-smoke.sh
@@ -0,0 +1,207 @@
+#!/usr/bin/env bash
+# Copyright (C) 2026 Marco Fortina
+# SPDX-License-Identifier: AGPL-3.0-or-later
+
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+ENGINE="${AUTOMAX_DATABASE_SMOKE_ENGINE:-all}"
+OUTPUT_DIR="${AUTOMAX_DATABASE_SMOKE_OUTPUT_DIR:-${ROOT}/dist/database-smoke}"
+LIST_ONLY=0
+
+usage() {
+  cat <<'USAGE'
+Usage: scripts/database-smoke.sh [--engine all|sqlite|postgres|mysql|oracle] [--output-dir DIR] [--list]
+
+Run controller-side database health smoke checks. SQLite always runs without an
+external service. PostgreSQL, MySQL/MariaDB and Oracle run only when their
+environment variables are provided.
+
+Options:
+  --engine NAME     Engine to run, defaults to all.
+  --output-dir DIR  Output directory, defaults to dist/database-smoke.
+  --list            List supported engines and exit.
+USAGE
+}
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --engine)
+      ENGINE="${2:-}"
+      [[ -n "$ENGINE" ]] || { echo "--engine requires a value" >&2; exit 2; }
+      shift 2
+      ;;
+    --output-dir)
+      OUTPUT_DIR="${2:-}"
+      [[ -n "$OUTPUT_DIR" ]] || { echo "--output-dir requires a path" >&2; exit 2; }
+      shift 2
+      ;;
+    --list)
+      LIST_ONLY=1
+      shift
+      ;;
+    -h|--help)
+      usage
+      exit 0
+      ;;
+    *)
+      echo "Unknown option: $1" >&2
+      usage >&2
+      exit 2
+      ;;
+  esac
+done
+
+if [[ "$LIST_ONLY" -eq 1 ]]; then
+  printf '%s\n' sqlite postgres mysql oracle
+  exit 0
+fi
+
+mkdir -p "$OUTPUT_DIR"
+
+PYTHONPATH="${ROOT}/src${PYTHONPATH:+:${PYTHONPATH}}" python - "$ENGINE" "$OUTPUT_DIR" <<'PY'
+from __future__ import annotations
+
+from datetime import datetime, timezone
+from pathlib import Path
+import json
+import os
+import sqlite3
+import sys
+
+from automax.core.models import ExecutionContext, Target
+from automax.plugins.base import PluginValidationError
+from automax.plugins.registry import build_builtin_registry
+
+engine_selector = sys.argv[1]
+output_dir = Path(sys.argv[2])
+allowed = {"all", "sqlite", "postgres", "mysql", "oracle"}
+if engine_selector not in allowed:
+    raise SystemExit(f"unsupported database smoke engine: {engine_selector}")
+
+run_id = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
+registry = build_builtin_registry()
+context = ExecutionContext(
+    run_id=f"database-smoke-{run_id}",
+    dry_run=False,
+    job={},
+    task={},
+    step={},
+    substep={},
+    target=Target(name="controller", host="localhost"),
+    vars={},
+    outputs={},
+    secrets={},
+)
+
+def selected(name: str) -> bool:
+    return engine_selector in {"all", name}
+
+def write_log(name: str, text: str) -> str:
+    path = output_dir / f"{name}.log"
+    path.write_text(text, encoding="utf-8")
+    return path.name
+
+def run_plugin(name: str, params: dict[str, object], *, log_name: str | None = None) -> tuple[str, str, int, str]:
+    log_target = log_name or name
+    try:
+        result = registry.get(name).execute(params, context)
+    except PluginValidationError as exc:
+        return ("failed", str(exc), 1, write_log(log_target, f"{exc}\n"))
+    except Exception as exc:  # Defensive: smoke output should still explain failures.
+        return ("failed", str(exc), 1, write_log(log_target, f"{type(exc).__name__}: {exc}\n"))
+    log_text = ""
+    if result.stdout:
+        log_text += result.stdout.rstrip() + "\n"
+    if result.stderr:
+        log_text += result.stderr.rstrip() + "\n"
+    if result.ok and isinstance(result.data, dict) and result.data.get("healthy") is False:
+        return ("failed", str(result.data.get("error") or "database health check failed"), 1, write_log(log_target, log_text or "failed\n"))
+    if result.ok:
+        return ("passed", result.message or "ok", 0, write_log(log_target, log_text or "ok\n"))
+    return ("failed", result.message or "failed", int(result.rc or 1), write_log(log_target, log_text or "failed\n"))
+
+def skipped(name: str, reason: str) -> dict[str, object]:
+    return {"engine": name, "status": "skipped", "rc": 77, "reason": reason, "log": write_log(name, f"SKIP: {reason}\n")}
+
+results: list[dict[str, object]] = []
+
+if selected("sqlite"):
+    sqlite_path = output_dir / "sqlite-smoke.db"
+    with sqlite3.connect(sqlite_path) as conn:
+        conn.execute("CREATE TABLE IF NOT EXISTS automax_smoke (id integer primary key, value text)")
+        conn.execute("INSERT INTO automax_smoke(value) VALUES ('ok')")
+    status, message, rc, log = run_plugin(
+        "database.sqlite.check",
+        {"connection": {"path": str(sqlite_path)}, "checks": ["connect", "select", "version", "integrity"], "output": "json"},
+        log_name="sqlite",
+    )
+    results.append({"engine": "sqlite", "status": status, "rc": rc, "message": message, "log": log})
+
+if selected("postgres"):
+    dsn = os.environ.get("AUTOMAX_POSTGRES_DSN")
+    if not dsn:
+        results.append(skipped("postgres", "set AUTOMAX_POSTGRES_DSN to run PostgreSQL smoke"))
+    else:
+        status, message, rc, log = run_plugin(
+            "database.postgres.check",
+            {"connection": {"dsn": dsn}, "checks": ["connect", "select", "version"], "output": "json"},
+            log_name="postgres",
+        )
+        results.append({"engine": "postgres", "status": status, "rc": rc, "message": message, "log": log})
+
+if selected("mysql"):
+    host = os.environ.get("AUTOMAX_MYSQL_HOST")
+    user = os.environ.get("AUTOMAX_MYSQL_USER")
+    password = os.environ.get("AUTOMAX_MYSQL_PASSWORD")
+    database = os.environ.get("AUTOMAX_MYSQL_DATABASE")
+    if not (host and user):
+        results.append(skipped("mysql", "set AUTOMAX_MYSQL_HOST and AUTOMAX_MYSQL_USER to run MySQL smoke"))
+    else:
+        connection = {"host": host, "user": user}
+        if os.environ.get("AUTOMAX_MYSQL_PORT"):
+            connection["port"] = int(os.environ["AUTOMAX_MYSQL_PORT"])
+        if password is not None:
+            connection["password"] = password
+        if database:
+            connection["database"] = database
+        status, message, rc, log = run_plugin(
+            "database.mysql.check",
+            {"connection": connection, "checks": ["connect", "select", "version"], "output": "json"},
+            log_name="mysql",
+        )
+        results.append({"engine": "mysql", "status": status, "rc": rc, "message": message, "log": log})
+
+if selected("oracle"):
+    dsn = os.environ.get("AUTOMAX_ORACLE_DSN")
+    user = os.environ.get("AUTOMAX_ORACLE_USER")
+    password = os.environ.get("AUTOMAX_ORACLE_PASSWORD")
+    if not (dsn and user and password):
+        results.append(skipped("oracle", "set AUTOMAX_ORACLE_DSN, AUTOMAX_ORACLE_USER and AUTOMAX_ORACLE_PASSWORD to run Oracle smoke"))
+    else:
+        status, message, rc, log = run_plugin(
+            "database.oracle.check",
+            {"connection": {"dsn": dsn, "user": user, "password": password}, "checks": ["connect", "select", "version"], "output": "json"},
+            log_name="oracle",
+        )
+        results.append({"engine": "oracle", "status": status, "rc": rc, "message": message, "log": log})
+
+failed = [row for row in results if row["status"] == "failed"]
+summary = {
+    "schema_version": 1,
+    "generated_at": datetime.now(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z"),
+    "run_id": run_id,
+    "engine": engine_selector,
+    "results": results,
+    "failed": len(failed),
+}
+(output_dir / "summary.json").write_text(json.dumps(summary, indent=2, sort_keys=True) + "\n", encoding="utf-8")
+lines = ["Automax database smoke", f"run_id={run_id}", f"engine={engine_selector}", ""]
+for row in results:
+    lines.append(f"{row['engine']}: {row['status']} rc={row['rc']} log={row['log']}")
+(output_dir / "summary.txt").write_text("\n".join(lines) + "\n", encoding="utf-8")
+
+for line in lines:
+    print(line)
+raise SystemExit(1 if failed else 0)
+PY
diff --git a/scripts/release-check.sh b/scripts/release-check.sh
@@ -97,6 +97,7 @@ check_release_metadata
 check_generated_plugin_reference
 check_runbook_index
 scripts/policy-risk-preflight.sh --path examples --fail-on high >/dev/null
+scripts/database-smoke.sh --engine sqlite --output-dir "${TMP_DIR}/database-smoke" >/dev/null
 
 python -m compileall -q src tests scripts
 python scripts/check-python39-compat.py
diff --git a/scripts/runtime-validation-packs.sh b/scripts/runtime-validation-packs.sh
@@ -115,6 +115,9 @@ run_step() {
     package-smoke)
       (cd "$ROOT" && scripts/package-smoke.sh) >>"$log" 2>&1 || rc=$?
       ;;
+    database-smoke)
+      (cd "$ROOT" && scripts/database-smoke.sh) >>"$log" 2>&1 || rc=$?
+      ;;
     plugin-checks)
       if [[ -z "${!SUDO_PASSWORD_ENV:-}" ]]; then
         echo "SKIP: set ${SUDO_PASSWORD_ENV} for plugin-checks" >>"$log"
diff --git a/tests/test_documentation_and_regressions.py b/tests/test_documentation_and_regressions.py
