feat(scan): persist raw per-scanner output for source scans

Extend the raw-output preservation already in place for container
scans to cover source scans. ArgusEngine.run() now accepts
raw_output_dir and copies each scanner's results.json / *.sarif /
stdout.txt under <output_dir>/raw/<scanner>/ alongside the
canonical argus-results.json — the same posture as the container
flow, so forensics and manual triage have the same surface area
regardless of which scan path produced the findings.

The CLI gains a unified --no-keep-raw flag (moved out of the
container-only group) and reporting.keep_raw replaces the
container-scoped containers.keep_raw key. CLI flag wins on
conflict; default remains keep-raw=true.

Author: eFAILution

argus.example.yml

@@ -55,6 +55,15 @@ reporting:
   severity_threshold: high
   output_dir: "./argus-results"
 
+  # Persist each scanner's raw output (results.json, *.sarif,
+  # stdout.txt) under ``<output_dir>/raw/<scanner>/`` alongside the
+  # canonical argus-results.json. Default true — useful for
+  # forensics, audit trails, and manual triage. Set false (or pass
+  # --no-keep-raw on the CLI) to skip; saves a few MB per scan in
+  # tight CI environments. Applies to both source scans (``argus
+  # scan``) and container scans (``argus scan container``).
+  keep_raw: true
+
 # Container lifecycle targets (consumed by ``argus scan container``).
 # Defining anything under this top-level ``containers:`` key activates
 # config-driven container scans — no need to pass --image / --discover
@@ -102,14 +111,10 @@ reporting:
 #   # Override which sub-scanners run; default is trivy + grype.
 #   scanners: [trivy, grype, syft]
 #
-#   # Persist raw per-scanner artifacts (trivy-results.json,
-#   # grype-results.json, syft-sbom.json) under
-#   # ``<output_dir>/raw/<image>/`` alongside the canonical
-#   # argus-results.json. Default is true so the artifacts are
-#   # available for forensics, audit, or manual triage. Set false
-#   # (or pass --no-keep-raw on the CLI) to skip — saves a few MB
-#   # per image when running in tight CI environments.
-#   keep_raw: true
+# Note: raw scanner-output preservation is configured via the
+# unified ``reporting.keep_raw`` knob above — the same flag covers
+# trivy/grype/syft container outputs and source-scan scanner
+# outputs. No separate container-only setting is needed.
 
 # Execution backend configuration
 execution:

argus/cli.py

@@ -590,6 +590,19 @@ def _build_scan_parser(subparsers: argparse._SubParsersAction) -> None:
         help="Disable DB cache volume mounts. Forces scanners to re-download "
              "vulnerability databases on every container run.",
     )
+    scan_parser.add_argument(
+        "--no-keep-raw",
+        action="store_true",
+        dest="no_keep_raw",
+        help="Do not persist raw per-scanner output files alongside the "
+             "canonical argus-results.json. Source scans normally drop "
+             "each scanner's results.json / *.sarif / stdout.txt under "
+             "<output_dir>/raw/<scanner>/; container scans drop "
+             "trivy-results.json / grype-results.json / syft-sbom.json "
+             "under <output_dir>/raw/<image>/. Pass --no-keep-raw to "
+             "skip that step in tight CI environments. The same effect "
+             "is available via 'reporting.keep_raw: false' in argus.yml.",
+    )
 
     # Container-specific flags (used with: argus scan container)
     container_group = scan_parser.add_argument_group(
@@ -616,18 +629,6 @@ def _build_scan_parser(subparsers: argparse._SubParsersAction) -> None:
         default=None,
         help="Sub-scanners for container scanning: trivy,grype,syft (default: trivy,grype)",
     )
-    container_group.add_argument(
-        "--no-keep-raw",
-        action="store_true",
-        dest="no_keep_raw",
-        help="Do not persist raw per-scanner output (trivy-results.json, "
-             "grype-results.json, syft-sbom.json) under "
-             "<output_dir>/raw/<image>/. By default raw artifacts are "
-             "kept alongside the canonical argus-results.json so users "
-             "can drill into individual scanner output for forensics or "
-             "manual triage. Set ``containers.keep_raw: false`` in argus.yml "
-             "for the same effect via config.",
-    )
 
     # ZAP DAST flags (used with: argus scan zap)
     dast_group = scan_parser.add_argument_group(
@@ -1147,6 +1148,15 @@ def _load_container_config(args: argparse.Namespace) -> dict:
             )
         config = dict(containers_section)
 
+        # Pull ``reporting.keep_raw`` from the same file so the
+        # container handler honors the unified config knob — same
+        # default-True semantics as ``_cmd_source_scan``. Stashed
+        # under a synthetic underscore key so it doesn't collide
+        # with any future ``containers:`` field a user might add.
+        reporting_section = file_config.get("reporting", {})
+        if isinstance(reporting_section, dict) and "keep_raw" in reporting_section:
+            config["_reporting_keep_raw"] = bool(reporting_section["keep_raw"])
+
     # CLI overrides — explicit > implicit. --image and --discover both
     # OVERWRITE the corresponding config keys so the user's intent is
     # unambiguous (and so we don't accidentally double-scan an image
@@ -1260,6 +1270,18 @@ def _cmd_source_scan(args: argparse.Namespace) -> int:
 
     log.info("Argus scan starting")
 
+    # Decide whether to persist raw per-scanner outputs alongside the
+    # canonical argus-results.json. Default ON — users running
+    # ``argus scan`` reasonably expect each scanner's raw results
+    # (results.json / *.sarif / stdout.txt) to be available for
+    # forensics or manual triage. Opt out via ``--no-keep-raw``
+    # (CLI) or ``reporting.keep_raw: false`` (argus.yml). CLI flag
+    # wins on conflict, matching the dispatcher's
+    # explicit-over-implicit posture used throughout.
+    keep_raw_config = getattr(config.reporting, "keep_raw", True)
+    keep_raw = bool(keep_raw_config) and not getattr(args, "no_keep_raw", False)
+    raw_output_root = str(Path(output_dir) / "raw") if keep_raw else None
+
     # Build engine and register scanners
     engine = ArgusEngine(config)
 
@@ -1364,6 +1386,7 @@ def _cmd_source_scan(args: argparse.Namespace) -> int:
                             use_default_excludes=not getattr(args, "no_default_excludes", False),
                             sbom_path=str(info.path),
                             sbom_format=info.format,
+                            raw_output_dir=raw_output_root,
                         )
                     except Exception as exc:
                         log.error(
@@ -1404,6 +1427,7 @@ def _cmd_source_scan(args: argparse.Namespace) -> int:
                     allow_local_versions=getattr(args, "allow_local_versions", False),
                     no_cache=getattr(args, "no_cache", False),
                     use_default_excludes=not getattr(args, "no_default_excludes", False),
+                    raw_output_dir=raw_output_root,
                 )
         if args.verbose and getattr(engine, "_last_resolutions", None):
             from argus.core.tool_config import format_resolutions_for_display
@@ -1752,7 +1776,13 @@ def _cmd_container_scan(
     # ``containers.keep_raw: false`` (argus.yml). CLI flag wins on
     # conflict, matching the rest of the dispatcher's
     # explicit-over-implicit posture.
-    keep_raw_config = config.get("keep_raw", True)
+    # ``reporting.keep_raw`` is the unified config home for raw-output
+    # preservation; the legacy ``containers.keep_raw`` is still read
+    # as a fallback so configs from earlier in this PR's lifecycle
+    # don't break. CLI ``--no-keep-raw`` wins over both.
+    keep_raw_config = config.get(
+        "_reporting_keep_raw", config.get("keep_raw", True),
+    )
     keep_raw = bool(keep_raw_config) and not getattr(args, "no_keep_raw", False)
     if keep_raw:
         config["_raw_output_root"] = str(Path(output_dir) / "raw")

argus/core/config.py

@@ -44,6 +44,14 @@ class ReportingConfig:
     formats: list[str] = field(default_factory=lambda: ["terminal"])
     severity_threshold: Optional[Severity] = None
     output_dir: str = "./argus-results"
+    # When True, the engine persists each scanner's raw output files
+    # (results.json / *.sarif / stdout.txt) under
+    # ``<output_dir>/raw/<scanner>/`` alongside the canonical
+    # ``argus-results.json``. Default ON since most users running
+    # ``argus scan`` would expect the artifacts to be available for
+    # forensics or manual triage; opt out via ``--no-keep-raw`` (CLI)
+    # or ``reporting.keep_raw: false`` for tight CI environments.
+    keep_raw: bool = True
 
 
 @dataclass
@@ -208,6 +216,7 @@ def _parse_reporting_config(raw: dict | None) -> ReportingConfig:
         formats=raw.get("formats", ["terminal"]),
         severity_threshold=_parse_severity(raw.get("severity_threshold")),
         output_dir=raw.get("output_dir", "./argus-results"),
+        keep_raw=bool(raw.get("keep_raw", True)),
     )
 
 

argus/core/engine.py

@@ -35,6 +35,7 @@ def __init__(self, config: ArgusConfig):
         self._no_cache: bool = False
         self._sbom_path: str | None = None
         self._sbom_format: str | None = None
+        self._raw_output_root: str | None = None
 
     def register_scanner(self, scanner: Scanner) -> None:
         """Register a scanner instance for use by the engine."""
@@ -59,6 +60,7 @@ def run(
         use_default_excludes: bool = True,
         sbom_path: str | None = None,
         sbom_format: str | None = None,
+        raw_output_dir: str | None = None,
     ) -> ScanSummary:
         """Run scanners and return an aggregated ScanSummary.
 
@@ -79,6 +81,14 @@ def run(
                 attribute is True, auto-enables them regardless of
                 argus.yml, and threads the SBOM path through
                 ``config_dict['sbom_path']``.
+            raw_output_dir: when set, ``_run_in_container`` copies each
+                scanner's raw output files (``results.json``,
+                ``stdout.txt``, ``*.sarif``) into
+                ``<raw_output_dir>/<scanner_name>/`` before the
+                per-scanner tempdir is cleaned up. Mirrors the
+                container-scan flow's ``raw/`` artifact preservation
+                so users can drill into individual scanner output
+                regardless of which scan flow produced it.
         """
         from .exclusions import build_exclusion_set, log_exclusion_set
 
@@ -87,6 +97,7 @@ def run(
         self._use_default_excludes = use_default_excludes
         self._sbom_path = sbom_path
         self._sbom_format = sbom_format
+        self._raw_output_root = raw_output_dir
 
         # Validate sbom_format if provided
         if sbom_format is not None and sbom_format not in SBOM_FORMAT_EXTENSIONS:
@@ -710,6 +721,28 @@ def _run_in_container(
                 result_files = [stdout_file]
                 logger.debug("No output files — captured stdout (%d bytes)", len(proc.stdout))
 
+            # Persist raw scanner output (best-effort) before the
+            # tempdir is wiped. Mirrors the container-scan flow's
+            # ``raw/`` artifact preservation: every scanner gets its
+            # own subdir under ``<raw_output_root>/<scanner.name>/``
+            # so ``argus-results.json`` (the canonical artifact)
+            # lives next to the per-scanner files (results.json,
+            # *.sarif, stdout.txt) for forensics or manual triage.
+            # Errors during copy are non-fatal — the scan succeeded,
+            # the canonical JSON is still emitted upstream.
+            if self._raw_output_root and result_files:
+                try:
+                    target_dir = Path(self._raw_output_root) / scanner.name
+                    target_dir.mkdir(parents=True, exist_ok=True)
+                    for src in result_files:
+                        if src.exists() and src.stat().st_size > 0:
+                            shutil.copy2(src, target_dir / src.name)
+                except OSError as exc:
+                    logger.warning(
+                        "Failed to persist raw output for '%s' under %s: %s",
+                        scanner.name, self._raw_output_root, exc,
+                    )
+
             if result_files:
                 logger.debug(
                     "Output files: %s",

argus/tests/test_engine.py

@@ -805,6 +805,127 @@ def mock_run(cmd, **_kwargs):
         result = engine._run_in_container(scanner, "/src", {})
         assert "execution_failed" not in result.metadata
 
+    def test_raw_output_dir_persists_per_scanner_files(self, monkeypatch, tmp_path):
+        """When ``raw_output_dir`` is set, the engine copies each
+        scanner's raw output (results.json / *.sarif / stdout.txt)
+        into ``<raw_output_dir>/<scanner.name>/`` before the tempdir
+        is wiped. Mirrors the container-scan flow's ``raw/`` artifact
+        preservation so source scans aren't an inconsistent
+        second-class case."""
+        engine = self._make_engine()
+        scanner = self._make_scanner(parse_results=lambda f: [])
+        engine._raw_output_root = str(tmp_path / "raw")
+
+        monkeypatch.setattr(engine, "_pull_image", lambda img: True)
+        monkeypatch.setattr(engine, "_get_image_digest", lambda img: "sha256:abc")
+
+        def mock_run(cmd, **_kwargs):
+            for i, arg in enumerate(cmd):
+                if ":/output" in str(arg):
+                    Path(arg.split(":")[0]).joinpath("results.json").write_text(
+                        '{"findings": []}'
+                    )
+                    break
+            return subprocess.CompletedProcess(
+                args=cmd, returncode=0, stdout="", stderr="",
+            )
+
+        monkeypatch.setattr(subprocess, "run", mock_run)
+        engine._run_in_container(scanner, "/src", {})
+
+        # File landed at <raw_output_root>/<scanner_name>/results.json
+        # — the per-scanner subdir keeps multi-scanner runs from
+        # colliding on common filenames.
+        persisted = tmp_path / "raw" / scanner.name / "results.json"
+        assert persisted.exists()
+        # Contents survived intact.
+        assert "findings" in persisted.read_text()
+
+    def test_raw_output_dir_none_does_not_persist_files(
+        self, monkeypatch, tmp_path,
+    ):
+        """Default behavior — ``raw_output_dir`` unset — leaves no
+        per-scanner artifacts on disk after the run. Confirms the
+        copy step is opt-in, not always-on."""
+        engine = self._make_engine()
+        scanner = self._make_scanner(parse_results=lambda f: [])
+        # Explicitly None (the default).
+        engine._raw_output_root = None
+
+        monkeypatch.setattr(engine, "_pull_image", lambda img: True)
+        monkeypatch.setattr(engine, "_get_image_digest", lambda img: "sha256:abc")
+
+        def mock_run(cmd, **_kwargs):
+            for i, arg in enumerate(cmd):
+                if ":/output" in str(arg):
+                    Path(arg.split(":")[0]).joinpath("results.json").write_text("{}")
+                    break
+            return subprocess.CompletedProcess(
+                args=cmd, returncode=0, stdout="", stderr="",
+            )
+
+        monkeypatch.setattr(subprocess, "run", mock_run)
+        engine._run_in_container(scanner, "/src", {})
+
+        # tmp_path is otherwise untouched.
+        assert list(tmp_path.iterdir()) == []
+
+    def test_raw_output_persists_stdout_fallback(self, monkeypatch, tmp_path):
+        """Some scanners write to stdout instead of a file (e.g.
+        ClamAV). The engine captures that as ``stdout.txt`` in the
+        per-scanner tempdir; raw-output preservation should pick it
+        up the same way it picks up regular result files."""
+        engine = self._make_engine()
+        scanner = self._make_scanner(parse_results=lambda f: [])
+        engine._raw_output_root = str(tmp_path / "raw")
+
+        monkeypatch.setattr(engine, "_pull_image", lambda img: True)
+        monkeypatch.setattr(engine, "_get_image_digest", lambda img: "sha256:abc")
+
+        def mock_run(cmd, **_kwargs):
+            return subprocess.CompletedProcess(
+                args=cmd, returncode=0,
+                stdout="scanner output line 1\nscanner output line 2\n",
+                stderr="",
+            )
+
+        monkeypatch.setattr(subprocess, "run", mock_run)
+        engine._run_in_container(scanner, "/src", {})
+
+        persisted = tmp_path / "raw" / scanner.name / "stdout.txt"
+        assert persisted.exists()
+        assert "scanner output line" in persisted.read_text()
+
+    def test_raw_output_skips_zero_byte_files(self, monkeypatch, tmp_path):
+        """0-byte files are explicit failure signals upstream
+        (``_validate_scanner_output`` rejects them in the container
+        sub-scanners). Don't persist them — making known-broken
+        output look authoritative on disk would mislead anyone
+        triaging from the saved artifacts."""
+        engine = self._make_engine()
+        scanner = self._make_scanner(parse_results=lambda f: [])
+        engine._raw_output_root = str(tmp_path / "raw")
+
+        monkeypatch.setattr(engine, "_pull_image", lambda img: True)
+        monkeypatch.setattr(engine, "_get_image_digest", lambda img: "sha256:abc")
+
+        def mock_run(cmd, **_kwargs):
+            for i, arg in enumerate(cmd):
+                if ":/output" in str(arg):
+                    Path(arg.split(":")[0]).joinpath("results.json").touch()
+                    break
+            return subprocess.CompletedProcess(
+                args=cmd, returncode=0, stdout="", stderr="",
+            )
+
+        monkeypatch.setattr(subprocess, "run", mock_run)
+        engine._run_in_container(scanner, "/src", {})
+
+        target_dir = tmp_path / "raw" / scanner.name
+        # Either no dir created or empty — never a 0-byte stub.
+        if target_dir.exists():
+            assert not list(target_dir.iterdir())
+
     def test_container_custom_entrypoint(self, monkeypatch):
         engine = self._make_engine()
         scanner = self._make_scanner(container_entrypoint="/bin/custom")

docs/cli-reference.md

@@ -64,8 +64,8 @@ argus scan [-h] [--path PATH] [--config CONFIG]
                   [--interface {terminal,browser}] [--fail-fast]
                   [--fail-on-scanner-error] [--timeout SECONDS]
                   [--no-parallel] [--allow-local-versions] [--no-cache]
-                  [--discover [PATH]] [--image REF] [--scanners SCANNERS]
-                  [--no-keep-raw] [--target URL] [--port PORT]
+                  [--no-keep-raw] [--discover [PATH]] [--image REF]
+                  [--scanners SCANNERS] [--target URL] [--port PORT]
                   [--env KEY=VALUE] [--scan-type {baseline,full}]
                   [--startup-timeout STARTUP_TIMEOUT]
                   [scanner]
@@ -100,6 +100,7 @@ argus scan [-h] [--path PATH] [--config CONFIG]
 | `--no-parallel` | Run scanners sequentially instead of concurrently. | `false` |
 | `--allow-local-versions` | Allow local tool versions that differ from argus-pinned versions. Use in airgapped environments where tool updates are constrained. | `false` |
 | `--no-cache` | Disable DB cache volume mounts. Forces scanners to re-download vulnerability databases on every container run. | `false` |
+| `--no-keep-raw` | Do not persist raw per-scanner output files alongside the canonical argus-results.json. Source scans normally drop each scanner's results.json / *.sarif / stdout.txt under <output_dir>/raw/<scanner>/; container scans drop trivy-results.json / grype-results.json / syft-sbom.json under <output_dir>/raw/<image>/. Pass --no-keep-raw to skip that step in tight CI environments. The same effect is available via 'reporting.keep_raw: false' in argus.yml. | `false` |
 
 **Container Scanning:**
 
@@ -108,7 +109,6 @@ argus scan [-h] [--path PATH] [--config CONFIG]
 | `--discover` | Discover Dockerfiles in PATH (default: current directory) |  |
 | `--image` | Container image to scan (can be repeated) |  |
 | `--scanners` | Sub-scanners for container scanning: trivy,grype,syft (default: trivy,grype) |  |
-| `--no-keep-raw` | Do not persist raw per-scanner output (trivy-results.json, grype-results.json, syft-sbom.json) under <output_dir>/raw/<image>/. By default raw artifacts are kept alongside the canonical argus-results.json so users can drill into individual scanner output for forensics or manual triage. Set ``containers.keep_raw: false`` in argus.yml for the same effect via config. | `false` |
 
 **Dast Scanning:**