Add Trivy security scanner

docs/docs/docker/index.mdx

@@ -396,17 +396,32 @@ It can help to simulate a hermetic environment by using `env -i`. With credentia
 
 :::
 
-## Linting Dockerfiles with Hadolint
+## Linting Dockerfiles
 
-Pants can run [Hadolint](https://github.com/hadolint/hadolint) on your Dockerfiles to check for errors and mistakes:
+Once Docker linting backends are enabled, lint Dockerfiles with
 
 ```
 ❯ pants lint src/docker/hw/Dockerfile
 ```
 
+
+### Linting Dockerfiles with Hadolint
+
+Pants can run [Hadolint](https://github.com/hadolint/hadolint) on your Dockerfiles to check for errors and mistakes.
 This must first be enabled by activating the Hadolint backend:
 
 ```toml title="pants.toml"
 [GLOBAL]
 backend_packages = ["pants.backend.docker.lint.hadolint"]
 ```
+
+### Linting Dockerfiles with Trivy
+
+Pants can run [Trivy](https://github.com/aquasecurity/trivy) on your Dockerfiles to check for security vulnerabilities.
+This must first be enabled by activating the Trivy backend:
+
+```toml title="pants.toml"
+[GLOBAL]
+backend_packages = ["pants.backend.docker.lint.trivy"]
+```
+

docs/docs/helm/index.mdx

@@ -65,11 +65,11 @@ Created src/helm/example/BUILD:
 
 If your workspace contains any Helm unit tests (under a `tests` folder), Pants will also idenfity them and create `helm_unittest_tests` targets for them. Additionally, if your unit tests also have snapshots (under a `tests/__snapshot__` folder), `tailor` will identify those files as test snapshots and will create `resources` targets for them. See "Snapshot testing" below for more info.
 
-### Basic operations
+## Basic operations
 
 The given setup is enough to now do some common operations on our Helm chart source code.
 
-#### Linting
+### Linting
 
 The Helm backend has an implementation of the Pants' `lint` goal which hooks it with the `helm lint` command:
 
@@ -100,7 +100,19 @@ Likewise, in a similar way you could enable strict linting globally and then cho
 
 You can set the field `skip_lint=True` on each `helm_chart` target to avoid linting it.
 
-#### Package
+#### Trivy
+
+Pants can run [Trivy](https://github.com/aquasecurity/trivy) on your Helm Charts and deployments. When run on a deployment, Pants run Trivy on the rendered Helm Chart with the specified values.
+
+This must first be enabled by activating the Trivy backend:
+
+```toml title="pants.toml"
+[GLOBAL]
+backend_packages = ["pants.backend.experimental.helm.lint.trivy"]
+```
+
+### Package
+### Package
 
 Packing helm charts is supported out of the box via the Pants' `package` goal. The final package will be saved as a `.tgz` file under the `dist` folder at your source root.
 
@@ -201,7 +213,7 @@ pants test ::
 ✓ testprojects/src/helm/example/tests/env-configmap_test.yaml succeeded in 0.75s.
 ```
 
-#### Feeding additional files to unit tests
+### Feeding additional files to unit tests
 
 In some cases we may want our tests to have access to additional files which are not part of the chart. This can be achieved by setting a dependency between our unit test targets and a `resources` target as follows:
 
@@ -304,7 +316,7 @@ not affected by the source roots setting. When using `relocated_files`, the file
 to the value set in the `dest` field.
 :::
 
-#### Snapshot testing
+### Snapshot testing
 
 Unit test snapshots are supported by Pants by wrapping the snapshots in resources targets, as shown in the previous section. Snapshot resources will be automatically inferred as dependencies of the tests where they reside, so there is no need to add a explicit `dependencies` relationship in your `helm_unittest_tests` targets.
 
@@ -318,7 +330,7 @@ This will generate test snapshots for tests that require them, with out-of-date
 
 If new `__snapshot__` folders are created after running the `generate-snapshots` target, we recommend running the `tailor` goal again so that Pants can detect these new folders and create `resources` targets as appropriate.
 
-#### Timeouts
+### Timeouts
 
 Pants can cancel tests that take too long, which is useful to prevent tests from hanging indefinitely.
 
@@ -352,7 +364,7 @@ If a target sets its `timeout` higher than `[test].timeout_maximum`, Pants will
 
 Use the option `pants test --no-timeouts` to temporarily disable timeouts, e.g. when debugging.
 
-#### Retries
+### Retries
 
 Pants can automatically retry failed tests. This can help keep your builds passing even with flaky tests, like integration tests.
 

docs/docs/helm/kubeconform.mdx

@@ -7,7 +7,7 @@
 
 ## Overview
 
-The Helm backend has opt-in support for using [Kubeconform](https://github.com/yannh/kubeconform) as a validation tool for both Helm charts and deployments. This gives the extra confidence that the templates defined in the different charts are conformant to a Kubernetes version or specification, plus the addtional benefit of ensuring that their final version (when used in deployments) also meets that criteria.
+The Helm backend has opt-in support for using [Kubeconform](https://github.com/yannh/kubeconform) as a validation tool for both Helm charts and deployments. This gives the extra confidence that the templates defined in the different charts are conformant to a Kubernetes version or specification, plus the additional benefit of ensuring that their final version (when used in deployments) also meets that criteria.
 
 To enable the usage of Kubeconform, first we need to activate the relevant backend in `pants.toml`:
 

docs/docs/terraform/index.mdx

@@ -192,3 +192,16 @@ To run `terraform plan`, use the `--dry-run` flag of the `experimental-deploy` g
 ```
 pants experimental-deploy --dry-run ::
 ```
+
+### Linters
+
+#### Trivy
+
+Pants can run [Trivy](https://github.com/aquasecurity/trivy) on your Terraform modules and deployments. When run against deployments, the relevant vars files will be used. When run against modules, no vars files will be passed.
+
+This must first be enabled by activating the Trivy backend:
+
+```toml title="pants.toml"
+[GLOBAL]
+backend_packages = ["pants.backend.experimental.terraform.lint.trivy"]
+```

docs/docs/writing-plugins/common-plugin-tasks/add-a-linter.mdx

@@ -210,7 +210,7 @@ At the bottom of your file, tell Pants about your rules:
 def rules():
     return [
       	*collect_rules(),
-        ShellcheckRequest.rules(partitioner_type=PartitionerType.DEFAULT_SINGLE_PARTITION),
+        *ShellcheckRequest.rules(),
         UnionRule(ExportableTool, Shellcheck),  # allows exporting the `shellcheck` binary
     ]
 ```

docs/docs/writing-plugins/common-subsystem-tasks.mdx

@@ -96,7 +96,7 @@ Only some language backends support `pants export`. These include the Python and
                 f"""
                 If true, Pants will include all relevant config files during runs.
 
-                Use `[{cls.options_scope}].config` and `[{cls.options_scope}].custom_check_dir` instead if your config is in a non-standard location.
+                Use `[{cls.options_scope}].config` instead if your config is in a non-standard location.
                 """
             ),
         )
@@ -161,7 +161,7 @@ Only some language backends support `pants export`. These include the Python and
         )
 
         args = []
-        if subsystem.config_request:
+        if subsystem.config:
             args.append(f"--config-file={subsystem.config}")
 
         # run your process with the digest and args

docs/notes/2.27.x.md

@@ -48,6 +48,12 @@ Pants has a new mechanism for `@rule` invocation in backends. In this release th
 
 [The `[dockerfile-parser].use_rust_parser` option](https://www.pantsbuild.org/2.27/reference/subsystems/dockerfile-parser) now defaults to true, meaning, by default, Dockerfiles are now parsed using the native Rust-based parser, which is faster and requires no external dependencies. The old parser is deprecated and will be removed in a future version of Pants.
 
+Run [Trivy](https://github.com/aquasecurity/trivy) on Dockerfiles to scan for vulnerable packages.
+
+#### Helm
+
+Run [Trivy](https://github.com/aquasecurity/trivy) on Helm Charts and deployments to scan for misconfigurations.
+
 #### Javascript
 
 Added support for formatting typescript and tsx files with Prettier.
@@ -88,6 +94,8 @@ Now supports codegen for module dependencies. Dependencies may specify a target
 
 Now `terraform init` is run in the same invocation as commands which depend on it. This should resolve an issue for users of remote caches where the Terraform provider cache is not initialised on different nodes. 
 
+Run [Trivy](https://github.com/aquasecurity/trivy) on Terraform modules and deployments to scan for misconfigurations.
+
 ### Plugin API changes
 
 * Processes can now specify their `concurrency` requirements, influencing when Pants will execute them. Use `exclusive` to be the only running process, `exactly(n)` to require exactly `n` cpu cores, or `range(max=n, min=1)` to accept a value between `min` and `max` which is templated into the process's argv as `{pants_concurrency}`. The `concurrency` field supersedes the `concurrency_available` field, which will be deprecated in the future.

src/python/pants/backend/docker/lint/trivy/BUILD

@@ -0,0 +1,8 @@
+# Copyright 2024 Pants project contributors (see CONTRIBUTORS.md).
+# Licensed under the Apache License, Version 2.0 (see LICENSE).
+python_sources()
+
+python_tests(
+    name="tests",
+    timeout=120,
+)

src/python/pants/backend/docker/lint/trivy/rules.py

@@ -0,0 +1,88 @@
+# Copyright 2024 Pants project contributors (see CONTRIBUTORS.md).
+# Licensed under the Apache License, Version 2.0 (see LICENSE).
+from dataclasses import dataclass
+from typing import Any, cast
+
+from pants.backend.docker.package_types import BuiltDockerImage
+from pants.backend.docker.target_types import DockerImageSourceField, DockerImageTarget
+from pants.backend.tools.trivy.rules import RunTrivyRequest, run_trivy
+from pants.backend.tools.trivy.subsystem import SkipTrivyField, Trivy
+from pants.core.goals.lint import LintResult, LintTargetsRequest
+from pants.core.goals.package import BuiltPackage, EnvironmentAwarePackageRequest, PackageFieldSet
+from pants.core.util_rules.partitions import PartitionerType
+from pants.engine.addresses import Addresses
+from pants.engine.internals.native_engine import EMPTY_DIGEST
+from pants.engine.internals.selectors import Get
+from pants.engine.rules import collect_rules, implicitly, rule
+from pants.engine.target import (
+    FieldSet,
+    FieldSetsPerTarget,
+    FieldSetsPerTargetRequest,
+    Target,
+    Targets,
+)
+from pants.util.logging import LogLevel
+
+
+@dataclass(frozen=True)
+class TrivyDockerFieldSet(FieldSet):
+    required_fields = (DockerImageSourceField,)
+
+    source: DockerImageSourceField
+
+    @classmethod
+    def opt_out(cls, tgt: Target) -> bool:
+        return tgt.get(SkipTrivyField).value
+
+
+class TrivyDockerRequest(LintTargetsRequest):
+    field_set_type = TrivyDockerFieldSet
+    tool_subsystem = Trivy
+    partitioner_type = PartitionerType.DEFAULT_ONE_PARTITION_PER_INPUT
+
+
+def command_args():
+    return (
+        # workaround for Trivy DB being overloaded on pulls
+        "--db-repository",
+        "ghcr.io/aquasecurity/trivy-db,public.ecr.aws/aquasecurity/trivy-db",
+        # quiet progress output, which just clutters logs
+        "--no-progress",
+    )
+
+
+@rule(desc="Lint Docker image with Trivy", level=LogLevel.DEBUG)
+async def run_trivy_docker(
+    request: TrivyDockerRequest.Batch[TrivyDockerFieldSet, Any],
+) -> LintResult:
+    addrs = tuple(e.address for e in request.elements)
+    tgts = await Get(Targets, Addresses(addrs))
+
+    field_sets_per_tgt = await Get(
+        FieldSetsPerTarget, FieldSetsPerTargetRequest(PackageFieldSet, tgts)
+    )
+    [field_set] = field_sets_per_tgt.field_sets
+
+    package = await Get(BuiltPackage, EnvironmentAwarePackageRequest(field_set))
+    built_image: BuiltDockerImage = cast(BuiltDockerImage, package.artifacts[0])
+    r = await run_trivy(
+        RunTrivyRequest(
+            command="image",
+            command_args=command_args(),
+            scanners=(),
+            target=built_image.image_id,
+            input_digest=EMPTY_DIGEST,
+            description=f"Run Trivy on docker image {','.join(built_image.tags)}",
+        ),
+        **implicitly(),
+    )
+
+    return LintResult.create(request, r)
+
+
+def rules():
+    return (
+        *collect_rules(),
+        *TrivyDockerRequest.rules(),
+        DockerImageTarget.register_plugin_field(SkipTrivyField),
+    )

src/python/pants/backend/docker/lint/trivy/trivy_integration_test.py

@@ -0,0 +1,100 @@
+# Copyright 2024 Pants project contributors (see CONTRIBUTORS.md).
+# Licensed under the Apache License, Version 2.0 (see LICENSE).
+from textwrap import dedent
+
+import pytest
+
+from pants.backend.docker.lint.trivy.rules import TrivyDockerFieldSet, TrivyDockerRequest
+from pants.backend.docker.lint.trivy.rules import rules as trivy_docker_rules
+from pants.backend.docker.rules import rules as docker_rules
+from pants.backend.docker.target_types import DockerImageTarget
+from pants.backend.tools.trivy.rules import rules as trivy_rules
+from pants.backend.tools.trivy.testutil import (
+    assert_trivy_output,
+    assert_trivy_success,
+    trivy_config,
+)
+from pants.core.goals import package
+from pants.core.goals.lint import LintResult
+from pants.core.util_rules import source_files
+from pants.core.util_rules.partitions import PartitionMetadata
+from pants.engine.internals.native_engine import Address
+from pants.engine.rules import QueryRule
+from pants.testutil.rule_runner import RuleRunner
+
+
+@pytest.fixture
+def rule_runner() -> RuleRunner:
+    from pants.core.goals.lint import LintResult
+    from pants.core.target_types import FileTarget
+
+    rule_runner = RuleRunner(
+        target_types=[DockerImageTarget, FileTarget],
+        rules=[
+            *trivy_docker_rules(),
+            *trivy_rules(),
+            *docker_rules(),
+            *package.rules(),
+            *source_files.rules(),
+            QueryRule(LintResult, [TrivyDockerRequest.Batch]),
+        ],
+    )
+    rule_runner.write_files(
+        {
+            "Dockerfile.good": GOOD_FILE,
+            "Dockerfile.bad": BAD_FILE,
+            "file.txt": "",
+            "BUILD": dedent(
+                """
+            file(name="file", source="file.txt")
+            docker_image(name="good", source="Dockerfile.good", dependencies=[":file"])
+            docker_image(name="bad", source="Dockerfile.bad")
+            """
+            ),
+            "trivy.yaml": trivy_config,
+        }
+    )
+    # DOCKER_HOST allows for humans with rootless docker to run docker-dependent tests
+    rule_runner.set_options(
+        ("--trivy-extra-env-vars=DOCKER_HOST",),
+        env_inherit={"PATH", "DOCKER_HOST"},
+    )
+
+    return rule_runner
+
+
+GOOD_FILE = "FROM scratch\nCOPY file.txt /"  # A Docker image with nothing but a file is secure
+
+BAD_FILE = (
+    "FROM alpine:3.14.9@sha256:fa26727c28837d1471c2f1524d297a0255c153b5d023d7badd1412be7e6e12a2"
+)
+BAD_IMAGE_TARGET = "sha256:9e02963d7df7e8da13c08d23fd2f09b9dcf779422151766a8963415994e74ae0 (alpine 3.14.9)"  # this is Trivy's "Target" field
+
+
+def test_trivy_good(rule_runner: RuleRunner) -> None:
+    tgt_good = rule_runner.get_target(Address("", target_name="good"))
+
+    result = rule_runner.request(
+        LintResult,
+        [
+            TrivyDockerRequest.Batch(
+                "trivy", (TrivyDockerFieldSet.create(tgt_good),), PartitionMetadata
+            )
+        ],
+    )
+
+    assert_trivy_success(result)
+
+
+def test_trivy_bad(rule_runner: RuleRunner) -> None:
+    tgt_bad = rule_runner.get_target(Address("", target_name="bad"))
+
+    result = rule_runner.request(
+        LintResult,
+        [
+            TrivyDockerRequest.Batch(
+                "trivy", (TrivyDockerFieldSet.create(tgt_bad),), PartitionMetadata
+            )
+        ],
+    )
+    assert_trivy_output(result, 1, BAD_IMAGE_TARGET, "image", 4)

src/python/pants/backend/experimental/docker/lint/trivy/register.py

@@ -0,0 +1,13 @@
+# Copyright 2024 Pants project contributors (see CONTRIBUTORS.md).
+# Licensed under the Apache License, Version 2.0 (see LICENSE).
+from pants.backend.docker.lint.trivy.rules import rules as trivy_docker_rules
+from pants.backend.docker.rules import rules as docker_rules
+from pants.backend.tools.trivy.rules import rules as trivy_rules
+
+
+def rules():
+    return (
+        *docker_rules(),
+        *trivy_rules(),
+        *trivy_docker_rules(),
+    )

src/python/pants/backend/experimental/helm/lint/trivy/BUILD

@@ -0,0 +1,4 @@
+# Copyright 2024 Pants project contributors (see CONTRIBUTORS.md).
+# Licensed under the Apache License, Version 2.0 (see LICENSE).
+
+python_sources()