[ABLD-395] Framework to create toolchains for OS provided tools (#54177)

### What does this PR do?

- Creates a tiny framework for finding an OS provided tool and telling Bazel it is a toolchain.
- Use it for macos pkgbuild and codesign.

The important feature of the way this is implemented is that the toolchain always safely resolves on all platforms, so you can `bazel cquery` across platforms, even though you will fail if you try to `build`. Combined with `exec_compatible_with` you can make targets that only build if the required tools are available. This is important when there are different tools available as OSes evolve (or are simply different by vendor, like debian and redhat) and we need to create paths that can gracefully use different tools depending on what is available.

There is also a capability to use a bazel target as the tool implementation instead of looking it up in $PATH. That is useful for building alternate implementations (let's say a windows tool on linux). That will come into use in full remote execution mode, when the target and exec hosts can be different architectures.

**Examples:**
- I'm building a tarball, see if I have the xz toolchain and use that if available, otherwise fall back to a slower compresser
- The OS is evolving a tool, if we have version 1, it might have one name. In version 2, there might be a better tool. We want to make rules that can do the right thing regardless of what OS we have. This is a huge win for external users who may be using a different base linux to build.


### Motivation

This is an important part of our supply chain security posture. We want to account for every binary we use to build the product. Ideally that would include even basic low level things, like bash and cat. In practice we can make the tradeoff to stop at declaring the things that might vary across OSes or from OS release to release.

Eventually, we will expand this the framework to include package metadata (https://github.com/bazel-contrib/supply-chain/tree/main/metadata) that can be easily gathered into a workspace BOM.

The more immediate need is to create toolchains for pkgbuild and codesign to create rules to make macos packages. #54115

### Describe how you validated your changes
With the BUILD file
```
genrule(
    name = "test",
    cmd = select({
        "@macos_codesign//:have_codesign": "echo GOT IT",
        "//conditions:default": "echo wump, wump, wump",
    }),
    outs = ["test.out"],
)
```

macos:
```
$ bazel cquery --output=build  //ztony:test
genrule(
  name = "test",
  outs = ["//ztony:test.out"],
  cmd = "echo GOT IT",
)
$ bazel cquery @macos_codesign//:all
INFO: Analyzed 7 targets (0 packages loaded, 6 targets configured).
INFO: Found 7 targets...
@macos_codesign//:codesign_auto (eab6d8f)
@macos_codesign//:codesign_toolchain (eab6d8f)
@macos_codesign//:codesign_toolchain_type (eab6d8f)
@macos_codesign//:have_codesign (eab6d8f)
@macos_codesign//:is_codesign_available (eab6d8f)
@macos_codesign//:no_codesign (eab6d8f)
@macos_codesign//:zzz_codesign_missing_toolchain (eab6d8f)
```
linux:
```
$ bazel cquery --output=build  //ztony:test
genrule(
  name = "test",
  outs = ["//ztony:test.out"],
  cmd = "echo wump, wump, wump",
)
$ bazel cquery @macos_codesign//:all
@macos_codesign//:codesign_auto (a938602)
@macos_codesign//:codesign_toolchain (a938602)
@macos_codesign//:codesign_toolchain_type (a938602)
@macos_codesign//:have_codesign (a938602)
@macos_codesign//:is_codesign_available (a938602)
@macos_codesign//:no_codesign (a938602)
@macos_codesign//:zzz_codesign_missing_toolchain (a938602)
```



### Additional
This is an alternate version of #54154. I think I like it better.
[ABLD-386]: https://datadoghq.atlassian.net/browse/ABLD-386



Co-authored-by: tony.aiuto <tony.aiuto@datadoghq.com>

Author: aiuto

MODULE.bazel

@@ -126,6 +126,30 @@ archive_override(
     urls = ["https://github.com/bazel-contrib/toolchains_llvm/archive/ffe407ca2423f4d171b7a0f3897c23bbb7e40310.tar.gz"],
 )
 
+#################################
+## Tools that ship with the OS ##
+#################################
+
+find_macos_codesign = use_repo_rule("//bazel/toolchains/codesign:configure.bzl", "find_macos_codesign")
+
+find_macos_codesign(
+    name = "macos_codesign",
+    exec_compatible_with = ["@platforms//os:macos"],
+)
+
+find_macos_pkgbuild = use_repo_rule("//bazel/toolchains/pkgbuild:configure.bzl", "find_macos_pkgbuild")
+
+find_macos_pkgbuild(
+    name = "macos_pkgbuild",
+    exec_compatible_with = ["@platforms//os:macos"],
+)
+
+register_toolchains(
+    "@macos_codesign//:all",
+    "@macos_pkgbuild//:all",
+    dev_dependency = True,
+)
+
 #########################
 ## Prebuilt binaries   ##
 #########################

bazel/toolchains/codesign/BUILD.bazel

@@ -0,0 +1,9 @@
+"""toolchain to wrap the codesign binary.
+
+Type: @codesign//:codesign_toolchain_type
+
+Toolchains:
+- @codesign//:codesign_toolchain: provides the tool
+- @codesign//:codesign_missing_toolchain: provides a fallback toolchain for
+  exec platforms where codesign might not be available.
+"""

bazel/toolchains/codesign/configure.bzl

@@ -0,0 +1,8 @@
+"""Repository rule to autoconfigure a toolchain using the system codesign."""
+
+load("//bazel/toolchains/common:defs.bzl", "make_toolchain_repository_rule")
+
+# This must match the repository name used by register_toolchains in consuming MODULE.bazel files.
+NAME = "macos_codesign"
+
+find_macos_codesign = make_toolchain_repository_rule(name = NAME, tool_name = "codesign")

bazel/toolchains/common/BUILD.bazel

@@ -0,0 +1 @@
+# //bazel/toolchains/common

bazel/toolchains/common/defs.bzl

@@ -0,0 +1,63 @@
+"""Utilitites for creating toolchains to wrap system provided tools."""
+
+def _write_toolchain_repo(rctx, repo_name, tool_name, tool_path, tool_version = "<unknown>", exec_compatible_with = None):
+    if not tool_path:
+        tool_path = ""
+    rctx.template(
+        "BUILD",
+        Label("@@//bazel/toolchains/common:toolchain_BUILD.tpl"),
+        substitutions = {
+            "{AVAILABLE}": "1" if tool_path else "0",
+            "{EXEC_COMPATIBLE_WITH}": repr(exec_compatible_with),
+            "{GENERATOR}": "//bazel/toolchains/common:defs.bzl",
+            "{REPO_NAME}": repo_name,
+            "{TOOL_NAME}": tool_name,
+            "{TOOL_PATH}": str(tool_path),
+            "{TOOL_VERSION}": tool_version,
+        },
+        executable = False,
+    )
+    rctx.template(
+        "defs.bzl",
+        Label("@@//bazel/toolchains/common:toolchain_defs.bzl.tpl"),
+        substitutions = {
+            "{AVAILABLE}": "1" if tool_path else "0",
+            "{GENERATOR}": "//bazel/toolchains/common:defs.bzl",
+            "{REPO_NAME}": repo_name,
+            "{TOOL_NAME}": tool_name,
+        },
+        executable = False,
+    )
+
+def _default_repo_builder_impl(rctx):
+    tool_name = rctx.attr.tool_name
+    tool_path = rctx.which(tool_name)
+    if rctx.attr.verbose:
+        if tool_path:
+            print("Found %s at '%s'" % (tool_name, tool_path))  # buildifier: disable=print
+        else:
+            print("No system %s found." % tool_name)  # buildifier: disable=print
+    _write_toolchain_repo(
+        rctx = rctx,
+        repo_name = rctx.original_name,
+        tool_name = rctx.attr.tool_name,
+        tool_path = tool_path,
+        exec_compatible_with = rctx.attr.exec_compatible_with,
+    )
+
+def make_toolchain_repository_rule(name, tool_name, impl = _default_repo_builder_impl):
+    return repository_rule(
+        implementation = impl,
+        doc = """Create a repository that defines a {name} toolchain based on tool in the default $PATH.""".format(name = name),
+        local = True,
+        environ = ["PATH"],
+        attrs = {
+            "tool_name": attr.string(doc = "The name of the tool to find.", default = tool_name),
+            "exec_compatible_with": attr.string_list(
+                doc = "exec_compatible_with list to apply to the created toolchain.",
+            ),
+            "verbose": attr.bool(
+                doc = "If true, print status messages.",
+            ),
+        },
+    )

bazel/toolchains/common/toolchain_BUILD.tpl

@@ -0,0 +1,51 @@
+# This content is generated by {GENERATOR}
+load(":defs.bzl", "is_{TOOL_NAME}_available", "{TOOL_NAME}_toolchain")
+
+toolchain_type(
+    name = "{TOOL_NAME}_toolchain_type",
+    visibility = ["//visibility:public"],
+)
+
+{TOOL_NAME}_toolchain(
+    name = "{TOOL_NAME}_auto",
+    path = "{TOOL_PATH}",
+    version = "{TOOL_VERSION}",
+)
+
+toolchain(
+    name = "{TOOL_NAME}_toolchain",
+    toolchain = ":{TOOL_NAME}_auto",
+    toolchain_type = ":{TOOL_NAME}_toolchain_type",
+    exec_compatible_with = {EXEC_COMPATIBLE_WITH},
+)
+
+# {TOOL_NAME}_missing_toolchain provides a fallback toolchain so that toolchain
+# resolution can succeed even on platforms that do not have that tool.
+# If this toolchain is selected, a rule can examine the valid field to determine
+# if the tool is really available.
+{TOOL_NAME}_toolchain(
+    name = "no_{TOOL_NAME}",
+)
+
+toolchain(
+    name = "zzz_{TOOL_NAME}_missing_toolchain",  # keep name lexicographically last
+    toolchain = ":no_{TOOL_NAME}",
+    toolchain_type = ":{TOOL_NAME}_toolchain_type",
+)
+
+# Expose the availability of the actual tool as a flag, so we can
+# create a config_setting from it.
+is_{TOOL_NAME}_available(
+    name = "is_{TOOL_NAME}_available",
+    build_setting_default = {AVAILABLE},
+)
+
+# Expose the availability of the toolchain as a config_setting, so we can
+# select() on it.
+config_setting(
+    name = "have_{TOOL_NAME}",
+    flag_values = {
+        ":is_{TOOL_NAME}_available": "1",
+    },
+    visibility = ["//visibility:public"],
+)

bazel/toolchains/common/toolchain_defs.bzl.tpl

@@ -0,0 +1,48 @@
+"""toolchain to provide the {TOOL_NAME} binary."""
+
+load("@@//bazel/toolchains:toolchain_info.bzl", "ToolInfo")
+
+def _{TOOL_NAME}_toolchain_impl(ctx):
+    if ctx.attr.label and ctx.attr.path:
+        fail("{TOOL_NAME}_toolchain must not specify both label and path.")
+    valid = bool(ctx.attr.label) or bool(ctx.attr.path)
+    toolchain_info = platform_common.ToolchainInfo(
+        {TOOL_NAME} = ToolInfo(
+            name = str(ctx.label),
+            valid = valid,
+            label = ctx.attr.label,
+            path = ctx.attr.path,
+            version = ctx.attr.version,
+        ),
+    )
+    return [toolchain_info]
+
+{TOOL_NAME}_toolchain = rule(
+    implementation = _{TOOL_NAME}_toolchain_impl,
+    attrs = {
+        "label": attr.label(
+            doc = "A valid label of a target to build or a prebuilt binary. Mutually exclusive with path.",
+            cfg = "exec",
+            executable = True,
+            allow_files = True,
+        ),
+        "path": attr.string(
+            doc = "The path to the executable. Mutually exclusive with label.",
+        ),
+        "version": attr.string(
+            doc = "The version string of the executable. This should be manually set.",
+        ),
+    },
+)
+
+# Expose the presence of {TOOL_NAME} as a flag.
+def _is_{TOOL_NAME}_available_impl(ctx):
+    return [config_common.FeatureFlagInfo(
+        value = ("1" if ctx.build_setting_value else "0"),
+    )]
+
+is_{TOOL_NAME}_available = rule(
+    implementation = _is_{TOOL_NAME}_available_impl,
+    attrs = {},
+    build_setting = config.bool(flag = False),
+)

bazel/toolchains/pkgbuild/BUILD.bazel

@@ -0,0 +1,8 @@
+"""Toolchain to wrap the pkgbuild binary.
+
+Type: @macos_pkgbuild//:tool_toolchain_type
+
+Toolchains:
+- @macos_pkgbuild//:macos_pkgbuild_toolchain: provides the tool
+- @macos_pkgbuild//:macos_pkgbuild_missing_toolchain: provides a fallback toolchain for exec platforms where pkgbuild might not be available.
+"""

bazel/toolchains/pkgbuild/configure.bzl

@@ -0,0 +1,8 @@
+"""Repository rule to autoconfigure a toolchain using the system pkgbuild."""
+
+load("//bazel/toolchains/common:defs.bzl", "make_toolchain_repository_rule")
+
+# This must match the name used by register_toolchains in MODULE.bazel.
+NAME = "macos_pkgbuild"
+
+find_macos_pkgbuild = make_toolchain_repository_rule(name = NAME, tool_name = "pkgbuild")