[DSEC-118] Build and ship Rust shared-library checks in agent packages (#53498)

### What does this PR do?

- Add [`shared_checks_manifest.yaml`](https://github.com/DataDog/datadog-agent/blob/aimene.belfodil/dsec/build-rust-checks/pkg/collector/sharedlibrary/rustchecks/shared_checks_manifest.yaml) and a [`rust-shared-checks.build`](https://github.com/DataDog/datadog-agent/blob/aimene.belfodil/dsec/build-rust-checks/tasks/rust_shared_checks.py) invoke task to build manifest-selected `cdylib` checks and stage them into `checks.d` as `libdatadog-agent-<id>.so` with `0500` permissions (Linux only).
- Wire the task into the omnibus agent build ([`datadog-agent.rb`](https://github.com/DataDog/datadog-agent/blob/aimene.belfodil/dsec/build-rust-checks/omnibus/config/software/datadog-agent.rb#L108-L112)); it runs for Linux only (macOS and Windows excluded).
- Move staged shared libs into the Linux package config during [`datadog-agent-finalize.rb`](https://github.com/DataDog/datadog-agent/blob/aimene.belfodil/dsec/build-rust-checks/omnibus/config/software/datadog-agent-finalize.rb#L99-L103) and enforce owner-only (`0500`) perms.
- Re-apply `0500` on the shared-library checks in the [`agent`](https://github.com/DataDog/datadog-agent/blob/aimene.belfodil/dsec/build-rust-checks/Dockerfiles/agent/Dockerfile#L214) and [`agent-ddot`](https://github.com/DataDog/datadog-agent/blob/aimene.belfodil/dsec/build-rust-checks/Dockerfiles/agent-ddot/Dockerfile#L21) Dockerfiles after the recursive config `chmod`.
- Fix the Rust shared-check FFI macro to resolve types via `$crate::` instead of `core::`.

### Motivation

Rust shared-library checks are not built or shipped in agent packages today. This wires the existing rustchecks workspace into omnibus/Docker so checks listed in the manifest can be included in Linux release artifacts.

### Describe how you validated your changes (with example check [enabled](https://github.com/DataDog/datadog-agent/blob/aimene.belfodil/dsec/build-rust-checks/pkg/collector/sharedlibrary/rustchecks/shared_checks_manifest.yaml#L4))

#### CI image (omnibus build)

Built from CI job [1848750354](https://gitlab.ddbuild.io/DataDog/datadog-agent/-/jobs/1848750354) and ran the agent with:

```yaml
services:
  agent:
    image: registry.ddbuild.io/ci/datadog-agent/agent:v124269439-678e1467-7-amd64
    container_name: new-agent-with-rust-checks
    environment:
      DD_API_KEY: ***
      DD_HOSTNAME: aimene-test-ci-image
      DD_SITE: datad0g.com
      DD_APM_ENABLED: "true"
      DD_APM_NON_LOCAL_TRAFFIC: "true"
      DD_LOGS_ENABLED: "true"
      DD_DOGSTATSD_NON_LOCAL_TRAFFIC: "true"
      DD_LOG_LEVEL: "INFO"
      DD_SHARED_LIBRARY_CHECK_ENABLED: "true"
      DD_SHARED_LIBRARY_CHECK_LIBRARY_FOLDER_PATH: "/etc/datadog-agent/checks.d"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./conf.d:/etc/datadog-agent/conf.d:ro
```

`conf.d` contains `example.d/conf.yaml`:

```yaml
init_config:
instances:
  -
    min_collection_interval: 15
```

The example shared-library check ran successfully and events were received in [Event Explorer (`host:aimene-test-ci-image`)](https://dd.datad0g.com/event/explorer?query=host%3Aaimene-test-ci-image&agg_m=count&agg_m_source=base&agg_t=count&clustering_pattern_field_path=message&cols=&messageDisplay=expanded-lg&options=&refresh_mode=paused&sort=DESC&from_ts=1783634400000&to_ts=1783687020000&live=false).

#### Hacky dev image

Also tested locally with:

```bash
dda env dev run -- dda inv agent.hacky-dev-image-build --target-image agent-with-rust-checks
```

Same check config and expected results (example shared-library check runs and [events are received](https://dd.datad0g.com/event/explorer?query=host%3Aaimene-test-hacky-build&agg_m=count&agg_m_source=base&agg_t=count&clustering_pattern_field_path=message&cols=&messageDisplay=expanded-lg&options=&refresh_mode=paused&sort=DESC&from_ts=1783634400000&to_ts=1783687020000&live=false)).

### Actions

- [x] Check selection is driven by [`include_in_build`](https://github.com/DataDog/datadog-agent/blob/aimene.belfodil/dsec/build-rust-checks/pkg/collector/sharedlibrary/rustchecks/shared_checks_manifest.yaml) in the manifest (default `false`). **Before merge:** set `example` to `include_in_build: false` since **the example check should not ship**.

### Question/Response

#### Why Only Linux?

To keep the scope and risk minimal, this PR targets the Linux platform only.

#### Why shipped shared libs need chmod 0500 (according to existing code)?
Shared-library loader rejects libs (rust checks) unless the owner is trusted and group/others have no access ([`CheckOwnerAndPermissionsAreRestricted`](https://github.com/DataDog/datadog-agent/blob/main/pkg/util/filesystem/permission_check.go#L14-L22) → [`CheckRights`](https://github.com/DataDog/datadog-agent/blob/main/pkg/util/filesystem/rights_nix.go#L17-L34), called from [`SharedLibraryLoader.Open`](https://github.com/DataDog/datadog-agent/blob/main/pkg/collector/sharedlibrary/ffi/library_loader.go#L111)). Packaging steps that `chmod` config broadly would otherwise leave libs group/world-readable and unloadable.


Co-authored-by: aimene.belfodil <aimene.belfodil@datadoghq.com>

Author: aimenebelfodil

.github/CODEOWNERS

@@ -849,6 +849,7 @@
 /tasks/static_quality_gates/            @DataDog/agent-build
 /tasks/files_inventory.py               @DataDog/agent-build
 /tasks/rtloader.py                      @DataDog/agent-runtimes
+/tasks/rust_shared_checks.py            @DataDog/agent-build @DataDog/agent-runtimes
 /tasks/secret_generic_connector.py      @DataDog/agent-configuration
 /tasks/security_agent.py                @DataDog/agent-security
 /tasks/sbomgen.py                       @DataDog/agent-security

Dockerfiles/agent-ddot/Dockerfile

@@ -16,3 +16,6 @@ RUN chmod 755 -R /opt/entrypoints \
   && rm -rf /var/run && mkdir -p /var/run/s6 && mkdir -p /var/run/datadog && mkdir -p /opt/datadog-agent/run \
   && chown -R dd-agent:root /etc/datadog-agent/ /etc/s6/ /var/run/s6/ /var/log/datadog/ /var/run/datadog/ /opt/datadog-agent/run \
   && chmod g+r,g+w,g+X -R /etc/datadog-agent/ /etc/s6/ /var/run/s6/ /var/log/datadog/ /var/run/datadog/ /opt/datadog-agent/run
+
+# Set owner-only (0500) perms on built-in Rust shared-library checks (no-op when none are present).
+RUN find /etc/datadog-agent/checks.d -name 'libdatadog-agent-*.so' -exec chmod 0500 {} \;

Dockerfiles/agent/Dockerfile

@@ -210,6 +210,9 @@ RUN adduser --system --no-create-home --disabled-password --ingroup root dd-agen
   # copy it on OpenShift where containers run with a random UID in the root group.
   && chmod u=r,g=r,o= /etc/datadog-agent/private-action-runner/*
 
+# Set owner-only (0500) perms on built-in Rust shared-library checks (no-op when none are present).
+RUN find /etc/datadog-agent/checks.d -name 'libdatadog-agent-*.so' -exec chmod 0500 {} \;
+
 # Check that the UID of dd-agent is still 100.
 #
 # The exact numeric value of the UID of the dd-agent user shouldn’t matter.

omnibus/config/software/datadog-agent-finalize.rb

@@ -94,6 +94,14 @@
             mkdir "#{output_config_dir}/etc/datadog-agent/checks.d"
             mkdir "/var/log/datadog"
 
+            # Move the built-in shared-library checks (built under install_dir) into the
+            # package's checks.d and set owner-only permissions on them.
+            Dir.glob("#{install_dir}/etc/datadog-agent/checks.d/libdatadog-agent-*.so").each do |lib|
+              dest = "#{output_config_dir}/etc/datadog-agent/checks.d/#{File.basename(lib)}"
+              move lib, dest, :force => true
+              command "chmod 0500 #{dest}"
+            end
+
             # Process manager config directory (read-only, under install dir)
             mkdir "#{install_dir}/processes.d"
 

omnibus/config/software/datadog-agent.rb

@@ -109,6 +109,13 @@
   else
     conf_dir = "#{install_dir}/etc/datadog-agent"
   end
+
+  # Stage Rust shared-library checks (Linux only).
+  if linux_target?
+    command "dda inv -- -e rust-shared-checks.build --checks-d-dir=\"#{conf_dir}/checks.d\"",
+      env: env,
+      :live_stream => Omnibus.logger.live_stream(:info)
+  end
   # TODO(agent-build): sort out the use of bin/agen/dist/conf.d
   # dda inv agent.build  leaves many files in bin/agen/dist/conf.d
   # Now we place them into the pacakge via the //packages/agent/product:post_build_install

pkg/collector/sharedlibrary/rustchecks/core/src/ffi.rs

@@ -8,7 +8,7 @@ macro_rules! generate_ffi {
             check_id_cstr: *const std::ffi::c_char,
             init_config_cstr: *const std::ffi::c_char,
             instance_config_cstr: *const std::ffi::c_char,
-            aggregator_ptr: *const core::Aggregator,
+            aggregator_ptr: *const $crate::Aggregator,
             error_handler: *mut *mut std::ffi::c_char,
         ) {
             if let Err(e) = create_and_run_check(
@@ -30,22 +30,22 @@ macro_rules! generate_ffi {
             check_id_cstr: *const std::ffi::c_char,
             init_config_cstr: *const std::ffi::c_char,
             instance_config_cstr: *const std::ffi::c_char,
-            aggregator_ptr: *const core::Aggregator,
+            aggregator_ptr: *const $crate::Aggregator,
         ) -> Result<(), Box<dyn std::error::Error>> {
             // convert C args to Rust structs
-            let check_id = core::to_rust_string(check_id_cstr)?;
+            let check_id = $crate::to_rust_string(check_id_cstr)?;
 
-            let init_config_str = core::to_rust_string(init_config_cstr)?;
-            let init_config = core::Config::from_str(&init_config_str)?;
+            let init_config_str = $crate::to_rust_string(init_config_cstr)?;
+            let init_config = $crate::Config::from_str(&init_config_str)?;
 
-            let instance_config_str = core::to_rust_string(instance_config_cstr)?;
-            let instance_config = core::Config::from_str(&instance_config_str)?;
+            let instance_config_str = $crate::to_rust_string(instance_config_cstr)?;
+            let instance_config = $crate::Config::from_str(&instance_config_str)?;
 
-            let aggregator = core::Aggregator::from_ptr(aggregator_ptr);
+            let aggregator = $crate::Aggregator::from_ptr(aggregator_ptr);
 
             // create the check instance
             let agent_check =
-                core::AgentCheck::new(check_id, init_config, instance_config, aggregator);
+                $crate::AgentCheck::new(check_id, init_config, instance_config, aggregator);
 
             // run the custom implementation
             $check_function(&agent_check)?;

pkg/collector/sharedlibrary/rustchecks/shared_checks_manifest.yaml

@@ -0,0 +1,6 @@
+checks:
+  - id: example
+    crate: example
+    include_in_build: false
+    platforms:
+      - linux

releasenotes/notes/ship-built-in-rust-check-in-linux-agent-packages-b6caf0effcca3ec2.yaml

@@ -0,0 +1,6 @@
+---
+enhancements:
+  - |
+    The Linux Agent packages can now ship built-in Rust-based checks as shared
+    libraries under ``/etc/datadog-agent/checks.d``, restricted to owner-only access.
+    No such check is shipped for now; this only adds the packaging support.

tasks/__init__.py

@@ -75,6 +75,7 @@
     release,
     renovate,
     rtloader,
+    rust_shared_checks,
     sbomgen,
     schema,
     secret_generic_connector,
@@ -252,6 +253,7 @@
 ns.add_collection(release)
 ns.add_collection(renovate)
 ns.add_collection(rtloader)
+ns.add_collection(rust_shared_checks)
 ns.add_collection(system_probe)
 ns.add_collection(process_agent)
 ns.add_collection(privateactionrunner)

tasks/agent.py

@@ -448,6 +448,19 @@ def hacky_dev_image_build(
             f'perl -0777 -pe \'s|{extracted_python_dir}(/opt/datadog-agent/embedded/lib/python\\d+\\.\\d+/../..)|substr $1."\\0"x length$&,0,length$&|e or die "pattern not found"\' -i dev/lib/libdatadog-agent-three.so'
         )
 
+    copy_checks_d = ""
+    copy_checks_d_final = ""
+    if sys.platform.startswith("linux"):
+        from tasks.rust_shared_checks import build as rust_shared_checks_build
+
+        checks_d_staging = "bin/agent/dist/checks.d"
+        rust_shared_checks_build(ctx, checks_d_dir=checks_d_staging)
+        if os.path.isdir(checks_d_staging) and any(
+            f.startswith("libdatadog-agent-") for f in os.listdir(checks_d_staging)
+        ):
+            copy_checks_d = f"COPY {checks_d_staging} /etc/datadog-agent/checks.d\n"
+            copy_checks_d_final = "COPY --from=bin /etc/datadog-agent/checks.d /etc/datadog-agent/checks.d\n"
+
     copy_extra_agents = ""
     if security_agent:
         from tasks.security_agent import build as security_agent_build
@@ -527,6 +540,7 @@ def hacky_dev_image_build(
 
 COPY bin/agent/agent                            /opt/datadog-agent/bin/agent/agent
 COPY bin/agent/dist/conf.d                      /etc/datadog-agent/conf.d
+{copy_checks_d}
 COPY dev/lib/libdatadog-agent-rtloader.so.0.1.0 /opt/datadog-agent/embedded/lib/libdatadog-agent-rtloader.so.0.1.0
 COPY dev/lib/libdatadog-agent-three.so          /opt/datadog-agent/embedded/lib/libdatadog-agent-three.so
 {copy_ebpf_assets}
@@ -564,6 +578,7 @@ def hacky_dev_image_build(
 COPY --from=bin /opt/datadog-agent/embedded/lib/libdatadog-agent-rtloader.so.0.1.0 /opt/datadog-agent/embedded/lib/libdatadog-agent-rtloader.so.0.1.0
 COPY --from=bin /opt/datadog-agent/embedded/lib/libdatadog-agent-three.so          /opt/datadog-agent/embedded/lib/libdatadog-agent-three.so
 COPY --from=bin /etc/datadog-agent/conf.d /etc/datadog-agent/conf.d
+{copy_checks_d_final}
 {copy_extra_agents}
 {copy_ebpf_assets_final}
 RUN agent          completion bash > /usr/share/bash-completion/completions/agent