Finish collapsing pkg/procmgr/rust onto Bazel - #53629
Finish collapsing pkg/procmgr/rust onto Bazel#53629gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit into
pkg/procmgr/rust onto Bazel#53629Conversation
rdesgroppes
added
changelog/no-changelog
github-actions
Bot
added
the
short review
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fab391dc42
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Gitlab CI Configuration ChangesRemoved Jobs
Changes Summary
ℹ️ Diff available in the job log. |
### What does this PR do? Remove the `procmgr_rust_tests` GitLab job and its entry from `JOBOWNERS`, and add the `rust_test` target this uncovered was missing for `dd-procmgr`. ### Motivation Investigating why `rawzip` bump (#53563) broke CI found `procmgr_rust_tests` failing because Cargo's internal registry proxy had not yet mirrored the new version, not because of anything wrong with the bump. The same registry-lag failure independently hit `uuid` bump (#53552), a dependency this crate uses directly. Bazel benefits from ADMS' centralized downloader mirror config (`.adms/bazel/adms.mirror.cfg`), while plain `cargo` instead relies on `.rust_internal_registry`'s bespoke GitLab CI redirect (`.adms/rust/gitlab.yaml`), which is what lagged, so `bazel:test:*` passed both times regardless. This is safe to rely on because `pkg/procmgr/rust/BUILD.bazel` has a `rust_test` target (since #46880) and `bazel:test:linux-amd64`, `:linux-arm64`, etc. already run every target in that file, with the same implied `rustfmt`/`clippy` `-Dwarnings` Cargo passes explicitly, on every pipeline. The one gap, caught by review on #53629, was that `cargo test --all-targets` also exercised `dd-procmgr.rs`'s own 15 `#[test]` functions via `--bins`, which no existing `rust_test` covered; this closes it with a `dd-procmgr_test` target wrapping that binary crate directly. The Cargo job also runs far more than its own path filter suggests: `.on_main_or_release_branch` matches on branch name alone, ahead of the path filter, so it fires on every push to `main`, ~900 times in the last 30 days for an average of ~140s per run. As #53615 already dropped the `extend` from the package-build jobs, removing it here finishes collapsing `pkg/procmgr/rust` onto Bazel as its only toolchain, one source of truth instead of two that are doomed to drift apart. ### Describe how you validated your changes Confirmed parity before removing anything: no test is gated by the `test-helpers` feature beyond `cfg(test)`, and `rules_rust`'s `clippy`/`rustfmt` aspects don't propagate through dependencies, so they only check whatever `//...` names directly, which is every library, binary, and test as its own target. Ran `bazel test //pkg/procmgr/rust:dd-procmgr_test --test_output=all`, confirming all 15 tests run and pass. ### Additional Notes Bazel's coverage is now a strict superset: it lints the plain and `test-helpers`-enabled lib variants as separate targets and tests the `dd-procmgr` CLI binary directly, whereas Cargo's single invocation only ever builds one feature resolution at a time.
fab391d to
a631cb0
Compare
rdesgroppes
changed the title
procmgr_rust_tests jobpkg/procmgr/rust onto Bazel
github-actions
Bot
added
medium review
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a631cb0e8b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| - cargo fmt --check | ||
| - cargo clippy --all-targets --features test-helpers -- -D warnings |
There was a problem hiding this comment.
When this job is deleted, procmgr changes no longer run Rust formatting or clippy checks: I checked .gitlab/build/bazel/test.yml, and the replacement Bazel jobs invoke bazel test ... without --config=ci or --config=lint; the only repo config that wires rustfmt/clippy is .bazelrc's common:lint, and a repo-wide search found no CI command enabling it. In PR/main pipelines that touch pkg/procmgr/rust, formatting or clippy regressions will now pass CI, unlike before these lines were removed.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
This isn't quite right: --config=ci/--config=lint aren't literal flags in .gitlab/build/bazel/test.yml, they're injected dynamically. Every bazel invocation in CI goes through the Bazelisk wrapper, which adds --config=ci whenever $CI is set and $GITHUB_ACTIONS isn't (exactly GitLab CI's environment):
.bazelrc then expands that:
Confirmed directly against this branch:
$ CI=1 tools/bazel test --announce_rc --nobuild //pkg/procmgr/rust:dd-procmgr_test
INFO: Found applicable config definition common:ci in .bazelrc: --config=adms --config=lint --noexperimental_convenience_symlinks --remote_download_regex=.*/test.xml$
INFO: Found applicable config definition common:lint in .bazelrc: --aspects=@rules_rust//rust:defs.bzl%rust_clippy_aspect --output_groups=+clippy_checks --aspects=@rules_rust//rust:defs.bzl%rustfmt_aspect --output_groups=+rustfmt_checks
So bazel:test:* does run clippy/rustfmt for pkg/procmgr/rust, same as it always has.
Files inventory check summaryFile checks results against ancestor 695123cb: Results for datadog-agent_7.83.0~devel.git.89.a631cb0.pipeline.124499843-1_amd64.deb:No change detected |
Static quality checks✅ Please find below the results from static quality gates 33 successful checks with minimal change (< 2 KiB)
|
Regression DetectorRegression Detector ResultsMetrics dashboard Baseline: 1758cec Optimization Goals: ✅ No significant changes detected
|
| perf | experiment | goal | Δ mean % | Δ mean % CI | trials | links |
|---|---|---|---|---|---|---|
| ➖ | quality_gate_idle | memory utilization | +0.43 | [+0.38, +0.48] | 1 | Logs bounds checks dashboard |
| ➖ | quality_gate_metrics_logs | memory utilization | +0.37 | [+0.12, +0.62] | 1 | Logs bounds checks dashboard |
| ➖ | quality_gate_security_mean_fs_load | memory utilization | +0.00 | [-0.04, +0.04] | 1 | Logs bounds checks dashboard |
| ➖ | quality_gate_security_idle | memory utilization | -0.04 | [-0.10, +0.02] | 1 | Logs bounds checks dashboard |
| ➖ | quality_gate_idle_all_features | memory utilization | -0.31 | [-0.38, -0.24] | 1 | Logs bounds checks dashboard |
| ➖ | quality_gate_security_no_fs_load | memory utilization | -0.42 | [-0.51, -0.32] | 1 | Logs bounds checks dashboard |
| ➖ | quality_gate_logs | % cpu utilization | -1.05 | [-2.01, -0.09] | 1 | Logs bounds checks dashboard |
Bounds Checks: ✅ Passed
| perf | experiment | bounds_check_name | replicates_passed | observed_value | links |
|---|---|---|---|---|---|
| ✅ | quality_gate_idle | intake_connections | 10/10 | 3 ≤ 4 | bounds checks dashboard |
| ✅ | quality_gate_idle | memory_usage | 10/10 | 147.11MiB ≤ 154MiB | bounds checks dashboard |
| ✅ | quality_gate_idle | total_bytes_received | 10/10 | 727.42KiB ≤ 819.20KiB | bounds checks dashboard |
| ✅ | quality_gate_idle_all_features | intake_connections | 10/10 | 3 ≤ 4 | bounds checks dashboard |
| ✅ | quality_gate_idle_all_features | memory_usage | 10/10 | 490.82MiB ≤ 495MiB | bounds checks dashboard |
| ✅ | quality_gate_idle_all_features | total_bytes_received | 10/10 | 1.12MiB ≤ 1.25MiB | bounds checks dashboard |
| ✅ | quality_gate_logs | intake_connections | 10/10 | 3 ≤ 6 | bounds checks dashboard |
| ✅ | quality_gate_logs | memory_usage | 10/10 | 184.90MiB ≤ 195MiB | bounds checks dashboard |
| ✅ | quality_gate_logs | missed_bytes | 10/10 | 0B = 0B | bounds checks dashboard |
| ✅ | quality_gate_logs | total_bytes_received | 10/10 | 264.06MiB ≤ 292MiB | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | cpu_usage | 10/10 | 354.48 ≤ 2000 | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | intake_connections | 10/10 | 4 ≤ 6 | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | memory_usage | 10/10 | 401.19MiB ≤ 430MiB | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | missed_bytes | 10/10 | 0B = 0B | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | total_bytes_received | 10/10 | 0.93GiB ≤ 1.04GiB | bounds checks dashboard |
| ✅ | quality_gate_security_idle | cpu_usage | 10/10 | 29.59 ≤ 40 | bounds checks dashboard |
| ✅ | quality_gate_security_idle | memory_usage | 10/10 | 305.10MiB ≤ 330MiB | bounds checks dashboard |
| ✅ | quality_gate_security_mean_fs_load | cpu_usage | 10/10 | 64.03 ≤ 80 | bounds checks dashboard |
| ✅ | quality_gate_security_mean_fs_load | memory_usage | 10/10 | 277.29MiB ≤ 310MiB | bounds checks dashboard |
| ✅ | quality_gate_security_no_fs_load | cpu_usage | 10/10 | 23.75 ≤ 40 | bounds checks dashboard |
| ✅ | quality_gate_security_no_fs_load | memory_usage | 10/10 | 285.41MiB ≤ 320MiB | bounds checks dashboard |
Explanation
Confidence level: 90.00%
Effect size tolerance: |Δ mean %| ≥ 5.00%
Performance changes are noted in the perf column of each table:
- ✅ = significantly better comparison variant performance
- ❌ = significantly worse comparison variant performance
- ➖ = no significant change in performance
A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".
For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:
-
Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
-
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
-
Its configuration does not mark it "erratic".
Replicate Execution Details
We run multiple replicates for each experiment/variant. However, we allow replicates to be automatically retried if there are any failures, up to 8 times, at which point the replicate is marked dead and we are unable to run analysis for the entire experiment. We call each of these attempts at running replicates a replicate execution. This section lists all replicate executions that failed due to the target crashing or being oom killed.
Note: In the below tables we bucket failures by experiment, variant, and failure type. For each of these buckets we list out the replicate indexes that failed with an annotation signifying how many times said replicate failed with the given failure mode. In the below example the baseline variant of the experiment named experiment_with_failures had two replicates that failed by oom kills. Replicate 0, which failed 8 executions, and replicate 1 which failed 6 executions, all with the same failure mode.
| Experiment | Variant | Replicates | Failure | Logs | Debug Dashboard |
|---|---|---|---|---|---|
| experiment_with_failures | baseline | 0 (x8) 1 (x6) | Oom killed | Debug Dashboard |
The debug dashboard links will take you to a debugging dashboard specifically designed to investigate replicate execution failures.
❌ Retried Profiling Replicate Execution Failures (ddprof)
Note: Profiling replicas may still be executing. See the debug dashboard for up to date status.
| Experiment | Variant | Replicates | Failure | Debug Dashboard |
|---|---|---|---|---|
| quality_gate_idle | comparison | 10 | Oom killed | Debug Dashboard |
| quality_gate_idle_all_features | comparison | 10 | Oom killed | Debug Dashboard |
| quality_gate_logs | baseline | 10 | Oom killed | Debug Dashboard |
| quality_gate_logs | comparison | 10 | Oom killed | Debug Dashboard |
| quality_gate_metrics_logs | baseline | 10 | Oom killed | Debug Dashboard |
| quality_gate_metrics_logs | comparison | 10 | Oom killed | Debug Dashboard |
| quality_gate_security_idle | baseline | 10 | Crashed (exit code: 134) | Debug Dashboard |
| quality_gate_security_idle | comparison | 10 | Oom killed | Debug Dashboard |
| quality_gate_security_mean_fs_load | baseline | 10 | Oom killed | Debug Dashboard |
| quality_gate_security_mean_fs_load | comparison | 10 | Crashed (exit code: 134) | Debug Dashboard |
| quality_gate_security_no_fs_load | baseline | 10 | Crashed (exit code: 134) | Debug Dashboard |
CI Pass/Fail Decision
✅ Passed. All Quality Gates passed.
- quality_gate_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_logs, bounds check total_bytes_received: 10/10 replicas passed. Gate passed.
- quality_gate_logs, bounds check missed_bytes: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check total_bytes_received: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check cpu_usage: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check missed_bytes: 10/10 replicas passed. Gate passed.
- quality_gate_security_mean_fs_load, bounds check cpu_usage: 10/10 replicas passed. Gate passed.
- quality_gate_security_mean_fs_load, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_idle, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_idle, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_idle, bounds check total_bytes_received: 10/10 replicas passed. Gate passed.
- quality_gate_security_no_fs_load, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_security_no_fs_load, bounds check cpu_usage: 10/10 replicas passed. Gate passed.
- quality_gate_idle_all_features, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_idle_all_features, bounds check total_bytes_received: 10/10 replicas passed. Gate passed.
- quality_gate_idle_all_features, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_security_idle, bounds check cpu_usage: 10/10 replicas passed. Gate passed.
- quality_gate_security_idle, bounds check memory_usage: 10/10 replicas passed. Gate passed.
596b9de
into
main
3 participants
What does this PR do?
Remove the
procmgr_rust_testsGitLab job and its entry fromJOBOWNERS, and add the missingrust_testtarget fordd-procmgrthat this uncovered.Motivation
Investigating why
rawzipbump (#53563) broke CI foundprocmgr_rust_testsfailing because Cargo's specific registry proxy had not yet mirrored the new version, not because of anything wrong with the bump.The same registry-lag failure independently hit
uuidbump (#53552), a dependency this crate uses directly.Bazel benefits from ADMS' centralized downloader mirror config (.adms/bazel/adms.mirror.cfg), while plain
cargoinstead relies on.rust_internal_registry's bespoke GitLab CI redirect (.adms/rust/gitlab.yaml), which is what lagged, sobazel:test:*passed both times regardless.This is safe to rely on because
pkg/procmgr/rust/BUILD.bazelhas arust_testtarget (since #46880) andbazel:test:linux-amd64,:linux-arm64, etc. already run every target in that file, with the same impliedrustfmt/clippy-DwarningsCargo passes explicitly, on every pipeline.The one gap was that
cargo test --all-targetsalso exerciseddd-procmgr.rs's own 15#[test]functions via--bins, which no existingrust_testcovered.This therefore closes it with a
dd-procmgr_testtarget wrapping that binary crate directly, while making sure it runs on all supported platforms.The Cargo job also runs far more than its own path filter suggests:
.on_main_or_release_branchmatches on branch name alone, ahead of the path filter, so it fires on every push tomain, ~900 times in the last 30 days for an average of ~2min per run.As #53615 already dropped the
extendfrom the package-build jobs, removing it here finishes collapsingpkg/procmgr/rustonto Bazel as its only toolchain, one source of truth instead of two that are doomed to drift apart.Describe how you validated your changes
Confirmed parity before removing anything: no test is gated by the
test-helpersfeature beyondcfg(test), andrules_rust'sclippy/rustfmtaspects don't propagate through dependencies, so they only check whatever//...names directly, which is every library, binary, and test as its own target.Ran
bazel test //pkg/procmgr/rust:dd-procmgr_test --test_output=all: all 15 tests run and pass.Additional Notes
Bazel's coverage is now a strict superset: it lints the plain and
test-helpers-enabled lib variants as separate targets and tests thedd-procmgrCLI binary directly, whereas Cargo's single invocation only ever builds one feature resolution at a time.