Skip to content

Commit d3b8c10

Browse files
chouetzchouetz
committed
feat(renovate): add coverage check for Bazel http_archive deps
Adds a CI lint that fails any PR introducing or modifying an `http_archive(...)` block in `deps/repos.MODULE.bazel` without a corresponding `customManager` entry in `renovate.json`. Without this check, native deps (openssl, krb5, libxml2, …) drift silently behind upstream — Renovate's `bazel-module` manager only covers `bazel_dep(...)` declarations from the BCR registry, not the `http_archive(...)` calls used for natives. None of the 29 such deps in the repo are tracked today. Pieces: - `tasks/renovate.py::check_bazel_coverage` — invoke task that diffs the set of `http_archive(name=...)` entries in the bazel module file against `customManagers[].depNameTemplate` in `renovate.json`, minus an explicit allowlist. Writes a markdown report to `$GITHUB_STEP_SUMMARY` on failure. - `deps/.renovate-untracked.json` — typed allowlist with non-empty rationale strings. Currently lists all 29 deps as pending tracking; follow-up PRs will move each entry from this file into a customManager in `renovate.json`. - `.github/workflows/validate-renovate-deps.yml` — runs the task on PRs touching `deps/`, `renovate.json`, or the task itself. Run locally with `dda inv -- renovate.check-bazel-coverage`. fix(renovate): replace regex with block parser for http_archive names The previous regex required `name` to be the first argument after the opening paren, silently skipping archives where a comment or another keyword preceded it. The new parser tracks parenthesis depth and string boundaries, then searches for `name = "..."` anywhere within each block. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> fix(renovate): scan all *.MODULE.bazel files in deps/, not just repos.MODULE.bazel Previously only deps/repos.MODULE.bazel was scanned, so http_archive calls in per-dep MODULE.bazel files (e.g. deps/cpython/, deps/curl/) were invisible to the coverage check. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> fix(renovate): extend coverage check to http_file in addition to http_archive deps/repos.MODULE.bazel already uses http_file (e.g. sqlite3_license, cacerts). These were invisible to the check because only http_archive was parsed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> feat(renovate): add bazel run //tasks:check_renovate_bazel_coverage target Adds a py_binary so the coverage check can be invoked via Bazel without needing dda/invoke. BUILD_WORKSPACE_DIRECTORY (set by `bazel run`) is used to locate the repo root; direct `python tasks/renovate.py` invocation falls back to the file's parent path. Updates the validate-renovate-deps.yml workflow to use the new Bazel target via the existing bazel-cache action, removing the dda dependency from that workflow. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> code review
1 parent b1af316 commit d3b8c10

5 files changed

Lines changed: 400 additions & 0 deletions

File tree

‎.github/workflows/validate-renovate-deps.yml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
name: "Validate Renovate coverage of Bazel native deps"
3+
on:
4+
pull_request:
5+
paths:
6+
- "deps/**"
7+
- "renovate.json"
8+
- "tasks/renovate.py"
9+
- "tasks/BUILD.bazel"
10+
- ".github/workflows/validate-renovate-deps.yml"
11+
12+
permissions: {}
13+
14+
jobs:
15+
check_coverage:
16+
runs-on: ubuntu-latest
17+
steps:
18+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
19+
- uses: ./.github/actions/bazel-cache
20+
- name: Verify every http_archive/http_file in deps/ has Renovate coverage
21+
run: bazel run //tasks:check_renovate_bazel_coverage

‎deps/.renovate-untracked.json‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
{
2+
"_comment": "Bazel http_archive and http_file deps in any deps/**/*.MODULE.bazel file that intentionally have no Renovate customManager. Every entry MUST include a non-empty rationale. The check is enforced by tasks/check_renovate_bazel_coverage.py and the validate-renovate-deps.yml workflow. As deps move from this allowlist to a customManager in renovate.json, remove the entry here in the same PR.",
3+
"intentionally_untracked": {
4+
"xz": "Pending tracking in Phase 2a (github-releases batch).",
5+
"zlib": "Pending tracking in Phase 2a (github-releases batch).",
6+
"openssl": "Pending tracking in Phase 2a (github-releases batch). Blocked by openssl drift cleanup (AIX 3.5.5, MSI 3.5).",
7+
"libffi": "Pending tracking in Phase 2a (github-releases batch).",
8+
"pcre2": "Pending tracking in Phase 2a (github-releases batch).",
9+
"util-linux": "Pending tracking in Phase 2a (github-releases batch).",
10+
"libseccomp": "Pending tracking in Phase 2a (github-releases batch).",
11+
"patchelf": "Pending tracking in Phase 2a (github-releases batch).",
12+
"nghttp2": "Pending tracking in Phase 2a (github-releases batch).",
13+
"popt": "Pending tracking in Phase 2a (github-releases batch).",
14+
"libyaml": "Pending tracking in Phase 2a (github-releases batch).",
15+
"zstd": "Pending tracking in Phase 2a (github-releases batch).",
16+
"lua": "Pending tracking in Phase 2a (github-releases batch).",
17+
"xmlsec": "Pending tracking in Phase 2a (github-releases batch).",
18+
"krb5": "Pending tracking in Phase 2b (tricky-tag batch — tag krb5-X.Y.Z-final differs from tarball).",
19+
"rpm": "Pending tracking in Phase 2b (tricky-tag batch — strip_prefix is rpm-rpm-X.Y.Z-release).",
20+
"systemd": "Pending tracking in Phase 2b (tricky-tag batch — single-integer versioning, loose).",
21+
"libsepol": "Pending tracking in Phase 2b (SELinux monorepo).",
22+
"libselinux": "Pending tracking in Phase 2b (SELinux monorepo).",
23+
"openssl_fips": "Pending tracking in Phase 2b. Will be added with `enabled: false` — pinned to 3.0.9 for FIPS 140-2 validation, must NOT auto-update.",
24+
"libxml2": "Pending tracking in Phase 2c (GitLab-tags batch).",
25+
"libxslt": "Pending tracking in Phase 2c (GitLab-tags batch).",
26+
"dbus": "Pending tracking in Phase 2c (GitLab-tags batch).",
27+
"sqlite3": "Hard: version is split across a tuple (sqlite_ver = (\"3\", \"53\", \"00\")) and embeds a release-year directory in the URL. Needs a refactor to a single literal before a Renovate regex can target it cleanly. Phase 3.",
28+
"bzip2": "Hard: sourceware.org tarballs; no native Renovate datasource. Needs a customDatasource. Phase 3.",
29+
"attr": "Hard: hosted on Savannah (savannah.nongnu.org); no native Renovate datasource. Needs a customDatasource. Phase 3.",
30+
"gpg-error": "Hard: hosted on gnupg.org FTP; no native Renovate datasource. Needs a customDatasource. Phase 3.",
31+
"gcrypt": "Hard: hosted on gnupg.org FTP; no native Renovate datasource. Needs a customDatasource. Phase 3.",
32+
"unixodbc": "Hard: hosted on unixodbc.org; no native Renovate datasource. Needs a customDatasource. Phase 3.",
33+
"acl": "Pending tracking. Hosted on Savannah (download.savannah.nongnu.org); needs a customDatasource. Phase 3.",
34+
"curl": "Pending tracking in Phase 2a (github-releases batch). Tarball is on curl.haxx.se but releases are tagged on github.com/curl/curl.",
35+
"cpython": "Pending tracking in Phase 2a (github-releases batch). Released on python.org; corresponding GitHub tags exist at github.com/python/cpython.",
36+
"freetds": "Pending tracking in Phase 2a (github-releases batch). Tarballs on freetds.org; releases tagged at github.com/FreeTDS/freetds.",
37+
"libpcap": "Pending tracking in Phase 2a (github-releases batch). Tarball on tcpdump.org; releases tagged at github.com/the-tcpdump-group/libpcap.",
38+
"nfsiostat": "Pending tracking in Phase 2c. Hosted on kernel.org mirrors (no GitHub mirror with reliable tags). Needs a customDatasource.",
39+
"openscap": "Pending tracking in Phase 2a (github-releases batch). Released at github.com/OpenSCAP/openscap.",
40+
"setuptools": "Pending tracking in Phase 2a (github-releases batch). Released at github.com/pypa/setuptools.",
41+
"compile_policy_x86_64": "Pending tracking in Phase 2a. DataDog/dd-policy-engine GitHub release; will be tracked as one bundle with the arm64 and windows variants since they share VERSION.",
42+
"compile_policy_arm64": "Pending tracking in Phase 2a. Shares VERSION with compile_policy_x86_64; tracked together.",
43+
"compile_policy_windows_x86_64": "Pending tracking in Phase 2a. Shares VERSION with compile_policy_x86_64; tracked together.",
44+
"gstatus_binary": "Pending tracking in Phase 2a (github-releases batch). Released at github.com/gluster/gstatus.",
45+
"msodbcsql18_deb_amd64": "Pending tracking. Microsoft .deb package; no native datasource for packages.microsoft.com. Both arches share VERSION and will be tracked as one bundle when a customDatasource is added. Phase 3.",
46+
"msodbcsql18_deb_arm64": "Pending tracking. Shares VERSION with msodbcsql18_deb_amd64; tracked together. Phase 3.",
47+
"snmp_traps": "Intentionally untracked. Datadog-internal SNMP traps DB published to dd-agent-omnibus S3; no public upstream to watch. Bumped manually when the traps DB is regenerated.",
48+
"cacert_pem": "Intentionally untracked. Bumped via a separate Datadog Synthetics-driven workflow (see deps/cacerts/cacerts.MODULE.bazel) that alerts team-agent-build on header changes; auto-PRs from Renovate would race that flow.",
49+
"cacert_license": "Intentionally untracked. Mozilla MPL-2.0 license text accompanying cacert_pem; updated together with cacert_pem via the same out-of-band flow.",
50+
"sqlite3_license": "Intentionally untracked here — version is pinned via the shared sqlite_ver tuple in deps/repos.MODULE.bazel and will be tracked together with sqlite3 once that refactor lands (see sqlite3 entry).",
51+
"sqlite_win": "Intentionally untracked here. Windows-only SQLite amalgamation; pinned to the same release as sqlite3 (manually kept in sync, see TODO in deps/cpython/cpython.MODULE.bazel). Will be folded in with sqlite3 in Phase 3.",
52+
"bzip2_win": "Pending tracking. Loop-emitted Windows variant in deps/cpython/cpython.MODULE.bazel; version pinned via the python_src_deps dict alongside the base bzip2. Will be tracked together with bzip2 (Phase 3 customDatasource).",
53+
"mpdecimal_win": "Pending tracking. Loop-emitted Windows variant in deps/cpython/cpython.MODULE.bazel; version pinned via the python_src_deps dict. No upstream `mpdecimal` dep exists today — needs its own customManager.",
54+
"xz_win": "Pending tracking. Loop-emitted Windows variant in deps/cpython/cpython.MODULE.bazel; version pinned via the python_src_deps dict alongside the base xz. Will be tracked together with xz (Phase 2a).",
55+
"zlib_win": "Pending tracking. Loop-emitted Windows variant in deps/cpython/cpython.MODULE.bazel; version pinned via the python_src_deps dict alongside the base zlib. Will be tracked together with zlib (Phase 2a).",
56+
"libffi_win": "Pending tracking. Loop-emitted Windows variant in deps/cpython/cpython.MODULE.bazel; version pinned via the python_bin_deps dict. Note: pinned to 3.4.4 (cpython-bin-deps), independent of the base libffi version — must be tracked separately.",
57+
"tcltk_win": "Pending tracking. Loop-emitted Windows variant in deps/cpython/cpython.MODULE.bazel; version pinned via the python_bin_deps dict. No upstream `tcltk` dep — needs its own customManager (cpython-bin-deps tags)."
58+
}
59+
}

‎tasks/BUILD.bazel‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
load("@rules_python//python:py_binary.bzl", "py_binary")
12
load("@rules_python//python:py_library.bzl", "py_library")
23

34
package(default_visibility = ["//visibility:private"])
@@ -13,3 +14,14 @@ py_library(
1314
)
1415

1516
exports_files(["core_checks.py"])
17+
18+
# No `data = [...]` for renovate.json / deps/ / .renovate-untracked.json on
19+
# purpose: the script resolves them via $BUILD_WORKSPACE_DIRECTORY (set by
20+
# `bazel run`) so it reads the live workspace, not the bazel sandbox. Adding
21+
# them as data deps would silently shadow the real files.
22+
py_binary(
23+
name = "check_renovate_bazel_coverage",
24+
srcs = ["renovate.py"],
25+
main = "renovate.py",
26+
deps = ["@py_dev_requirements//invoke"],
27+
)

‎tasks/__init__.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,7 @@
6969
python_version,
7070
quality_gates,
7171
release,
72+
renovate,
7273
rtloader,
7374
sbomgen,
7475
schema,
@@ -240,6 +241,7 @@
240241
ns.add_collection(setup)
241242
ns.add_collection(systray)
242243
ns.add_collection(release)
244+
ns.add_collection(renovate)
243245
ns.add_collection(rtloader)
244246
ns.add_collection(system_probe)
245247
ns.add_collection(process_agent)

0 commit comments

Comments
 (0)