chore(security): cool down npm installs and freeze the dev lockfile - #8629
Closed
BridgeAR wants to merge 41 commits into
Closed
chore(security): cool down npm installs and freeze the dev lockfile#8629BridgeAR wants to merge 41 commits into
BridgeAR wants to merge 41 commits into
Conversation
Yarn 1.x is unmaintained. The root install in CI already runs bun (`.github/actions/install/action.yml`); audit, size-report, flakiness, release, and a few helper actions still shelled out to yarn. Make the contract explicit so contributor installs match what CI runs: a root `bunfig.toml` that forces the text lockfile and hoisted linker, the lockfile committed (removed from `.gitignore`), and the install action's cache step renamed (`yarn-cache` -> `install-cache`) and keyed on `bun.lock`. `audit.yml` swaps both `yarn audit` calls (root and vendor/) to `bun audit`. The advisory set may differ; review and pin a baseline if anything trips.
The per-plugin sandbox install relied on `workspaces.nohoist: ['**/**']` (yarn 1) plus `installConfig.hoistingLimits = 'workspaces'` (yarn 2/3 for aerospike) to keep each `versions/<plugin>@<ver>/` sandbox isolated from the others. Bun's `--linker=isolated` is the structural equivalent and is now pinned in `versions/bunfig.toml` instead of re-declared in every synthesized `package.json`. Pre-resolving each declared range to its highest published version before writing the synthesized `package.json` keeps matrix coverage equivalent to yarn 1's prior behaviour. Yarn 1 picked the highest matching version on each install; bun picks the lowest. Without pre-resolution the raw-suffix sandboxes (`pino@4`, `mongodb@5`, `pino@>=6.8.0`, ...) collapse onto the same versions as the coerced ones (`pino@4.0.0`, `mongodb@5.0.0`, ...). The helper calls `bun pm view <pkg>@<range> version --json`, caches per process, and falls back to install-time resolution with a warning on registry failure. `scripts/test/install-plugin-modules.spec.js` spawns the install script with yarn removed from `$PATH` and deep-equals each pino sandbox's resolved version against its declared range. `.github/workflows/install-smoke.yml` runs that test plus a cheap pino plugin spec on every PR.
Three coordinated pieces: 1. 17 workflow/action files swap every `yarn run <script>` for `npm run <script>`, and `integration-tests/esbuild/package.json`'s `link` script swaps `yarn link` for `npm link`. The `yarn config set ignore-engines true` steps and per-line `--ignore-engines` flags go with them — yarn 1 enforced `engines.node` strictly, bun does not. 2. `scripts/check_licenses.js` walks `bun.lock` and `vendor/package-lock.json` directly instead of shelling out to `yarn list` + `npm list`. The lockfiles record the union of installed packages across platforms; `npm ls --omit=dev` only lists the optionals installed on the current host, so on Darwin/arm64 the cross-platform `@oxc-parser/binding-*` and its native helper transitives would report extraneous against the CSV. 3. `scripts/verify-exercised-tests.js` drops its yarn-detection paths now that workflows reference scripts only via `npm run`. `platform.yml`'s `yarn` and `yarn-berry` matrix entries stay untouched — they cover the user-PM contract, not the dev tooling.
Yarn 1.x is unmaintained. The docs build was the last install path still on yarn. `docs/yarn.lock` is replaced with `docs/bun.lock` (text format); TypeDoc and tsc still drive the build, so generated docs are unchanged.
Yarn 1.x is unmaintained. The lockfile, the dedupe-bot jobs, the husky hook, and the `yarn-deduplicate` devDep all exist only to keep yarn 1 healthy; with installs, run-scripts, and docs already on bun and npm, they are accommodations to nothing. A cleanup without a regression guard rots — yarn slips back in the next time a workflow or contributor PR is reviewed without the migration in mind. Three coordinated pieces: 1. Delete the dead yarn machinery: `yarn.lock`, `.yarnrc`, the `yarn-dedupe`/`yarn-dedupe-push` jobs in `project.yml` and their STS policy file under `.github/chainguard/`, the husky pre-commit hook, the `dependencies:dedupe` script and `yarn-deduplicate` devDep, the matching entry in the sirun startup require-preload list, and the `yarn-cache-` prefix on the install-action cache key (already keyed off `bun.lock`). 2. Update `.gitignore` (drop `yarn-debug.log*`, `yarn-error.log*`, `.yarn-integrity`; keep the `**/yarn.lock` entries the `next` plugin and appsec next fixtures still emit), the contributor docs (`CONTRIBUTING.md`, `AGENTS.md`), and the apm-integrations agent skill files. 3. Add `scripts/test/no-yarn-dev-references.spec.js`. Walks `git ls-files`, allowlists the four intentional yarn-reference categories (product-code user-PM detection, user-PM test fixtures, user-facing docs, and the regression-test specs that name yarn literally), and `assert.deepStrictEqual`s the offending list against `[]`. The `test:scripts` glob in `package.json` already covers it. Drive-by fix: * Drop the `bluebird@3.7.2` transitive entry from `packages/dd-trace/test/plugins/externals.js` under `moleculer`. Moleculer 0.14 (the fixture's pinned line) dropped its bluebird dep, so the entry installs an unused package. * Tighten `release.yml`'s two docs-job `bun install` calls to `--frozen-lockfile`; both lockfiles are committed.
The packed `dd-trace.tgz` does not include `vendor/package-lock.json` (the `files` field ships pre-built `vendor/dist/**` artifacts only), so the `prepare` script (`cd vendor && npm ci --include=dev`) cannot re-bootstrap vendor in a consumer install context. Yarn 1's `--prod` mode skipped lifecycle scripts altogether; bun runs them by default. `--ignore-scripts` matches the prior behaviour, and the script is not needed for measuring the published package's install size.
Three coordinated fixes for the per-plugin sandbox install:
1. `--linker=isolated` placed every dependency under
`versions/node_modules/.bun/<pkg>@<ver>/` and only symlinked the
declared dep into each sandbox. Cross-workspace `require()` walks
from inside one sandbox could not reach a sibling's package, so
moleculer's runtime `require('bluebird')` fallback returned
`Cannot find module`. Switching to `linker = "hoisted"` lifts
shared deps to `versions/node_modules/<pkg>` while still nesting
different versions under their owning sandbox; matrix isolation
holds and the cross-workspace lookups resolve.
2. Bun runs `install` / `postinstall` only for packages listed in
the workspace root's `trustedDependencies`. Native plugins
(`aerospike`, `@confluentinc/kafka-javascript`, `pg-native`, ...)
installed without their `.node` files compiled, so `bindings`
threw `Could not locate the bindings file` at test time. The
script now collects every package name it installs (workspaces
plus peer-dep injections) and writes them to
`versions/package.json:trustedDependencies`.
3. `assertPeerDependencies` was writing the raw range from the
parent's `peerDependencies`/`devDependencies` straight into the
sandbox `package.json`. Bun resolves ranges to the lowest
matching version where yarn 1 picked the highest, so peer deps
like `@smithy/node-http-handler` (only present in newer
`@aws-sdk` releases) landed on an older sibling that did not
ship them. Pre-resolving each range via `bun pm view` matches
the prior coverage.
`versions/bunfig.toml` moves from a committed file (with a
`.gitignore` exception) to a script-generated one — single source of
truth alongside the workspace `package.json`. The regression test
walks Node's resolution from inside each sandbox via `createRequire`
so it passes against both hoisted and isolated layouts.
Moleculer's transit/util layer falls back to `require('bluebird')`
at runtime even in 0.14.x. Dropping the externals fixture entry on
the strength of moleculer's manifest alone (which does not list
bluebird) broke the plugin's tests with `Cannot find module
'bluebird'` across seven test cases.
Restored as a sibling sandbox; bun's hoisted linker lifts it into
`versions/node_modules/bluebird` where moleculer's `require()` walk
finds it.
`--linker=hoisted` lifts a single copy of each package to
`versions/node_modules/<pkg>`, which broke five plugin spec files
(`next`, `kafkajs`, `rhea`, `memcached`, `apollo`) that hard-code
paths like `versions/<plugin>@<ver>/node_modules/<plugin>/<internal>`:
the highest-version sandbox loses its nested copy under hoisted, so
the require fails with `Cannot find module`. Bun's `--linker=isolated`
keeps a per-sandbox symlink at exactly that path and resolves it
through the central `.bun/` store; the hard paths work again.
Cross-workspace `require()` lookups (moleculer's runtime
`require('bluebird')` fallback was the canary) now go through the
externals fixture's `dep: true, forced: true` shape: bluebird is
injected as a direct dep of each moleculer sandbox, so bun's
isolated layout symlinks it into
`versions/moleculer@<ver>/node_modules/bluebird` where moleculer's
require walk from the central store finds it.
This fixes the `Channel credentials must be a ChannelCredentials object` failure across all 22 `@google-cloud/pubsub@1.2.0` plugin specs under bun's isolated linker. `pubsub@1.2.0`'s `pubsub.js` source-requires `@grpc/grpc-js` when `PUBSUB_EMULATOR_HOST` (or `apiEndpoint`) is set, but the manifest does not declare it. The parent walk lands on whatever bun hoists at the workspace root — currently the 1.8+ tree shared with `google-gax@3.5.7` — and the credentials produced there fail the `instanceof ChannelCredentials` check inside 1.3.x's Channel constructor, the version pubsub@1.2.0's nested `google-gax@1.15.4` uses (`@grpc/grpc-js: ~1.3.6`). yarn 1 hoisted 1.3.x at the right place; bun does not. `assertPeerDependencies` already injects forced direct deps into every sandbox of a named package — moleculer's bluebird is the precedent. The existing `latests[name]` source ships `@grpc/grpc-js@1.14.3`, too new for `google-gax@1.15.4`; add an entry-level `version` override and inject `@grpc/grpc-js: ~1.3.6` for every `@google-cloud/pubsub` sandbox so bun hoists 1.3.x at the workspace root.
This fixes the langchain plugin tests under bun's isolated linker. The `langchain@>=0.1` sandbox cannot resolve `@langchain/openai` at all - bun isolated does not hoist transitive deps to the workspace root - and the only other source for the require, the `latests` pin in `packages/dd-trace/test/plugins/versions/package.json` (`1.2.2`), throws `Cannot use 'in' operator to search for 'output_version' in undefined` in the chat-model converter when paired with a `@langchain/core@0.1.0` BaseMessage, and otherwise sends a 1.x request shape that misses every recorded cassette and 401s on the proxy fallback. Yarn classic happened to hoist `@langchain/openai@0.0.34` (the transitive dep of `langchain@0.1.0`) to the workspace root, so every `langchain@<version>` sandbox resolved to that version via NODE_PATH walking and the recorded cassettes match its `OpenAI/JS 4.x` request shape. Pinning the same version explicitly in the generated `versions/package.json` puts bun's isolated layout on that version too. Recording cassettes against the newer `@langchain/openai` is a follow-up that earns its own change.
Both fixup commits added inline comments explaining the bun-vs-yarn hoisting difference; the no-yarn-dev-references guardrail flagged them because cross-PM contrast prose belongs in the commit body, not in code (per the comment-skip rule for "accuracy depends on code outside this file"). The structural why — bun's isolated linker does not hoist transitives, so we pin the missing transitive directly — is what stays in the comments; the historical contrast is gone.
Three holes in the yarn → bun migration of `versions/` surfaced in CI:
1. `pg-native` is in `trustedDependencies` so its install script
runs, but the actual `node-gyp` build sits in `libpq` (a
transitive). Bun's `trustedDependencies` does not cascade, so
`libpq`'s `addon.node` never builds and the `bindings` package
rejects the load at test time. Add `libpq` directly to the
trusted set.
2. `pkgJsonPath()` in the generated sandbox `index.js` resolved
`<name>/package.json`, which throws `ERR_PACKAGE_PATH_NOT_EXPORTED`
for packages that ship an `exports` map without a `./package.json`
entry (moleculer, react, ...). Walk `module.paths` directly so
the lookup stays exports-blind, mirroring `requirePackageJson`.
3. The bedrock-runtime LLMObs spec reaches into the workspace's own
`@smithy/node-http-handler` via `.get('@smithy/node-http-handler')`.
Under the isolated linker that transitive sits only inside
aws-sdk's private store and is not reachable from the workspace
root, so inject it as a direct dep of every
`@aws-sdk/client-bedrock-runtime` sandbox via the existing
`dep: true, forced: true` channel.
`dd-license-attribution` only enumerates root deps from `yarn.lock` or `package-lock.json`; the migration ships only `bun.lock`, so the upstream tool drops every root runtime/optional row on each run and the auto-update workflow fails on every PR. Until bun.lock support lands upstream (https://github.com/watson/dd-license-attribution), `scripts/check_licenses.js` (run via `npm run lint`) is the source of truth for CSV completeness against `bun.lock`, `vendor/package-lock.json`, and `vendored-dependencies.csv`. The workflow now does the bare minimum needed to keep the required check name green — assert that `LICENSE-3rdparty.csv` is present and non-empty on the PR head. The trigger no longer fires on `bun.lock` changes since we have no way to act on them; it stays on the vendor and vendored-deps paths so future tooling work has somewhere to plug back in.
Bun's isolated linker keeps a single shared copy of every package under `versions/node_modules/.bun/<name>@<ver>/`, so a native binding compiled by `pg-native` → `libpq`'s install hook under one Node major is reused verbatim on the second `npm run services` invocation under a different Node major. The Plugin and Instrumentation actions install once with the oldest LTS, then switch to the latest Node and run the same sandbox a second time — the cached `addon.node` then crashes on load with `undefined symbol: _ZN2v86Object16SetInternalFieldEiNS_5LocalINS_5ValueEEE`. Yarn 1's per-workspace layout rebuilt every binding on every install pass and never hit this. Wipe the central store (and the matching `bun.lock`) when the recorded Node ABI does not match the current process so the next `bun install --trust` reruns lifecycle scripts and rebuilds against the live runtime.
The workflow header ran afoul of the new `no-yarn-dev-references` spec that scans tracked files for `\byarn\b`. Reword the comment to explain the same constraint without using the word.
Two unrelated yarn → bun differences hide the same way: the test
process deadlocks for hours in `should not alter the default behavior
with pretty print` for pino@5/6/7, and the langchain `JSON message
input` regression rejects every plain `{role, content}` payload with
`messageLike is not iterable`. Both come down to bun installing a
different transitive than yarn classic did.
1. `bun pm view <pkg>@<range> version` returns the lowest matching
version, not the highest. `resolveLatestSatisfying` now pulls the
full version list and runs `semver.maxSatisfying` to mirror yarn
classic's choice. Same effect across the matrix: every cap that
yarn pinned to its highest under-cap version pins to that version
again under bun. Concretely, `@langchain/core@>0.1.56 <0.3.0`
resolves to `0.2.36` instead of `0.1.63`, where
`coerceMessageLikeToMessage` accepts JSON messages.
2. Bun's isolated linker keeps a central
`versions/node_modules/.bun/node_modules/` directory of one
symlink per package, picking the highest installed version
across every workspace. From inside `.bun/pino@5.0.0/.../pino/lib/tools.js`,
Node's resolution walks up into that directory and finds the
central `pino-pretty@13.1.3` symlink before falling through to
`NODE_PATH`'s per-workspace `pino-pretty@1.0.1`. Pino@5's pretty
integration then crashes with `pretty is not a function`, the
throw happens inside an internal pino write loop and never
surfaces a mocha failure — the runner just sits forever. Wipe
the central directory after every `bun install --trust` so
per-workspace resolution wins.
The benchmarking-platform image is rebuilt from `master` and does not yet ship bun, so every dev-tooling install path the benchmark exercises (`bun install` at the repo root, `npm run services` ↦ `scripts/install_plugin_modules.js` ↦ `bun install --trust`, `benchmark/sirun/runall.sh`'s own bootstrap) fails the moment bun is invoked. Add a `before_script` to `.benchmarks` that installs bun once per job and exports `$HOME/.bun/bin` on `PATH`, and move the runall.sh bootstrap out of its subshell so `PATH` stays set for the sirun loop instead of leaking with the subshell exit. Drop the benchmarks.yml `before_script` once the image bakes bun in.
`devflow/mergegate` rejects PRs that introduce a tracked file with no `CODEOWNERS` entry. `.github/workflows/install-smoke.yml` was added in this PR and exercises the lang-platform-js sandbox-install path, so route it to the same team that owns `/scripts/**`.
…to-sts `dd-license-attribution` does not yet read `bun.lock`, so the auto-update workflow had been disabled while the migration landed. Replace the upstream tool with `scripts/generate-3rdparty-licenses.js`, which walks the same locks and CSV the lint check already trusts (`bun.lock`, `vendor/package-lock.json`, `.github/vendored-dependencies.csv`) and fetches license + origin metadata from the npm registry. The script preserves rows that already exist in `LICENSE-3rdparty.csv` to keep the diff tight on re-runs and records the project itself from `package.json` so the self-row stays stable across registry hiccups. The workflow restores the original two-stage shape: `check-licenses` regenerates the CSV and uploads it as an artifact when the bot owns the PR; `auto-commit-licenses` mints a token via `DataDog/dd-octo-sts-action` (governed by the existing `update-3rdparty-licenses.sts.yaml` chainguard policy) and pushes the file via the GitHub API. Human PRs that drift the CSV continue to fail with the regen instructions, same shape as before. The matching CSV update sorts the two vendored entries (`aws-lambda-nodejs-runtime-interface-client`, `is-git-url`) into their alphabetical position so the regen and the committed file match byte-for-byte going forward.
The plugin smoke test is a real test surface; it should show up in Datadog Test Optimization the same way the full plugin matrix does. Mint a Datadog API key via `dd-sts-api-key` and call the shared `push_to_test_optimization` action under `if: !cancelled()`, so a failed install or smoke run still uploads its junit artifact for triage.
The previous fix wiped `versions/node_modules/.bun/node_modules/` wholesale, which silenced the `pino-pretty` shadow but also removed the central symlinks every other sandbox relied on for transitive resolution. `q@2`'s `collections`, `@grpc/grpc-js`'s `@grpc/proto-loader`, and `knex@0.8`'s `sqlite3` lookups all walk through that central store, and the sandbox install jobs for them broke as a side effect. Tighten the pruner to remove central symlinks only for packages that have more than one major installed in the `.bun/` store. That keeps hoisted resolution intact for single-version transitives (`@grpc/proto-loader@0.x`, `sqlite3@5.x`, `libpq@1.x`, ...) while still forcing per-workspace resolution for packages whose central pin shadows an incompatible major that a specific sandbox needs (`pino-pretty@13.x` shadowing `pino@5`'s `pino-pretty@1.0.1`). Drop the leftover yarn references from the regen script's JSDoc so the `no-yarn-dev-references` regression test stays green.
`q@2.0.0`'s manifest declares `collections: ^2.0.0`, but `q.js` does
`require('collections/shim')` and `shim.js` only exists in
`collections@>=5`. The previous package manager's flat hoist always
served `collections@5` from the workspace root, so q's resolution
walked up to it. Bun's isolated linker honours the `^2.0.0` range
and lands `collections@2.0.3` in q's per-package store, where the
require throws `Cannot find module 'collections/shim'`.
Pin `collections` to `^5.0.0` via the workspace `overrides` field so
every q sandbox lands on the same version the previous tooling
served. Other consumers in the workspace already declare or transit
via 5.x, so the override does not regress anything else.
CodeQL flagged the `author.replace(/\s*<[^>]+>\s*/, '')` regex as a potential HTML-injection sink because the npm-registry-supplied `author` / `contributors` strings are technically untrusted input. Split on the first `<` or `(` instead — npm's `Name <email> (url)` shape is well-defined enough that a literal slice gives the same result without the regex pattern that CodeQL warns about. Same output for every package in the current CSV, so the regen produces byte-identical rows.
The vertex-ai plugin spec stubs `GoogleAuth.getAccessToken` via
`require('versions/@google-cloud/vertexai@<ver>').get('google-auth-library/...')`.
`google-auth-library` is a regular transitive of `@google-cloud/vertexai`,
so under bun's isolated linker it lives only inside vertexai's private
store and isn't reachable from the workspace root. The previous package
manager's flat hoist masked this; bun didn't, and the spec hung at the
first `before` hook because `require.resolve` walks until it gives up.
Inject it as a direct dep of every `@google-cloud/vertexai` sandbox via
the same `dep: true, forced: true` channel that fixes the bedrock spec
right above it.
The `bun.sh/install` script unzips the release archive, so the install fails with `error: unzip is required to install bun` on the benchmarking-platform image because `ubuntu:22.04` does not ship `unzip`. Install it via `apt-get` only when missing — keeps the fast path on hosts that already have it (sirun's `runall.sh` bootstrap, eventually the image once it bakes bun in).
`docs/bun.lock` was added in this PR (replacing `docs/yarn.lock`) and devflow/mergegate flags any tracked file with no owner. The `type:doc:build` / `type:doc:test` scripts use the same toolchain as the rest of `/scripts/**`, so route the directory to the same team.
`@langchain/openai@0.0.34`'s manifest declares
`@langchain/core: >0.1.56 <0.3.0`. The langchain regression spec
(`instruments a langchain openai chat model call for a JSON message
input`) only works against a `0.2.x` core — the older
`coerceMessageLikeToMessage` in `0.1.x` only knows the
`[role, content]` tuple shape and crashes on the spec's
`{role, content}` object input.
Bun's isolated linker resolves that range deterministically per
host, but the chosen version varies: it lands `0.2.36` on the local
macOS dev machine and `0.1.63` on the github-hosted runner image.
The previous package manager always served the highest workspace
copy via flat hoist, so this never surfaced. Pin the floor for the
`@langchain/openai@0.0.34/@langchain/core` pair to `^0.2.0` so both
hosts agree, without affecting the `@langchain/openai@1.x.x` peer
constraint resolved elsewhere in the workspace.
`ai@4.0.2` declares `zod` as an optional peer (`^3.0.0`); `@ai-sdk/openai` @1.3.23+ as a required peer. Yarn 1's flat hoist served the standalone `zod` workspace's copy from the workspace root to both sandboxes; bun's isolated linker honours each package's own manifest and skips optional peers, so `versions/ai@4.0.2` lands without `zod` and the first `ai.generateText` invocation throws `Cannot find module 'zod'` from inside the SDK's response-parsing path. Inject `zod` as a direct dep of every `ai` sandbox via the existing `dep: true` channel so bun materialises it alongside `ai` in the isolated store.
Three coordinated fix-ups: 1. actionlint: the install action grew a `cache` input on master but the declaration didn't follow, so every workflow that passed `cache: 'true'` tripped `input "cache" is not defined`; and `integration-esbuild` still gated its steps on `matrix.bundler`, which no longer exists in that job's matrix (only `version` and `esbuild_version`). 2. verify-exercised-tests: `test:plugins:ci`, `test:integration:crashtracking`, `test:integration:electron`, and `test:integration:webpack` were still invoked via `yarn` in four workflows. The verifier matches workflow calls to package.json scripts and saw zero coverage for those entries. 3. integration-playwright: the datadog-ci composite reached for `oven-sh/setup-bun`, which extracts its archive through `unzip`. The Playwright runner image doesn't ship unzip, so the action failed before `datadog-ci` ever landed on PATH. Install via `npm install --no-save --ignore-scripts` instead and drop the orphaned `yarn.lock` / `.yarnrc` that lived next to it.
The lockfile drifted from the workspace `package.json` during the rebase (devDep bumps in `axios`, `bun`, `eslint-plugin-cypress`, `eslint-plugin-n`, `nock`, `semver`, `sinon`, `yaml`, plus the `@datadog/*` and `oxc-parser` runtime deps). `supported-integrations` regenerates the supported-versions files and asserts that only those two CSVs changed in the job's worktree; the stale lockfile let `bun install` rewrite the workspace dep table mid- job and tripped the integrity check. `generate-3rdparty-licenses.js` reads the same lockfile, so `check-licenses` was reporting one stale row against the current CSV. Regenerating synchronises both jobs against the post-rebase dep tree.
Two failures the previous yarn flat-hoist masked: 1. vertex-ai: `version: '*'` resolves to `google-auth-library@10`, but every published `@google-cloud/vertexai` declares its transitive at `^9.0.0`. Bun's isolated linker keeps both physical copies — the test stubs `GoogleAuth.prototype.getAccessToken` on the @10 copy, the SDK loads the @9 copy at run-time, the stub never applies, and the real credential lookup throws `GoogleAuthError`. Pin the direct dep to `^9.0.0` so bun dedupes to a single `.bun/google-auth-library@9.x.y` entry and the prototype stub propagates to the SDK. 2. ai: `@ai-sdk/ui-utils` ranges `zod-to-json-schema` at `^3.0.0` and `ai@4.0.2` ranges `zod` at `^3.0.0`; bun picks `zod-to-json-schema@3.25.2` next to `zod@3.23.8`. `3.25.x` switched its zod imports to the `zod/v3` subpath, which only exists in `zod@>=3.25.32` and `zod@4`, so the sandbox crashes at load time with `ERR_PACKAGE_PATH_NOT_EXPORTED`. Bun does not honour nested overrides (oven-sh/bun#6608), so a flat `zod-to-json-schema: <3.25.0` override is the only shape that works; the only other consumer (`langchain` / `langgraph`) declares `>=3.0.0` and the constraint still holds.
The `installs every pino sandbox ...` regression guard forwards a
constrained `PATH` to the spawned `install_plugin_modules.js` so a
re-introduced `yarn` invocation fails closed. The PATH it built started
with `~/.bun/bin`, but the node composite action installs bun through
`npm install -g bun@<ver>`; the binary lands under `npm prefix -g`,
not under `~/.bun`. Every `spawn('bun', …)` in the child returned
`ENOENT`, the install script exited with an empty stderr that hid the
cause, and `Sandbox install (pino)` failed on the assert. Resolve bun's
actual location via `command -v bun`, with a `BUN_BIN` override for
unusual layouts.
Drive-by fix:
* Rewrite the `patch-istanbul-lib-coverage.js` doc comment to drop a
stray `yarn upgrade` mention (no yarn invocation, purely cosmetic) and
allowlist `scripts/verify-exercised-tests.js` in the no-yarn spec —
its `indexOf('yarn ')` is structural workflow-string parsing, not a
yarn invocation.
`b4912ad7eb` removed the repo-root `.yarnrc` (`ignore-engines true`). `openfeature.spec.js` still shells out to yarn after `bun add` pulls in `@hono/node-server@2.x` (`engines.node >= 20`), so on the node-oldest matrix entry yarn errors before the build step and the integration-esbuild job hangs past 30 minutes against ~160s on master. `iast-stack-traces-with-sourcemaps` follows the same shape with deps that happen to be Node-18-clean today; pass the flag inline there too to keep the contract explicit so a future dep bump can't silently break the same matrix entry again.
The Windows install action enables the bun-output cache via the
`cache: "true"` input added in `46d98c9756`. On a cache hit it
`tar -xf`s the saved archive and skips `bun install`, but the original
archive only contained `node_modules`. `bun install`'s `prepare` step
runs `cd vendor && npm ci --include=dev`, whose `node rspack`
postinstall lands `vendor/dist/{limiter, ...}` outside that path, so
every cache-hit Windows job now fails at first require with
`Cannot find module '../../../vendor/dist/limiter'`.
Capture `vendor/node_modules` and `vendor/dist` in the archive, fold
`vendor/package-lock.json` into the cache key so a transitive bump
invalidates the entry, and rename the archive plus bump the key
suffix to `-v3` so already-poisoned entries miss.
The bun matrix entry in `package-manager` stalls indefinitely at `Resolving dependencies` after the install action's `bun install` primes the cache. Without a cap it has held the job for 45+ minutes on each PR push, while the four other package managers complete the same step against the same dd-trace.tgz in 3-5s. Wrap the consumer install in `nick-fields/retry` with a 2-minute per-attempt cap so a stalled bun fails fast and retries (warm cache on the second attempt) instead of pinning the runner. The other PMs finish well inside the cap, so they pay nothing for the wrapper. The symptom matches the bun resolver-stall reports collected under oven-sh/bun#5831.
The sandbox is already populated by `useSandbox` (which `bun add`s the declared deps and the dd-trace tarball), so the follow-up `yarn` pass in the `before` block ran the same resolution a second time with a different package manager and produced no test-specific signal. The spec doesn't exercise any yarn behaviour the way `openfeature.spec.js` (rm -rf node_modules + install yarn globally + yarn) or `test-optimization-startup.spec.js` (PM-detection matrix) do. Drop the call and the matching `no-yarn-dev-references` allowlist entry.
…ructural The isolation contract for plugin sandboxes lived only in the writer inside `scripts/install_plugin_modules.js`, and the resulting `versions/bunfig.toml` was caught by `/versions/*` in `.gitignore`. A fresh checkout had no bunfig until `npm run services` ran once, and a direct `bun install` invoked from `versions/` (manual debugging, sandbox repair) would pick up the host bun config instead of `linker = "isolated"` — the exact silent regression the original plan flagged. Negate the path in `.gitignore`, commit the three-line bunfig, and drop the now-redundant writer. The explanatory comment about isolated bun's symlink layout moves to the `bun install --trust` call site, which is the operative consumer.
The composite action ran \`npm install --no-save\`, so every CI run let the install resolve fresh against the npm registry. \`package.json\` pinned the top-level \`@datadog/datadog-ci\` version, but a republished tarball at the same version number would have been picked up silently; \`yarn.lock\` (deleted earlier in this PR) used to carry that integrity hash and the rewrite lost it. Add a committed \`package-lock.json\` for the action and switch the install to \`npm ci\`. The lockfile holds the tarball integrity hash, so a tampered or republished tarball at \`5.16.0\` is rejected before the binary lands on PATH.
The npm supply-chain pattern that keeps repeating lands a compromised version on the registry, takes effect on the next CI install, and gets yanked once detection catches up — usually within hours, sometimes a day or two. Three days is enough lead time that most flagged releases get pulled before they reach our installs, and the lockfile freeze makes the policy actually stick on every CI run. 1. bunfig.toml and versions/bunfig.toml set minimumReleaseAge = 259200 (three days). @datadog/* bypasses the wait — our publishing pipeline is the trust boundary for those — and the value mirrors .github/dependabot.yml's cooldown.exclude so the two policies stay aligned. 2. .github/actions/install/action.yml and benchmark/sirun/runall.sh pass --frozen-lockfile to bun install. CI fails the install step when bun.lock no longer matches package.json, so a contributor who edits package.json (or a Dependabot bump) has to ship the regenerated lock alongside. Without the freeze, the install would silently resolve fresh against the registry and the minimumReleaseAge filter would be the only thing left protecting that run. 3. The four @datadog/native-* packages plus bun move from the CLI's --trust flag into trustedDependencies in package.json. Bun refuses --trust and --frozen-lockfile together, so the trust list has to be declarative; this is the same pattern the plugin sandboxes already use via scripts/install_plugin_modules.js. CONTRIBUTING.md documents the policy and the escape valve for an advisory whose patch is still inside the three-day window: a maintainer runs `bun update <pkg>@<exact> --minimum-release-age=0` locally, commits the regenerated lock, and calls out the override in the PR description. The override stays out of bunfig.toml so every bypass shows up in a reviewable diff. Signed-off-by: Ruben Bridgewater <ruben.bridgewater@datadoghq.com>
Contributor
Overall package sizeSelf size: 5.63 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.0.1 | 82.56 kB | 817.39 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
|
BenchmarksBenchmark execution time: 2026-05-25 12:27:55 Comparing candidate commit c201603 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 1496 metrics, 97 unstable metrics. |
BridgeAR
force-pushed
the
BridgeAR/2026-05-09-remove-yarn
branch
6 times, most recently
from
July 15, 2026 15:16
17970e4 to
b74badb
Compare
Member
Author
|
Closing, as superseded by the original PR |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Tightens the npm supply-chain attack surface on top of #8386:
bunfig.toml+versions/bunfig.tomlsetminimumReleaseAge = 259200(three days). Mirrors the existingcooldown.exclude: ["@datadog/*"]policy in.github/dependabot.yml..github/actions/install/action.ymlandbenchmark/sirun/runall.shusebun install --frozen-lockfile, so CI fails whenbun.lockandpackage.jsondisagree.@datadog/native-*packages plusbunmove out of the CLI's--trustflag into a declarativetrustedDependenciesinpackage.json(bun refuses--trustand--frozen-lockfiletogether).CONTRIBUTING.mddocuments the policy and the escape valve for urgent advisories.Why
The npm supply-chain pattern that keeps repeating lands a compromised version on the registry, takes effect on the next CI install, and gets yanked once detection catches up — usually within hours, sometimes a day or two. Three days is enough lead time that most flagged releases get pulled before they reach our installs, and
--frozen-lockfilemakes the policy actually stick on every CI run (without it, a contributor who editspackage.jsonwithout re-runningbun installwould silently resolve fresh against the registry and bypass the cooldown).Escape valve
For an advisory whose patch is still inside the three-day window, a maintainer runs
bun update <pkg>@<exact> --minimum-release-age=0locally, commits the regeneratedbun.lock, and calls out the override in the PR description. The override stays out ofbunfig.tomlso every bypass shows up in a reviewable diff.Stacking
Based on #8386 (
BridgeAR/2026-05-09-remove-yarn). The bunfig / lockfile / trustedDependencies changes only make sense once that PR's bun migration lands, so the base will move tomasteronce #8386 merges.Test plan
bun install --frozen-lockfile --linker=hoisted --network-concurrency 8runs clean from a freshnode_modules/vendor(619 packages, 0 untrusted).bun pm untrustedreports zero blocked lifecycle scripts after the install.--frozen-lockfileerrors on a genuinely new dep (verified locally with a throwaway addition).scripts/test/no-yarn-dev-references.spec.jsandscripts/test/install-plugin-modules.spec.jsstill pass.