[rum-privacy] add sourcemaps and telemetry by cy-moi · Pull Request #187 · DataDog/build-plugins

cy-moi · 2025-07-04T15:50:15Z

What and why?

Improve the privacy plugin and update to the latest js-instrument-wasm library.
We want to add sourcemaps and telemetry to our privacy plugin.

How?

Change enforce: pre to post for privacy plugin
Make sdk injection plugin optional when opt-in for rum plugin
Add Typescript end to end testing and support
Update transform options to make sourcemaps working
Add logs for observability

sethfowler-datadog

Looks great! I posted a few nits below:

…re effective

yoannmoinet

Started a review, will continue later on.

yoannmoinet · 2025-07-15T12:38:51Z

-        pluginOptions.module === 'cjs' ? './privacy-helpers.js' : './privacy-helpers.mjs',
-    );
-
+    const privacyHelpersModuleId = pluginOptions.helpersModule ?? PRIVACY_HELPERS_MODULE_ID;


Do we need the ?? ... part?
We don't do it in buildTransformOptions().

yoannmoinet

I'm not a fan of the addition to the verifyProjectBuild.
It's very complex, I'll have a look, but I think this can be simplified.

yoannmoinet · 2025-07-15T13:39:02Z

+
+    // Get the entry for this specific bundler
+    const bundlerEntry = buildConfigOverride?.entry?.[bundler] || './index.js';
+
+    // Handle TypeScript compilation for each bundler
+    const additionalPlugins = [...(buildConfigOverride?.plugins || [])];
+
+    // Check if any entry is a TypeScript file
+    const hasTypeScriptEntries = Object.values(buildConfigOverride?.entry || {}).some((entry) =>
+        entry.endsWith('.ts'),
+    );
+
+    if (hasTypeScriptEntries) {
+        if (bundler === 'rollup' || bundler === 'vite') {
+            // Use @rollup/plugin-typescript for Rollup and Vite
+            additionalPlugins.push(
+                typescript({
+                    tsconfig: path.resolve(cwd, 'tsconfig.json'),
+                }),
+            );
+        }
+        // ESBuild has built-in TypeScript support, no additional plugins needed
+    }
+


Can't this logic live at the test level?

As-in, if your test requires to override the build's configuration, it should also handle this part as well (adding the ts plugin to the build).

Can you also give me more details on why you need this @rollup/plugin-typescript plugin exactly and not re-use the esbuild plugin?

I did try this, but it requires changing the params for verifyProjectBuild even more, as we need to be able to accept different plugins for different bundlers. We are using @rollup/plugin-typescript to isolate issues of generated sourcemaps and avoid having two bundlers for the same build, per discussions in slack previously.

yoannmoinet

Very good update!
Left some comments and nits.

yoannmoinet · 2025-07-28T08:20:19Z

+                    // FIXME: Handle sourcemaps.
+                    await fsp.writeFile(output, data.code);
+                } catch (e) {
+                    // skip if the file doesn't exist


Can you specify the error if we only want to skip if the file doesn't exist?

Suggested change

// skip if the file doesn't exist

// Skip if the file doesn't exist

if (e.code !== 'ENOENT') {

throw e;

}

Also, in which circonstances will it happen?

For example, when we have sub-builds in web-ui, the build entries are not actual entries and it may not exist when injecting. I'm not sure we should throw the error when the file does not exist, as it is not the entry file we want to inject anyway. I added comments and a log here instead.

You still need to filter on the e.code, otherwise, you will skip every kind of error, and not just the "missing file" error.

My suggestion isn't throwing when the file does not exist, on the contrary, it only throws when it's NOT a missing file error.

Oh yes indeed! Sorry I read wrong

Your current code is skipping any error that could happen in fsp.readFile, esbuild.transform and fsp.writeFile, which is not what we want, and not what the comment is suggesting.

This should be fixed in the latest commit.

Co-authored-by: Yoann Moinet <597828+yoannmoinet@users.noreply.github.com>

yoannmoinet

Small nits.
Waiting for the update on the esbuild skipping errors then I'll ✅

yoannmoinet · 2025-07-30T09:28:19Z

    // Validate and add defaults sub-options.
    const sdkResults = validateSDKOptions(options);
-    const privacyResults = validatePrivacyOptions(options);
+    const privacyResults = validatePrivacyOptions(options, log);


NIT: instead of passing the log here, you could log what you need on line 42.

Co-authored-by: Yoann Moinet <597828+yoannmoinet@users.noreply.github.com>

yoannmoinet

LGTM, well done!

Co-authored-by: Yoann Moinet <597828+yoannmoinet@users.noreply.github.com>

cy-moi · 2025-08-04T11:32:55Z

/merge

dd-devflow-routing-codex · 2025-08-04T11:32:59Z

View all feedbacks in Devflow UI.

2025-08-04 11:32:59 UTC ℹ️ Start processing command /merge

2025-08-04 11:33:04 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 0s (p90).

2025-08-04 13:33:18 UTC ❌ MergeQueue: The build pipeline has timeout

The merge request has been interrupted because the build 0 took longer than expected. The current limit for the base branch 'master' is 120 minutes.

Possible reasons:

target branch of PR is restricted to only allow up-to-date branches, but the pr is now outdated

feat:add sourcemaps and telemetry

ecceaf7

cy-moi force-pushed the congyao/add-sourcemaps-and-telemetry branch from eaaf2b5 to ecceaf7 Compare July 7, 2025 13:33

cy-moi added 3 commits July 9, 2025 18:51

feat:add sourcemaps and telemetry

655da60

update js-instrument

5954af8

fix:integrity

8a9fbd0

sethfowler-datadog approved these changes Jul 10, 2025

View reviewed changes

cy-moi added 8 commits July 10, 2025 15:32

fix: add loadInclude; remove unused code

ae53259

fix: integrity

6cd5f50

fix: remove too many logs

5f51ae3

fix: exclude by default spcial charaters; remove logs of apikey

2031208

fix:make configuration slim; make sdk opt-in with config; make e2e mo…

df69942

…re effective

fix:integrity

b8eedfc

feat:upgrade js-instrument

92fd9fb

Merge branch 'master' into congyao/add-sourcemaps-and-telemetry

cdbd040

yoannmoinet reviewed Jul 15, 2025

View reviewed changes

cy-moi force-pushed the congyao/add-sourcemaps-and-telemetry branch from ff4c50b to cdbd040 Compare July 15, 2025 13:15

yoannmoinet reviewed Jul 15, 2025

View reviewed changes

cy-moi added 8 commits July 15, 2025 17:45

fix:update helpers function to not extract twice

3d4189f

fix:minor polish and add privacy plugin back to e2e full config

63d80d6

fix:improve privacy-helper

1bff567

feat: use global variable to be more reliable

b900a91

fix: check if file exists before injection

2f3fb8a

fix: add to DD_ALLOW in lower case

5a450ff

fix: Improve code

91dd248

fix: improve types

8a0f8ab

cy-moi force-pushed the congyao/add-sourcemaps-and-telemetry branch from 2b630f6 to 8a0f8ab Compare July 25, 2025 08:03

fix: update configurations

b3cab69

cy-moi marked this pull request as ready for review July 25, 2025 09:07

cy-moi requested a review from a team as a code owner July 25, 2025 09:07

fix: make injected function pure

f1cb0b1

yoannmoinet reviewed Jul 28, 2025

View reviewed changes

cy-moi and others added 2 commits July 28, 2025 18:08

Update packages/plugins/rum/src/privacy/constants.ts

a4d8a8b

Co-authored-by: Yoann Moinet <597828+yoannmoinet@users.noreply.github.com>

Improve code

31a8910

yoannmoinet reviewed Jul 30, 2025

View reviewed changes

cy-moi and others added 4 commits July 30, 2025 15:16

Update packages/plugins/rum/src/built/privacy-helpers.ts

03fc9a1

Co-authored-by: Yoann Moinet <597828+yoannmoinet@users.noreply.github.com>

Throw errors unless injection files non-exist

6b75c77

fix:integrity

2d32edd

fix:add log forward

5101569

yoannmoinet approved these changes Jul 30, 2025

View reviewed changes

Comment thread packages/plugins/injection/src/esbuild.ts Outdated

Update packages/plugins/injection/src/esbuild.ts

37ead59

Co-authored-by: Yoann Moinet <597828+yoannmoinet@users.noreply.github.com>

dd-devflow Bot added mergequeue-status: queued mergequeue-status: in_progress mergequeue-status: rejected and removed mergequeue-status: queued mergequeue-status: in_progress labels Aug 4, 2025

Merge branch 'master' into congyao/add-sourcemaps-and-telemetry

5933f3e

yoannmoinet enabled auto-merge August 4, 2025 13:45

yoannmoinet merged commit 06ad51b into master Aug 4, 2025
5 checks passed

yoannmoinet deleted the congyao/add-sourcemaps-and-telemetry branch August 4, 2025 13:49

-                    // skip if the file doesn't exist
+                    // Skip if the file doesn't exist
+                    if (e.code !== 'ENOENT') {
+                        throw e;
+                    }

Uh oh!

Conversation

cy-moi commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What and why?

How?

Uh oh!

sethfowler-datadog left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yoannmoinet left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yoannmoinet left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cy-moi Jul 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

yoannmoinet left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yoannmoinet left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yoannmoinet left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cy-moi commented Aug 4, 2025

Uh oh!

dd-devflow-routing-codex Bot commented Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cy-moi commented Jul 4, 2025 •

edited

Loading

cy-moi Jul 15, 2025 •

edited

Loading

dd-devflow-routing-codex Bot commented Aug 4, 2025 •

edited

Loading