[rum-privacy] add sourcemaps and telemetry #187

cy-moi · 2025-07-04T15:50:15Z

What and why?

Improve the privacy plugin and update to the latest js-instrument-wasm library.
We want to add sourcemaps and telemetry to our privacy plugin.

How?

Change enforce: pre to post for privacy plugin
Make sdk injection plugin optional when opt-in for rum plugin
Add Typescript end to end testing and support
Update transform options to make sourcemaps working
Add logs for observability

sethfowler-datadog

Looks great! I posted a few nits below:

packages/plugins/rum/src/privacy/index.ts

packages/plugins/rum/src/privacy/transform.ts

packages/plugins/rum/src/privacy/types.ts

packages/tests/src/e2e/privacyPluginTypescript/privacyPluginTypescript.spec.ts

…re effective

yoannmoinet

Started a review, will continue later on.

packages/plugins/rum/src/privacy/index.ts

yoannmoinet · 2025-07-15T12:38:51Z

packages/plugins/rum/src/privacy/index.ts

-        pluginOptions.module === 'cjs' ? './privacy-helpers.js' : './privacy-helpers.mjs',
-    );
-
+    const privacyHelpersModuleId = pluginOptions.helpersModule ?? PRIVACY_HELPERS_MODULE_ID;


Do we need the ?? ... part?
We don't do it in buildTransformOptions().

packages/plugins/rum/src/built/privacy-helpers.ts

packages/plugins/rum/src/privacy/index.ts

packages/plugins/rum/src/sdk.ts

packages/plugins/rum/src/validate.test.ts

packages/plugins/rum/src/validate.ts

yoannmoinet

I'm not a fan of the addition to the verifyProjectBuild.
It's very complex, I'll have a look, but I think this can be simplified.

packages/plugins/rum/src/types.ts

packages/plugins/rum/src/validate.ts

packages/tests/src/_playwright/helpers/buildProject.ts

yoannmoinet · 2025-07-15T13:39:02Z

packages/tests/src/_playwright/helpers/buildProject.ts

+
+    // Get the entry for this specific bundler
+    const bundlerEntry = buildConfigOverride?.entry?.[bundler] || './index.js';
+
+    // Handle TypeScript compilation for each bundler
+    const additionalPlugins = [...(buildConfigOverride?.plugins || [])];
+
+    // Check if any entry is a TypeScript file
+    const hasTypeScriptEntries = Object.values(buildConfigOverride?.entry || {}).some((entry) =>
+        entry.endsWith('.ts'),
+    );
+
+    if (hasTypeScriptEntries) {
+        if (bundler === 'rollup' || bundler === 'vite') {
+            // Use @rollup/plugin-typescript for Rollup and Vite
+            additionalPlugins.push(
+                typescript({
+                    tsconfig: path.resolve(cwd, 'tsconfig.json'),
+                }),
+            );
+        }
+        // ESBuild has built-in TypeScript support, no additional plugins needed
+    }
+


Can't this logic live at the test level?

As-in, if your test requires to override the build's configuration, it should also handle this part as well (adding the ts plugin to the build).

Can you also give me more details on why you need this @rollup/plugin-typescript plugin exactly and not re-use the esbuild plugin?

I did try this, but it requires changing the params for verifyProjectBuild even more, as we need to be able to accept different plugins for different bundlers. We are using @rollup/plugin-typescript to isolate issues of generated sourcemaps and avoid having two bundlers for the same build, per discussions in slack previously.

packages/tools/src/bundlers.ts

packages/tools/src/plugins.ts

yoannmoinet

Very good update!
Left some comments and nits.

yoannmoinet · 2025-07-28T08:20:19Z

packages/plugins/injection/src/esbuild.ts

+                    // FIXME: Handle sourcemaps.
+                    await fsp.writeFile(output, data.code);
+                } catch (e) {
+                    // skip if the file doesn't exist


Can you specify the error if we only want to skip if the file doesn't exist?

Suggested change

// skip if the file doesn't exist

// Skip if the file doesn't exist

if (e.code !== 'ENOENT') {

throw e;

}

Also, in which circonstances will it happen?

For example, when we have sub-builds in web-ui, the build entries are not actual entries and it may not exist when injecting. I'm not sure we should throw the error when the file does not exist, as it is not the entry file we want to inject anyway. I added comments and a log here instead.

packages/plugins/rum/src/privacy/constants.ts

packages/plugins/rum/src/built/privacy-helpers.ts

packages/plugins/rum/src/privacy/index.ts

packages/plugins/rum/src/privacy/transform.ts

packages/plugins/rum/src/validate.ts

yoannmoinet · 2025-07-28T08:36:19Z

packages/published/rollup-plugin/package.json

-        "./dist-basic/src": "./dist-basic/src/index.js",
-        "./dist-basic/src/*": "./dist-basic/src/*",


I believe this got removed automatically once again.
The process has been fixed in a separate branch.

yoannmoinet · 2025-07-28T08:39:11Z

packages/plugins/rum/src/index.ts

@@ -55,12 +55,15 @@ export const getPlugins: GetPlugins = ({ options, context }) => {
        });
    }

-    if (validatedOptions.privacy) {
+    if (validatedOptions.privacy && !validatedOptions.privacy.disabled) {


NIT: This should be handled in the validate process.
The same as for the sdk counterpart.

I removed the privacy.disabled option as well, per bug bash feedback.

yoannmoinet · 2025-07-28T08:40:48Z

packages/plugins/rum/src/index.test.ts

@@ -35,7 +35,7 @@ describe('RUM Plugin', () => {
        },
        {
            type: 'sdk',
-            config: { sdk: { applicationId: 'app-id' } },
+            config: { sdk: { applicationId: 'app-id', clientToken: '123' } },


Weird, what was the issue with this, how did we miss it before?

There is no longer an issue after restoring to previous types. clientToken is not required. I restored this as well.

Co-authored-by: Yoann Moinet <[email protected]>

feat:add sourcemaps and telemetry

ecceaf7

cy-moi force-pushed the congyao/add-sourcemaps-and-telemetry branch from eaaf2b5 to ecceaf7 Compare July 7, 2025 13:33

cy-moi added 3 commits July 9, 2025 18:51

feat:add sourcemaps and telemetry

655da60

update js-instrument

5954af8

fix:integrity

8a9fbd0

sethfowler-datadog approved these changes Jul 10, 2025

View reviewed changes

cy-moi added 8 commits July 10, 2025 15:32

fix: add loadInclude; remove unused code

ae53259

fix: integrity

6cd5f50

fix: remove too many logs

5f51ae3

fix: exclude by default spcial charaters; remove logs of apikey

2031208

fix:make configuration slim; make sdk opt-in with config; make e2e mo…

df69942

…re effective

fix:integrity

b8eedfc

feat:upgrade js-instrument

92fd9fb

Merge branch 'master' into congyao/add-sourcemaps-and-telemetry

cdbd040

yoannmoinet reviewed Jul 15, 2025

View reviewed changes

cy-moi force-pushed the congyao/add-sourcemaps-and-telemetry branch from ff4c50b to cdbd040 Compare July 15, 2025 13:15

yoannmoinet reviewed Jul 15, 2025

View reviewed changes

cy-moi added 8 commits July 15, 2025 17:45

fix:update helpers function to not extract twice

3d4189f

fix:minor polish and add privacy plugin back to e2e full config

63d80d6

fix:improve privacy-helper

1bff567

feat: use global variable to be more reliable

b900a91

fix: check if file exists before injection

2f3fb8a

fix: add to DD_ALLOW in lower case

5a450ff

fix: Improve code

91dd248

fix: improve types

8a0f8ab

cy-moi force-pushed the congyao/add-sourcemaps-and-telemetry branch from 2b630f6 to 8a0f8ab Compare July 25, 2025 08:03

fix: update configurations

b3cab69

cy-moi marked this pull request as ready for review July 25, 2025 09:07

cy-moi requested a review from a team as a code owner July 25, 2025 09:07

fix: make injected function pure

f1cb0b1

yoannmoinet reviewed Jul 28, 2025

View reviewed changes

cy-moi and others added 2 commits July 28, 2025 18:08

Update packages/plugins/rum/src/privacy/constants.ts

a4d8a8b

Co-authored-by: Yoann Moinet <[email protected]>

Improve code

31a8910

-                    // skip if the file doesn't exist
+                    // Skip if the file doesn't exist
+                    if (e.code !== 'ENOENT') {
+                        throw e;
+                    }

		"./dist-basic/src": "./dist-basic/src/index.js",
		"./dist-basic/src/": "./dist-basic/src/",

[rum-privacy] add sourcemaps and telemetry #187

Are you sure you want to change the base?

[rum-privacy] add sourcemaps and telemetry #187

Conversation

cy-moi commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What and why?

How?

Uh oh!

sethfowler-datadog left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yoannmoinet left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yoannmoinet left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cy-moi Jul 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

yoannmoinet left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cy-moi commented Jul 4, 2025 •

edited

Loading

cy-moi Jul 15, 2025 •

edited

Loading