Skip to content

Conversation

cy-moi
Copy link
Collaborator

@cy-moi cy-moi commented Jun 19, 2025

Motivation

Mask action name with allowlists generated from rum privacy build plugin(WIP). This approach is purely client side and allowlist-based. We aim to mask all action names (custom & auto) OOTB with build time configuration using a build plugin.

The raw string literals would be extracted at build time and loaded on demand in runtime with pre-injected helpers. In SDK, we only check action names and node text in SR with allowlist to mask action names.

Note: As we are using innerText to get action names, we can over mask in some cases.

Changes

  • Add allowlist processing helpers
  • Mask all action names with the allowlist when privacy build plugin is opt-in

Test instructions

Tests with BrowserStack should pass (for regEx compatibility)

Checklist

  • Tested locally
  • Tested on staging
  • Added unit tests for this change.
  • Added e2e/integration tests for this change.

@codecov-commenter
Copy link

codecov-commenter commented Jun 19, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.22%. Comparing base (ec025eb) to head (cf3170f).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3648      +/-   ##
==========================================
+ Coverage   92.20%   92.22%   +0.02%     
==========================================
  Files         333      336       +3     
  Lines        8322     8357      +35     
  Branches     1874     1894      +20     
==========================================
+ Hits         7673     7707      +34     
- Misses        649      650       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cy-moi cy-moi force-pushed the congyao/RUM-10415-add-privacy-allowlist-support branch 2 times, most recently from 8e29d87 to dc2ecea Compare June 23, 2025 10:05
Copy link

cit-pr-commenter bot commented Jun 23, 2025

Bundles Sizes Evolution

📦 Bundle Name Base Size Local Size 𝚫 𝚫% Status
Rum 151.43 KiB 152.39 KiB 978 B 0.63%
Rum Recorder 19.06 KiB 19.06 KiB 0 B 0.00%
Rum Profiler 5.17 KiB 5.17 KiB 0 B 0.00%
Logs 54.70 KiB 54.70 KiB 0 B 0.00%
Flagging N/A 931 B 931 B N/A%
Rum Slim 109.92 KiB 110.79 KiB 884 B 0.79%
Worker 23.60 KiB 23.60 KiB 0 B 0.00%
🚀 CPU Performance
Action Name Base Average Cpu Time (ms) Local Average Cpu Time (ms) 𝚫
addglobalcontext 0.012 0.016 0.004
addaction 0.032 0.032 -0.001
addtiming 0.005 0.006 0.001
adderror 0.030 0.031 0.001
startstopsessionreplayrecording 0.005 0.005 0.000
startview 0.010 0.010 0.001
logmessage 0.035 0.033 -0.003
🧠 Memory Performance
Action Name Base Consumption Memory (bytes) Local Consumption Memory (bytes) 𝚫 (bytes)
addglobalcontext 26.32 KiB 28.95 KiB 2.62 KiB
addaction 45.11 KiB 47.11 KiB 2.00 KiB
addtiming 24.19 KiB 26.82 KiB 2.64 KiB
adderror 49.04 KiB 50.45 KiB 1.42 KiB
startstopsessionreplayrecording 23.74 KiB 25.60 KiB 1.86 KiB
startview 422.00 KiB 428.05 KiB 6.05 KiB
logmessage 83.68 KiB 94.30 KiB 10.62 KiB

🔗 RealWorld

@cy-moi cy-moi force-pushed the congyao/RUM-10415-add-privacy-allowlist-support branch from 9654e39 to 489bf17 Compare June 23, 2025 13:06
@cy-moi cy-moi changed the title ✨ [RUM-10415] Add support for privacy plugin extracted data for masking ✨ [RUM-10415] Add support for action name allowlist masking Jun 23, 2025
@cy-moi cy-moi marked this pull request as ready for review June 23, 2025 14:26
@cy-moi cy-moi requested a review from a team as a code owner June 23, 2025 14:26
lifeCycle.notify(LifeCycleEventType.RAW_RUM_EVENT_COLLECTED, processAction(action))
)
const actionNameDictionary = createActionAllowList()
addAllowlistObserver(actionNameDictionary)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could createActionAllowList() register the observer? It feels like this code shouldn't need to know about that. (Maybe createActionAllowList() is really startActionAllowListObserver() or something?)

Going a bit further: right now, there is no code that removes the allowlist observer when startActionCollection()'s stop() is called. So we create a new dictionary and add another observer whenever startActionCollection() is called, but nothing ever removes them, and they keep building up. I don't think we want that.

We should pick one of two approaches:

  • Use a single global allowlist observer and dictionary, and never replace them or clear them. (Reasonable, since $DD_ALLOW is also global.)
  • Register the allowlist observer and set up the dictionary when action collection starts; unregister the allowlist observer and clear the underlying dictionary when action collection stops.

The first avoids reprocessing the raw strings when recording restarts, so it has a performance benefit, but naturally you'll have to add some additional affordances for testing with that approach.

Copy link
Collaborator Author

@cy-moi cy-moi Jun 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see.
With the first approach, do we need end-to-end tests or just keeping the contexts in unit tests would be enough? I'm ok to proceed with the second approach for now but clearing the dictionary and re-process could get expensive. We might need some field data on this.
Fixed with the 2nd approach.


let masked = false
return {
name: name.replace(getMatchRegex(), (word: string) => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ question: IIUC, when multiple tokens are masked, we might end up with a name like "MASKED MASKED MASKED", which could be confusing from a UI perspective. Especially, given that the other masking strategy displays "Masked Element", shouldn’t we aim for consistency? Do we have product input on this?

Copy link
Collaborator Author

@cy-moi cy-moi Jun 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, although Masked Element is not idea for tokenized the names either, maybe we should seek another way, ie XXX XXX in session replay. Will ping product on this.

@@ -14,8 +14,9 @@ export const enum ActionNameSource {
TEXT_CONTENT = 'text_content',
STANDARD_ATTRIBUTE = 'standard_attribute',
BLANK = 'blank',
MASK_DISALLOWED = 'mask_disallowed',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💬 suggestion: ‏I find this source name confusing. As mentioned in my other comment, I think we should ensure consistent behavior across our masking strategies. That way, we can keep using MASK_PLACEHOLDER as the source. If we need to identify which masking strategy was applied, we could include it as a separate event attribute. Wdyt?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, yes we would like to distinguish the two strategies. We did add mask_disallowed as a name source in rum-event-format as a separate attribute. But I'm open to change it to another name.

@cy-moi cy-moi force-pushed the congyao/RUM-10415-add-privacy-allowlist-support branch 3 times, most recently from 1de7145 to ce564c5 Compare June 27, 2025 13:30
@cy-moi cy-moi force-pushed the congyao/RUM-10415-add-privacy-allowlist-support branch 2 times, most recently from 617489d to 129b1aa Compare June 27, 2025 14:55
@cy-moi cy-moi force-pushed the congyao/RUM-10415-add-privacy-allowlist-support branch from 129b1aa to 549159f Compare June 27, 2025 14:55
@cy-moi cy-moi force-pushed the congyao/RUM-10415-add-privacy-allowlist-support branch from 16affe6 to 6f20b2b Compare June 27, 2025 15:49
@cy-moi cy-moi force-pushed the congyao/RUM-10415-add-privacy-allowlist-support branch from d69e70c to 6e5063b Compare June 30, 2025 12:45
@cy-moi cy-moi force-pushed the congyao/RUM-10415-add-privacy-allowlist-support branch from 7c8f655 to 5105112 Compare July 1, 2025 14:59
@cy-moi
Copy link
Collaborator Author

cy-moi commented Jul 7, 2025

/to-staging

@cy-moi cy-moi force-pushed the congyao/RUM-10415-add-privacy-allowlist-support branch from aea0f2c to da92015 Compare July 9, 2025 11:36
@cy-moi cy-moi force-pushed the congyao/RUM-10415-add-privacy-allowlist-support branch from ca8df7f to 045d430 Compare July 9, 2025 12:16
@cy-moi cy-moi force-pushed the congyao/RUM-10415-add-privacy-allowlist-support branch from 065c031 to febd0bb Compare July 21, 2025 15:09
@cy-moi cy-moi force-pushed the congyao/RUM-10415-add-privacy-allowlist-support branch from 39038bd to 5196475 Compare July 22, 2025 10:05
@cy-moi
Copy link
Collaborator Author

cy-moi commented Jul 23, 2025

Per discussion internally, we changed to the exact match with DD_ALLOW approach.

@cy-moi
Copy link
Collaborator Author

cy-moi commented Jul 23, 2025

/to-staging

@dd-devflow-routing-codex
Copy link

dd-devflow-routing-codex bot commented Jul 23, 2025

View all feedbacks in Devflow UI.

2025-07-23 14:52:07 UTC ℹ️ Start processing command /to-staging


2025-07-23 14:52:22 UTC ℹ️ Branch Integration: starting soon, merge expected in approximately 11m53s (p90)

Commit cf3170f0f0 will soon be integrated into staging-30.


2025-07-23 14:52:51 UTC 🚨 Branch Integration: this merge request has conflicts which couldn't be solved automatically

We couldn't automatically merge the commit cf3170f0f0 into staging-30!

To solve the conflicts directly in Github, click here to create a fix pull request.

Alternatively, you can also click here reset the integration branch or use the following Slack command: /devflow reset-branch -r browser-sdk -b staging-30

dd-devflow bot added a commit that referenced this pull request Jul 23, 2025
@dd-devflow
Copy link
Contributor

dd-devflow bot commented Jul 23, 2025

🚂 Branch Integration: starting soon, merge expected in approximately 11m53s (p90)

Commit cf3170f0f0 will soon be integrated into staging-30.

@dd-devflow
Copy link
Contributor

dd-devflow bot commented Jul 23, 2025

🚂 Branch Integration

Commit cf3170f0f0 has been merged into staging-30 in merge commit 5897d98d7e.

Check out the triggered pipeline on Gitlab 🦊

If you need to revert this integration, you can use the following command: /code revert-integration -b staging-30

@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Aug 1, 2025

⚠️ Tests

⚠️ Warnings

🧪 1 Test failed

recorder › scroll positions › should be recorded across navigation from recorder/recorder.scenario.ts (Datadog)
createTest.ts:204:3 should be recorded across navigation

[Firefox] › ../lib/framework/createTest.ts:204:3 › recorder › scroll positions › should be recorded across navigation 

    Error: expect(received).toBe(expected) // Object.is equality

    Expected: 100
    Received: 150

       at recorder/recorder.scenario.ts:802
...

ℹ️ Info

❄️ No new flaky tests detected

Code coverage: total 92.23%, base diff 0.02%, patch 100.00% (view details)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 3ecbb66 | Docs | Was this helpful? Give us feedback!

@cy-moi cy-moi closed this Sep 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants