[network-plugin] Feat: Capture network events (2)

.changeset/config.json

@@ -22,7 +22,9 @@
       "@rrweb/rrweb-plugin-sequential-id-record",
       "@rrweb/rrweb-plugin-sequential-id-replay",
       "@rrweb/rrweb-plugin-canvas-webrtc-record",
-      "@rrweb/rrweb-plugin-canvas-webrtc-replay"
+      "@rrweb/rrweb-plugin-canvas-webrtc-replay",
+      "@rrweb/rrweb-plugin-network-record",
+      "@rrweb/rrweb-plugin-network-replay"
     ]
   ],
   "linked": [],

.changeset/six-lions-guess.md

@@ -0,0 +1,4 @@
+---
+---
+
+network-plugin

docs/recipes/network.md

@@ -0,0 +1,108 @@
+# network recorder and replayer
+
+Starting from v2.0.0, we add the plugin to record network output.
+This feature aims to provide developers with more information about the bug scene. There are some options for recording and replaying network output.
+
+### Enable recording network
+
+You can enable using default option like this:
+
+```js
+import rrweb from 'rrweb';
+import { getRecordNetworkPlugin } from '@rrweb/rrweb-plugin-network-record';
+
+rrweb.record({
+  emit: function emit(event) {
+    events.push(event);
+  },
+  // to use default record option
+  plugins: [getRecordNetworkPlugin()],
+});
+```
+
+You can also customize the behavior of logger like this:
+
+```js
+import rrweb from 'rrweb';
+import { getRecordNetworkPlugin } from '@rrweb/rrweb-plugin-network-record';
+
+rrweb.record({
+  emit: function emit(event) {
+    fetch('https://api.my-server.com/events', {
+      method: 'POST',
+      headers: {
+        'Content-Type': 'application/json',
+      },
+      body: JSON.stringify({
+        events: [event],
+      }),
+    });
+  },
+  // customized record options
+  plugins: [
+    getRecordNetworkPlugin({
+      initiatorTypes: ['fetch', 'xmlhttprequest'],
+      // mask/block recording event for request
+      transformRequestFn: (request) => {
+        // request.name is url
+        if (request.name.contains('rrweb-collector-api.com')) return; // skip request
+        delete request.requestHeaders['Authorization']; // remove sensetive data
+        request.responseBody = maskTextFn(request.responseBody);
+        return request;
+      },
+      recordHeaders: true,
+      recordBody: true,
+      recordInitialRequests: false,
+    }),
+  ],
+});
+```
+
+**alert**: If you are uploading events to a server, you should always use `transformRequestFn` to mask/block recording events for these requests or else you will cause a nasty loop.
+
+All options are described below:
+
+| key                   | default                                | description                                                                                                                                                                                         |
+| --------------------- | -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| initiatorTypes        | `['fetch','xmlhttprequest','img',...]` | Default value contains names of all [initiator types](https://developer.mozilla.org/en-US/docs/Web/API/PerformanceResourceTiming/initiatorType). You can override it by setting the types you need. |
+| transformRequestFn    | `(request) => request`                 | Transform recording event for request to block (skip) or mask/transofrm request (e.g. to hide sensetive data)                                                                                       |
+| recordHeaders         | `false`                                | Record the request & response headers for `fetch` and `xmlhttprequest` requests                                                                                                                     |
+| recordBody            | `false`                                | Record the request & response bodies for `fetch` and `xmlhttprequest` requests                                                                                                                      |
+| recordInitialRequests | `false`                                | Record an event for all requests prior to rrweb.record() being called                                                                                                                               |
+
+## replay network
+
+It is up to you to decide how to best replay your network events using the `onNetworkData` callback.
+
+```js
+import rrweb from 'rrweb';
+import { getReplayNetworkPlugin } from '@rrweb/rrweb-plugin-network-replay';
+
+const replayer = new rrweb.Replayer(events, {
+  plugins: [
+    getReplayNetworkPlugin({
+      onNetworkData: ({ requests }) => {
+        for (const request of requests) {
+          const name = request.name; // url
+          const method = request.method;
+          const status = request.status;
+          console.log(`${method} ${name} ${status}`);
+        }
+      },
+    }),
+  ],
+});
+replayer.play();
+```
+
+Description of replay option is as follows:
+
+| key           | default     | description                                                                                |
+| ------------- | ----------- | ------------------------------------------------------------------------------------------ |
+| onNetworkData | `undefined` | You could use this interface to replay the network requests in a simulated browser console |
+
+## technical implementation
+
+This implementation records [`fetch`](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) and [`XMLHttpRequest`](https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest) by patching their object & methods. We record document navigation using [`PerformanceNavigationTiming`](https://developer.mozilla.org/en-US/docs/Web/API/PerformanceNavigationTiming) and we use [`PerformanceResourceTiming`](https://developer.mozilla.org/en-US/docs/Web/API/PerformanceResourceTiming) for recording everything else (script, img, link etc.) via [`PerformanceObserver`](https://developer.mozilla.org/en-US/docs/Web/API/PerformanceObserver) API.
+
+For more information please see [[network-plugin] Feat: Capture network events #1105](https://github.com/rrweb-io/rrweb/pull/1105) PR.

docs/recipes/plugin.md

@@ -10,6 +10,8 @@ The plugin API is designed to extend the function of rrweb without bump the size
 - [@rrweb/rrweb-plugin-sequential-id-replay](packages/plugins/rrweb-plugin-sequential-id-replay): A plugin for replaying sequential IDs.
 - [@rrweb/rrweb-plugin-canvas-webrtc-record](packages/plugins/rrweb-plugin-canvas-webrtc-record): A plugin for stream `<canvas>` via WebRTC.
 - [@rrweb/rrweb-plugin-canvas-webrtc-replay](packages/plugins/rrweb-plugin-canvas-webrtc-replay): A plugin for playing streamed `<canvas>` via WebRTC.
+- [@rrweb/rrweb-plugin-network-record](packages/plugins/rrweb-plugin-network-record): A plugin for recording network requests (xhr/fetch).
+- [@rrweb/rrweb-plugin-network-replay](packages/plugins/rrweb-plugin-network-replay): A plugin for replaying network requests (xhr/fetch).
 
 ## Interface
 

docs/recipes/plugin.zh_CN.md

@@ -10,6 +10,8 @@
 - [@rrweb/rrweb-plugin-sequential-id-replay](packages/plugins/rrweb-plugin-sequential-id-replay)：一个用于回放顺序 ID 的插件。
 - [@rrweb/rrweb-plugin-canvas-webrtc-record](packages/plugins/rrweb-plugin-canvas-webrtc-record)：一个用于通过 WebRTC 流式传输 `<canvas>` 的插件。
 - [@rrweb/rrweb-plugin-canvas-webrtc-replay](packages/plugins/rrweb-plugin-canvas-webrtc-replay)：一个用于通过 WebRTC 播放流式 `<canvas>` 的插件。
+- [@rrweb/rrweb-plugin-network-record](packages/plugins/rrweb-plugin-network-record): 一个用于记录网络请求的插件 (xhr/fetch)。
+- [@rrweb/rrweb-plugin-network-replay](packages/plugins/rrweb-plugin-network-replay): 一个用于回放网络请求的插件 (xhr/fetch)。
 
 ## 接口
 

guide.md

@@ -56,6 +56,8 @@ Besides the `rrweb` and `@rrweb/record` packages, rrweb also provides other pack
 - [@rrweb/rrweb-plugin-sequential-id-replay](packages/plugins/rrweb-plugin-sequential-id-replay): A plugin for replaying sequential IDs.
 - [@rrweb/rrweb-plugin-canvas-webrtc-record](packages/plugins/rrweb-plugin-canvas-webrtc-record): A plugin for stream `<canvas>` via WebRTC.
 - [@rrweb/rrweb-plugin-canvas-webrtc-replay](packages/plugins/rrweb-plugin-canvas-webrtc-replay): A plugin for playing streamed `<canvas>` via WebRTC.
+- [@rrweb/rrweb-plugin-network-record](packages/plugins/rrweb-plugin-network-record): A plugin for recording network requests (xhr/fetch).
+- [@rrweb/rrweb-plugin-network-replay](packages/plugins/rrweb-plugin-network-replay): A plugin for replaying network requests (xhr/fetch).
 
 ### NPM
 

guide.zh_CN.md

@@ -53,6 +53,8 @@ rrweb 代码分为录制和回放两部分，大多数时候用户在被录制
 - [@rrweb/rrweb-plugin-sequential-id-replay](packages/plugins/rrweb-plugin-sequential-id-replay)：一个用于回放顺序 ID 的插件。
 - [@rrweb/rrweb-plugin-canvas-webrtc-record](packages/plugins/rrweb-plugin-canvas-webrtc-record)：一个用于通过 WebRTC 流式传输 `<canvas>` 的插件。
 - [@rrweb/rrweb-plugin-canvas-webrtc-replay](packages/plugins/rrweb-plugin-canvas-webrtc-replay)：一个用于通过 WebRTC 播放流式 `<canvas>` 的插件。
+- [@rrweb/rrweb-plugin-network-record](packages/plugins/rrweb-plugin-network-record): 一个用于记录网络请求的插件 (xhr/fetch)。
+- [@rrweb/rrweb-plugin-network-replay](packages/plugins/rrweb-plugin-network-replay): 一个用于回放网络请求的插件 (xhr/fetch)。
 
 ### 通过 npm 引入
 

packages/plugins/rrweb-plugin-network-record/README.md

@@ -0,0 +1,178 @@
+# @rrweb/rrweb-plugin-network-record
+
+Please refer to the [network recipe](../../../docs/recipes/network.md) on how to use this plugin.
+See the [guide](../../../guide.md) for more info on rrweb.
+
+## Sponsors
+
+[Become a sponsor](https://opencollective.com/rrweb#sponsor) and get your logo on our README on Github with a link to your site.
+
+### Gold Sponsors 🥇
+
+<div dir="auto">
+
+<a href="https://opencollective.com/rrweb/tiers/gold-sponsor/0/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/gold-sponsor/0/avatar.svg?requireActive=false&avatarHeight=225" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/gold-sponsor/1/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/gold-sponsor/1/avatar.svg?requireActive=false&avatarHeight=225" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/gold-sponsor/2/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/gold-sponsor/2/avatar.svg?requireActive=false&avatarHeight=225" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/gold-sponsor/3/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/gold-sponsor/3/avatar.svg?requireActive=false&avatarHeight=225" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/gold-sponsor/4/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/gold-sponsor/4/avatar.svg?requireActive=false&avatarHeight=225" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/gold-sponsor/5/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/gold-sponsor/5/avatar.svg?requireActive=false&avatarHeight=225" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/gold-sponsor/6/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/gold-sponsor/6/avatar.svg?requireActive=false&avatarHeight=225" alt="sponsor"></a>
+
+</div>
+
+### Silver Sponsors 🥈
+
+<div dir="auto">
+
+<a href="https://opencollective.com/rrweb/tiers/silver-sponsor/0/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/silver-sponsor/0/avatar.svg?requireActive=false&avatarHeight=158" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/silver-sponsor/1/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/silver-sponsor/1/avatar.svg?requireActive=false&avatarHeight=158" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/silver-sponsor/2/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/silver-sponsor/2/avatar.svg?requireActive=false&avatarHeight=158" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/silver-sponsor/3/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/silver-sponsor/3/avatar.svg?requireActive=false&avatarHeight=158" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/silver-sponsor/4/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/silver-sponsor/4/avatar.svg?requireActive=false&avatarHeight=158" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/silver-sponsor/5/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/silver-sponsor/5/avatar.svg?requireActive=false&avatarHeight=158" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/silver-sponsor/6/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/silver-sponsor/6/avatar.svg?requireActive=false&avatarHeight=158" alt="sponsor"></a>
+
+</div>
+
+### Bronze Sponsors 🥉
+
+<div dir="auto">
+
+<a href="https://opencollective.com/rrweb/tiers/sponsors/0/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/sponsors/0/avatar.svg?requireActive=false&avatarHeight=70" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/sponsors/1/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/sponsors/1/avatar.svg?requireActive=false&avatarHeight=70" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/sponsors/2/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/sponsors/2/avatar.svg?requireActive=false&avatarHeight=70" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/sponsors/3/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/sponsors/3/avatar.svg?requireActive=false&avatarHeight=70" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/sponsors/4/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/sponsors/4/avatar.svg?requireActive=false&avatarHeight=70" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/sponsors/5/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/sponsors/5/avatar.svg?requireActive=false&avatarHeight=70" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/sponsors/6/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/sponsors/6/avatar.svg?requireActive=false&avatarHeight=70" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/sponsors/7/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/sponsors/7/avatar.svg?requireActive=false&avatarHeight=70" alt="sponsor"></a>
+<a href="https://opencollective.com/rrweb/tiers/sponsors/8/website?requireActive=false" target="_blank"><img src="https://opencollective.com/rrweb/tiers/sponsors/8/avatar.svg?requireActive=false&avatarHeight=70" alt="sponsor"></a>
+
+</div>
+
+### Backers
+
+<a href="https://opencollective.com/rrweb#sponsor" rel="nofollow"><img src="https://opencollective.com/rrweb/tiers/backers.svg?avatarHeight=36"></a>
+
+## Core Team Members
+
+<table>
+  <tr>
+    <td align="center">
+      <a href="https://github.com/Yuyz0112">
+        <img
+          src="https://avatars.githubusercontent.com/u/13651389?s=100"
+          width="100px;"
+          alt=""
+        />
+        <br /><sub><b>Yuyz0112</b></sub>
+        <br /><br />
+      </a>
+    </td>
+    <td align="center">
+      <a href="https://github.com/YunFeng0817">
+        <img
+          src="https://avatars.githubusercontent.com/u/27533910?s=100"
+          width="100px;"
+          alt=""
+        />
+        <br /><sub><b>Yun Feng</b></sub>
+        <br /><br />
+      </a>
+    </td>
+    <td align="center">
+      <a href="https://github.com/eoghanmurray">
+        <img
+          src="https://avatars.githubusercontent.com/u/156780?s=100"
+          width="100px;"
+          alt=""
+        />
+        <br /><sub><b>eoghanmurray</b></sub>
+        <br /><br />
+      </a>
+    </td>
+    <td align="center">
+      <a href="https://github.com/Juice10">
+        <img
+          src="https://avatars.githubusercontent.com/u/4106?s=100"
+          width="100px;"
+          alt=""
+        />
+        <br /><sub><b>Juice10</b></sub>
+        <br /><sub>open for rrweb consulting</sub>
+      </a>
+    </td>
+  </tr>
+</table>
+
+## Who's using rrweb?
+
+<table>
+  <tr>
+    <td align="center">
+      <a href="http://www.smartx.com/" target="_blank">
+        <img width="195px" src="https://www.rrweb.io/logos/smartx.png">
+      </a>
+    </td>
+    <td align="center">
+      <a href="https://posthog.com?utm_source=rrweb&utm_medium=sponsorship&utm_campaign=open-source-sponsorship" target="_blank">
+        <img width="195px" src="https://www.rrweb.io/logos/posthog.png">
+      </a>
+    </td>
+    <td align="center">
+      <a href="https://statcounter.com/session-replay/" target="_blank">
+        <img width="195px" src="https://statcounter.com/images/logo-statcounter-arc-blue.svg">
+      </a>
+    </td>
+    <td align="center">
+      <a href="https://recordonce.com/" target="_blank">
+        <img width="195px" alt="Smart screen recording for SaaS" src="https://uploads-ssl.webflow.com/5f3d133183156245630d4446/5f3d1940abe8db8612c23521_Record-Once-logo-554x80px.svg">
+      </a>
+    </td>
+  </tr>
+    <tr>
+    <td align="center">
+      <a href="https://cux.io" target="_blank">
+        <img style="padding: 8px" alt="The first ever UX automation tool" width="195px" src="https://cux.io/cux-logo.svg">
+      </a>
+    </td>
+    <td align="center">
+      <a href="https://remsupp.com" target="_blank">
+        <img style="padding: 8px" alt="Remote Access & Co-Browsing" width="195px" src="https://remsupp.com/images/logo.png">
+      </a>
+    </td>
+    <td align="center">
+      <a href="https://highlight.io" target="_blank">
+        <img style="padding: 8px" alt="The open source, fullstack Monitoring Platform." width="195px" src="https://github.com/highlight/highlight/raw/main/highlight.io/public/images/logo.png">
+      </a>
+    </td>
+    <td align="center">
+      <a href="https://analyzee.io" target="_blank">
+        <img style="padding: 8px" alt="Comprehensive data analytics platform that empowers businesses to gain valuable insights and make data-driven decisions." width="195px" src="https://cdn.analyzee.io/assets/analyzee-logo.png">
+      </a>
+    </td>
+  </tr>
+  <tr>
+    <td align="center">
+      <a href="https://requestly.io" target="_blank">
+        <img style="padding: 8px" alt="Intercept, Modify, Record & Replay HTTP Requests." width="195px" src="https://github.com/requestly/requestly/assets/16779465/652552db-c867-44cb-9bb5-94a2026e04ca">
+      </a>
+    </td>
+    <td align="center">
+      <a href="https://gleap.io" target="_blank">
+        <img style="padding: 8px" alt="In-app bug reporting & customer feedback platform." width="195px" src="https://assets-global.website-files.com/6506f3f29c68b1724807619d/6506f56010237164c6306591_GleapLogo.svg">
+      </a>
+    </td>
+    <td align="center">
+      <a href="https://uxwizz.com" target="_blank">
+        <img style="padding: 8px" alt="Self-hosted website analytics with heatmaps and session recordings." width="195px" src="https://github.com/UXWizz/public-files/raw/main/assets/logo.png">
+      </a>
+    </td>
+    <td align="center">
+      <a href="https://www.howdygo.com" target="_blank">
+        <img style="padding: 8px" alt="Interactive product demos for small marketing teams" width="195px" src="https://assets-global.website-files.com/650afb446f1dd5bd410f00cc/650b2cec6188ff54dd9b01e1_Logo.svg">
+      </a>
+    </td>
+  </tr>
+</table>