Skip to content

Commit 3178ec1

Browse files
rickzxrickzx
authored
Add a new Chrome extension example using WebGPU and service worker (mlc-ai#296)
- Chrome has added WebGPU support in Service Worker in this [commit](https://chromium-review.googlesource.com/c/chromium/src/+/5190750). This example shows how we can create a Chrome extension using WebGPU and service worker. The project structure is as follows: - `manifest.json`: A required file that lists important information about the structure and behavior of that extension. Here we are using manifest V3. - `popup.ts`: Script of the extension pop-up window. - `background.ts`: Script of the service worker. An extension service worker is loaded when it is needed, and unloaded when it goes dormant. - `content.js`: Content script that interacts with DOM. - To run the extension, first make sure you are on [Google Chrome Canary](https://www.google.com/chrome/canary/). - In Chrome Canary, go to `chrome://flags/#enable-experimental-web-platform-features` and enable the `#enable-experimental-web-platform-features` flag. **Relaunch the browser**. - Run ```bash npm install npm run build ``` This will create a new directory at `./dist/`. To load the extension into Chrome, go to Extensions > Manage Extensions and select Load Unpacked. Add the `./dist/` directory. You can now pin the extension to your toolbar and use it to chat with your favorite model! **Note**: This example disables chatting using the contents of the active tab by default. To enable it, set `useContext` in `popup.ts` to `true`. More info about this feature can be found [here](mlc-ai#190). However, if the web content is too large, it might run into issues. We recommend using `example.html` to test this feature.
1 parent ec2662f commit 3178ec1

File tree

14 files changed

+572
-1
lines changed

14 files changed

+572
-1
lines changed

‎examples/chrome-extension-webgpu-service-worker/README.md

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# WebLLM Chrome Extension using WebGPU Running on Service Worker
2+
3+
![Chrome Extension](https://github.com/mlc-ai/mlc-llm/assets/11940172/0d94cc73-eff1-4128-a6e4-70dc879f04e0)
4+
5+
- Chrome has added WebGPU support in Service Worker in this [commit](https://chromium-review.googlesource.com/c/chromium/src/+/5190750). This example shows how we can create a Chrome extension using WebGPU and service worker.
6+
The project structure is as follows:
7+
- `manifest.json`: A required file that lists important information about the structure and behavior of that extension. Here we are using manifest V3.
8+
- `popup.ts`: Script of the extension pop-up window.
9+
- `background.ts`: Script of the service worker. An extension service worker is loaded when it is needed, and unloaded when it goes dormant.
10+
- `content.js`: Content script that interacts with DOM.
11+
- To run the extension, first make sure you are on [Google Chrome Canary](https://www.google.com/chrome/canary/).
12+
- In Chrome Canary, go to `chrome://flags/#enable-experimental-web-platform-features` and enable the `#enable-experimental-web-platform-features` flag. **Relaunch the browser**.
13+
- Run
14+
```bash
15+
npm install
16+
npm run build
17+
```
18+
19+
This will create a new directory at `./dist/`. To load the extension into Chrome, go to Extensions > Manage Extensions and select Load Unpacked. Add the `./dist/` directory. You can now pin the extension to your toolbar and use it to chat with your favorite model!
20+
21+
**Note**: This example disables chatting using the contents of the active tab by default.
22+
To enable it, set `useContext` in `popup.ts` to `true`. More info about this feature can be found
23+
[here](https://github.com/mlc-ai/web-llm/pull/190).
24+
However, if the web content is too large, it might run into issues. We recommend using `example.html` to
25+
test this feature.

‎examples/chrome-extension-webgpu-service-worker/package.json

+23
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
{
2+
"name": "chrome-extension",
3+
"version": "1.0.0",
4+
"description": "",
5+
"private": true,
6+
"scripts": {
7+
"build": "parcel build src/manifest.json --config @parcel/config-webextension"
8+
},
9+
"author": "",
10+
"license": "ISC",
11+
"devDependencies": {
12+
"@parcel/config-webextension": "^2.9.3",
13+
"@types/chrome": "^0.0.242",
14+
"buffer": "^6.0.3",
15+
"parcel": "^2.9.3",
16+
"process": "^0.11.10",
17+
"url": "^0.11.1"
18+
},
19+
"dependencies": {
20+
"@mlc-ai/web-llm": "^0.2.18",
21+
"progressbar.js": "^1.1.0"
22+
}
23+
}

‎examples/chrome-extension-webgpu-service-worker/src/background.ts

+52
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
import {ChatRestModule, ChatInterface, ChatModule, InitProgressReport} from "@mlc-ai/web-llm";
2+
3+
// TODO: Surface this as an option to the user
4+
const useWebGPU = true;
5+
var model_loaded = false;
6+
7+
var cm: ChatInterface;
8+
if (!useWebGPU) {
9+
cm = new ChatRestModule();
10+
} else {
11+
cm = new ChatModule();
12+
}
13+
14+
// Set reponse callback for chat module
15+
const generateProgressCallback = (_step: number, message: string) => {
16+
// send the answer back to the content script
17+
chrome.runtime.sendMessage({ answer: message });
18+
};
19+
20+
var context = "";
21+
chrome.runtime.onMessage.addListener(async function (request) {
22+
// check if the request contains a message that the user sent a new message
23+
if (request.input) {
24+
var inp = request.input;
25+
if (context.length > 0) {
26+
inp = "Use only the following context when answering the question at the end. Don't use any other knowledge.\n"+ context + "\n\nQuestion: " + request.input + "\n\nHelpful Answer: ";
27+
}
28+
console.log("Input:", inp);
29+
const response = await cm.generate(inp, generateProgressCallback);
30+
}
31+
if (request.context) {
32+
context = request.context;
33+
console.log("Got context:", context);
34+
}
35+
if (request.reload) {
36+
if (!model_loaded) {
37+
var appConfig = request.reload;
38+
console.log("Got appConfig: ", appConfig);
39+
40+
cm.setInitProgressCallback((report: InitProgressReport) => {
41+
console.log(report.text, report.progress);
42+
chrome.runtime.sendMessage({ initProgressReport: report.progress});
43+
});
44+
45+
await cm.reload("Mistral-7B-Instruct-v0.2-q4f16_1", undefined, appConfig);
46+
console.log("Model loaded");
47+
model_loaded = true;
48+
} else {
49+
chrome.runtime.sendMessage({ initProgressReport: 1.0});
50+
}
51+
}
52+
});

‎examples/chrome-extension-webgpu-service-worker/src/content.js

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
// Only the content script is able to access the DOM
2+
chrome.runtime.onConnect.addListener(function(port) {
3+
port.onMessage.addListener(function(msg) {
4+
port.postMessage({contents: document.body.innerHTML});
5+
});
6+
});

‎examples/chrome-extension-webgpu-service-worker/src/example.html

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
In the year 2154, humanity had colonized several planets in the distant reaches of the galaxy. The planet of Xylophia-IV was one of the most remote and inhospitable, with temperatures often dropping to -200 degrees Celsius. Despite these harsh conditions, a team of scientists had established a research station on the planet to study the unique geological formations and exotic flora and fauna.
2+
3+
One day, while conducting a routine survey of the planet's surface, the team discovered an strange object buried deep in the ice. As they examined it closer, they realized it was a small, metallic capsule with a glowing blue symbol etched onto its surface.
4+
5+
The team's leader, a brilliant scientist named Dr. Maria Rodriguez, was immediately intrigued by the capsule's mysterious origins. She ordered her team to bring it back to the research station for further analysis.
6+
7+
After weeks of studying the capsule, the team finally cracked the code to the symbol etched onto its surface. It was a message from an alien race, warning Earth of an impending attack from an unknown threat.
8+
9+
The team was shocked and dismayed by the news, but they knew they had to act quickly to warn the rest of humanity. They transmitted the message to the nearest space station, which relayed it to Earth's government.
10+
11+
As the threat of attack loomed near, the team remained on high alert, ready to face whatever dangers lay ahead. They had uncovered a secrets of the universe, and now they were determined to protect their planet and its inhabitants at all costs.

‎examples/chrome-extension-webgpu-service-worker/src/icons/icon-128.png

3.71 KB
Loading

‎examples/chrome-extension-webgpu-service-worker/src/icons/icon-16.png

581 Bytes
Loading

‎examples/chrome-extension-webgpu-service-worker/src/icons/icon-32.png

1.03 KB
Loading

‎examples/chrome-extension-webgpu-service-worker/src/icons/icon-64.png

1.93 KB
Loading

‎examples/chrome-extension-webgpu-service-worker/src/manifest.json

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
{
2+
"manifest_version": 3,
3+
"name": "MLCBot",
4+
"version": "0.1.0",
5+
"description": "Chat with your browser",
6+
"icons": {
7+
"16": "icons/icon-16.png",
8+
"32": "icons/icon-32.png",
9+
"64": "icons/icon-64.png",
10+
"128": "icons/icon-128.png"
11+
},
12+
"content_security_policy": {
13+
"extension_pages": "style-src-elem 'self' https://cdnjs.cloudflare.com; font-src 'self' https://cdnjs.cloudflare.com; script-src 'self' 'wasm-unsafe-eval'; default-src 'self' data:; connect-src 'self' data: http://localhost:8000 https://huggingface.co https://cdn-lfs.huggingface.co https://cdn-lfs-us-1.huggingface.co https://raw.githubusercontent.com"
14+
},
15+
"action": {
16+
"default_title": "MLCBot",
17+
"default_popup": "popup.html"
18+
},
19+
"content_scripts": [
20+
{
21+
"matches": ["<all_urls>"],
22+
"js": ["content.js"]
23+
}
24+
],
25+
"background": {
26+
"service_worker": "background.ts",
27+
"type": "module"
28+
},
29+
"permissions": [
30+
"storage",
31+
"tabs",
32+
"webNavigation"
33+
]
34+
}

‎examples/chrome-extension-webgpu-service-worker/src/popup.css

+235
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,235 @@
1+
*,
2+
*::before,
3+
*::after {
4+
margin: 0;
5+
padding: 0;
6+
box-sizing: border-box;
7+
}
8+
9+
html {
10+
font-family: -apple-system, BlinkMacSystemFont, Segoe UI, Helvetica, Arial,
11+
sans-serif;
12+
color: #222;
13+
}
14+
15+
body {
16+
margin: 0;
17+
padding: 0.5rem;
18+
background-color: #778da9;
19+
width: 320px;
20+
font-size: small;
21+
}
22+
23+
p {
24+
margin: 0;
25+
}
26+
27+
/* LOADING BAR */
28+
#loadingContainer {
29+
margin-bottom: 15px;
30+
width: 300px;
31+
height: 8px;
32+
}
33+
34+
/* INPUT AREA */
35+
#query-input {
36+
border: 1px solid #ccc;
37+
border-radius: 4px;
38+
}
39+
40+
.input-container {
41+
display: flex;
42+
flex-direction: row;
43+
align-items: center;
44+
}
45+
46+
.input-container input {
47+
width: 100%;
48+
outline: none;
49+
padding: 0.5rem;
50+
margin-right: 0.5rem;
51+
}
52+
53+
/* SUBMIT BUTTON */
54+
.btn {
55+
background-color: #1b263b;
56+
color: white;
57+
font-size: small;
58+
cursor: pointer;
59+
border-radius: 4px;
60+
border: none;
61+
padding: 0.5rem;
62+
}
63+
64+
.btn:hover {
65+
background-color: #d0d0d0;
66+
}
67+
68+
.btn:disabled {
69+
background-color: #a7a7a7;
70+
color: rgb(255, 255, 255);
71+
cursor: default;
72+
}
73+
74+
.btn img {
75+
width: 1rem;
76+
height: 1rem;
77+
}
78+
79+
/* LOADING */
80+
81+
.stage {
82+
display: flex;
83+
justify-content: center;
84+
align-items: center;
85+
position: relative;
86+
margin: 0 -5%;
87+
overflow: hidden;
88+
}
89+
90+
#loading-indicator {
91+
display: none;
92+
color: white;
93+
margin-top: 0.5rem;
94+
}
95+
96+
.dot-flashing {
97+
position: relative;
98+
width: 10px;
99+
height: 10px;
100+
border-radius: 5px;
101+
background-color: #1b263b;
102+
color: #1b263b;
103+
animation: dot-flashing 0.4s infinite linear alternate;
104+
animation-delay: 0.2s;
105+
}
106+
107+
.dot-flashing::before,
108+
.dot-flashing::after {
109+
content: "";
110+
display: inline-block;
111+
position: absolute;
112+
top: 0;
113+
}
114+
115+
.dot-flashing::before {
116+
left: -15px;
117+
width: 10px;
118+
height: 10px;
119+
border-radius: 5px;
120+
background-color: #1b263b;
121+
color: #1b263b;
122+
animation: dot-flashing 0.4s infinite alternate;
123+
animation-delay: 0s;
124+
}
125+
126+
.dot-flashing::after {
127+
left: 15px;
128+
width: 10px;
129+
height: 10px;
130+
border-radius: 5px;
131+
background-color: #1b263b;
132+
color: #1b263b;
133+
animation: dot-flashing 0.4s infinite alternate;
134+
animation-delay: 0.4s;
135+
}
136+
137+
@keyframes dot-flashing {
138+
0% {
139+
background-color: #1b263b;
140+
}
141+
142+
50%,
143+
100% {
144+
background-color: #415a77;
145+
}
146+
}
147+
148+
/* ANSWERS */
149+
#queriesAnswersContainer {
150+
display: block;
151+
color: white;
152+
margin-top: 0.5rem;
153+
}
154+
155+
#answer {
156+
color: #333333;
157+
}
158+
159+
#answerWrapper {
160+
display: none;
161+
background-color: #ffd166;
162+
border-radius: 8px;
163+
padding: 0.5rem;
164+
margin-top: 0.5rem;
165+
}
166+
167+
.queriesAnswers {
168+
border-radius: 8px;
169+
background-color: #ffd166;;
170+
padding: 0.5rem;
171+
color: #333333;
172+
}
173+
174+
#lastQuery {
175+
color: rgb(188, 188, 188);
176+
}
177+
178+
#lastAnswer {
179+
color: white;
180+
margin-top: 0.5rem;
181+
}
182+
183+
#lastRequest {
184+
padding: 0.5rem;
185+
margin-top: 0.5rem;
186+
background-color: #333333;
187+
border-radius: 4px;
188+
}
189+
190+
/* ANSWER OPTIONS */
191+
.timeStamp {
192+
color: #9a8c98;
193+
}
194+
195+
.copyRow {
196+
display: flex;
197+
flex-direction: row;
198+
align-items: end;
199+
justify-content: space-between;
200+
color: #a7a7a7;
201+
margin-top: 0.5rem;
202+
}
203+
204+
.copyText {
205+
display: none;
206+
color: #a7a7a7;
207+
margin-right: 0.5rem;
208+
}
209+
210+
.copyButton {
211+
color: #415a77;
212+
background-color: transparent;
213+
border: none;
214+
cursor: pointer;
215+
padding: 0;
216+
margin-left: 0.5rem;
217+
}
218+
219+
.copyButton:hover {
220+
color: #5e80a7;
221+
background-color: transparent;
222+
}
223+
224+
.removeButton {
225+
color: #415a77;
226+
background-color: transparent;
227+
border: none;
228+
cursor: pointer;
229+
padding: 0;
230+
}
231+
232+
.removeButton:hover {
233+
color: #5e80a7;
234+
background-color: transparent;
235+
}

0 commit comments

Comments
 (0)