add cross site attack #3

TomCJones · 2024-10-16T19:49:04Z

No description provided.

TallTed

If both models/AI in the Browser.md and models/ai-in-browser.md are intended to be retained, their content should be the same. It seems more likely that the content should be made the same, and then one document dropped.

TallTed · 2024-10-16T21:30:38Z

models/AI in the Browser.md

@@ -0,0 +1,55 @@
+# AI in the Browser
+
+Artificial Intelligence (aka LLM) is getting added to everything, including the Web Browser, which will have some severe unanticipated downside for the user. 


Suggested change

Artificial Intelligence (aka LLM) is getting added to everything, including the Web Browser, which will have some severe unanticipated downside for the user.

Artificial Intelligence (AI), today in the form of Large Language Models (LLMs), is getting added to everything, including the Web Browser, which will have some severe unanticipated downsides for the user.

TallTed · 2024-10-16T21:32:38Z

models/AI in the Browser.md

+
+## Context
+
+Google on Chromium and others in the W3C have been trying to make web apps that are downloaded from web sites, as attractive and useful as native apps, that are downloaded from the app store. Now that AI access is getting added to the browser it is important to look at the impact on the user. The following is a quote from the introduction of one API into Chromium. We can expect more APIs enabling access to AI soon.


Suggested change

Google on Chromium and others in the W3C have been trying to make web apps that are downloaded from web sites, as attractive and useful as native apps, that are downloaded from the app store. Now that AI access is getting added to the browser it is important to look at the impact on the user. The following is a quote from the introduction of one API into Chromium. We can expect more APIs enabling access to AI soon.

Google (via Chromium) and others in the W3C have been trying to make web apps, that are downloaded from web sites, as attractive and useful as native apps, that are downloaded from app stores. As AI access is getting added to the browser, it is important to look at the impact on the user. The following is a quote from the addition of one AI API into Chromium. We can expect more APIs enabling access to AI soon.

TallTed · 2024-10-16T21:34:53Z

models/AI in the Browser.md

+
+Google on Chromium and others in the W3C have been trying to make web apps that are downloaded from web sites, as attractive and useful as native apps, that are downloaded from the app store. Now that AI access is getting added to the browser it is important to look at the impact on the user. The following is a quote from the introduction of one API into Chromium. We can expect more APIs enabling access to AI soon.
+
+Browsers and operating systems are increasingly expected to gain access to a language model. By exposing this built-in model, we avoid every website needing to download their own multi-gigabyte language model, or send input text to third-party APIs. The rewriter API in particular exposes a high-level API for interfacing with a language model in order to transform inputs for a variety of use cases, in a way that does not depend on the specific language model in question. [https://github.com/explainers-by-googlers/writing-assistance-apis/blob/main/README.md\#rewriter-api](https://github.com/explainers-by-googlers/writing-assistance-apis/blob/main/README.md#rewriter-api)


Suggested change

Browsers and operating systems are increasingly expected to gain access to a language model. By exposing this built-in model, we avoid every website needing to download their own multi-gigabyte language model, or send input text to third-party APIs. The rewriter API in particular exposes a high-level API for interfacing with a language model in order to transform inputs for a variety of use cases, in a way that does not depend on the specific language model in question. [https://github.com/explainers-by-googlers/writing-assistance-apis/blob/main/README.md\#rewriter-api](https://github.com/explainers-by-googlers/writing-assistance-apis/blob/main/README.md#rewriter-api)

Browsers and operating systems are increasingly expected to gain access to an LLM. By exposing this built-in model, we avoid the need to download a multi-gigabyte language model for each website, or to send input text to third-party APIs. In particular, [the rewriter API](https://github.com/explainers-by-googlers/writing-assistance-apis/blob/main/README.md#rewriter-api) exposes a high-level API for interfacing with a language model in order to transform inputs for a variety of use cases, in a way that does not depend on the specific language model in question.

TallTed · 2024-10-16T21:46:06Z

models/AI in the Browser.md

+
+## Vulnerabilities
+
+These all arise from providing the website with nearly complete control of what JavaScript runs whenever their page is activated. The above API does include the following language "Finally, we intend to prohibit (in the specification) any use of user-specific information that is not directly supplied through the API. For example, it would not be permissible to fine-tune the language model based on information the user has entered into the browser in the past." The problem here is that the browser does not have control of the LLM  that is provided to the browser or whether the user has provided personal information to that LLM by interactions outside of the browser. The LLM (or other AI) envisioned here is provided in yet another user agent in the user device completely independent of the browser and used by other functions running in the device.


Suggested change

These all arise from providing the website with nearly complete control of what JavaScript runs whenever their page is activated. The above API does include the following language "Finally, we intend to prohibit (in the specification) any use of user-specific information that is not directly supplied through the API. For example, it would not be permissible to fine-tune the language model based on information the user has entered into the browser in the past." The problem here is that the browser does not have control of the LLM that is provided to the browser or whether the user has provided personal information to that LLM by interactions outside of the browser. The LLM (or other AI) envisioned here is provided in yet another user agent in the user device completely independent of the browser and used by other functions running in the device.

A number of vulnerabilities arise from providing websites with nearly complete control of the JavaScript that runs whenever their pages are activated. The rewriter API does include the following language: "Finally, we intend to prohibit (in the specification) any use of user-specific information that is not directly supplied through the API. For example, it would not be permissible to fine-tune the language model based on information the user has entered into the browser in the past." There are problems here, in that the browser does not have control of the LLM that is provided to the browser nor whether the user has provided personal information to that LLM by interactions outside of the browser. The LLM (or other AI) envisioned here is provided as yet another user agent in the user's device, completely independent of the browser, and used by other functions running in the device.

TallTed · 2024-10-17T14:11:44Z

models/AI in the Browser.md

+
+**User Profiling**
+
+The web site will be able to ask the AI loaded on the user's device for a UI that would match what the user would see as the local AI is used in that personal user device. Trying different responses to the same user (via the local AI agent) would give the website information about the user's preferences and behavior. Clearly this is a way to avoid asking the user’s consent to share information by trying to extract it from the user's AI without the user's permission or knowledge. 


Suggested change

The web site will be able to ask the AI loaded on the user's device for a UI that would match what the user would see as the local AI is used in that personal user device. Trying different responses to the same user (via the local AI agent) would give the website information about the user's preferences and behavior. Clearly this is a way to avoid asking the user’s consent to share information by trying to extract it from the user's AI without the user's permission or knowledge.

A web site will be able to ask the AI loaded on the user's device for a UI that would match what the user would see when the local AI is used in that personal user device. Trying different responses to the same user (via the local AI agent) would give the website information about the user's preferences and behavior. This is a clear way to avoid asking the user’s consent to share information by trying to extract it directly from the user's AI, without the user's permission or knowledge.

TallTed · 2024-10-17T14:21:59Z

models/AI in the Browser.md

+
+**Provisioning Malware**
+
+The supply chain that can be attacked includes the AI (LLM) module within the device. It is assumed that there may be multiple AI modules in the future, some of uncertain provenance.  It is not at all clear why the browser API should trust the LLM provided.


Suggested change

The supply chain that can be attacked includes the AI (LLM) module within the device. It is assumed that there may be multiple AI modules in the future, some of uncertain provenance. It is not at all clear why the browser API should trust the LLM provided.

The supply chain that can be attacked includes the AI (LLM) module within the device. It is assumed that there may be multiple AI modules in the future, some of uncertain provenance. It is not at all clear that the browser API should trust the local LLM, simply because it is local.

TallTed · 2024-10-17T14:27:03Z

models/AI in the Browser.md

+There is a current set of vulnerabilities for caching today that are being addressed by mitigations described in the feature listed below. Any cross-site vulnerability found there could equally apply to shared use of a user’s local AI not only within the browser but by any other app on the user’s device.
+
+See the Feature: [Incorporating navigation initiator into the HTTP cache partition key](https://chromestatus.com/feature/5190577638080512) 
+and [the slide deck](https://docs.google.com/presentation/d/1StMrI1hNSw_QSmR7bg0w3WcIoYnYIt5K8G2fG01O0IA/edit#slide=id.g2f87bb2d5eb_0_4)
+## Mitigations


Suggested change

There is a current set of vulnerabilities for caching today that are being addressed by mitigations described in the feature listed below. Any cross-site vulnerability found there could equally apply to shared use of a user’s local AI not only within the browser but by any other app on the user’s device.

See the Feature: [Incorporating navigation initiator into the HTTP cache partition key](https://chromestatus.com/feature/5190577638080512)

and [the slide deck](https://docs.google.com/presentation/d/1StMrI1hNSw_QSmR7bg0w3WcIoYnYIt5K8G2fG01O0IA/edit#slide=id.g2f87bb2d5eb_0_4)

## Mitigations

There is a current set of vulnerabilities in caching that are being addressed today by mitigations in [incorporating navigation initiator into the HTTP cache partition key](https://chromestatus.com/feature/5190577638080512) (also see [the associated slide deck](https://docs.google.com/presentation/d/1StMrI1hNSw_QSmR7bg0w3WcIoYnYIt5K8G2fG01O0IA/edit#slide=id.g2f87bb2d5eb_0_4)). Any cross-site vulnerability found there could equally apply to shared use of a user’s local AI, not only within the browser, but also by any other app on the user’s device.

## Mitigations

TallTed · 2024-10-17T14:29:04Z

models/AI in the Browser.md

+
+**AI Isolation**
+
+Only AI that has no interaction with the device holder may be accessed by any user agent that hosts pages from a web site that is not fully trusted by the holder or device owner. Specifically, the impact of the prompts entered by an origin site should not be able to impact either the holder or other origin site’s interactions with the holder.


Suggested change

Only AI that has no interaction with the device holder may be accessed by any user agent that hosts pages from a web site that is not fully trusted by the holder or device owner. Specifically, the impact of the prompts entered by an origin site should not be able to impact either the holder or other origin site’s interactions with the holder.

Only AI that has no interaction with the device holder may be accessed by any user agent that hosts pages from a web site that is not fully trusted by the holder or device owner. Specifically, the impact of the prompts entered by an origin site should not be able to impact either the holder or another origin site’s interactions with the holder.

TallTed · 2024-10-17T14:29:41Z

models/AI in the Browser.md

+
+**Throttling**
+
+Particularly for battery operated devices, the amount of power allocated to any one origin must be limited. This could be part of a setting that the holder or device owner was permitted to change based on trusted origins.


Suggested change

Particularly for battery operated devices, the amount of power allocated to any one origin must be limited. This could be part of a setting that the holder or device owner was permitted to change based on trusted origins.

Particularly for battery operated devices, the amount of power allocated to any one origin must be limited. This could be part of a setting that the holder or device owner is permitted to change based on trusted origins.

TallTed · 2024-10-17T14:31:04Z

models/AI in the Browser.md

+
+## References
+
+  Bruce Schneier, LLM's Data-Control Path Insecurity CACM 67 No 9 page 31-32 downloaded from [LLMs’ Data-Control Path Insecurity – Communications of the ACM](https://cacm.acm.org/opinion/llms-data-control-path-insecurity/)


Suggested change

Bruce Schneier, LLM's Data-Control Path Insecurity CACM 67 No 9 page 31-32 downloaded from [LLMs’ Data-Control Path Insecurity – Communications of the ACM](https://cacm.acm.org/opinion/llms-data-control-path-insecurity/)

* Bruce Schneier, LLMs' Data-Control Path Insecurity CACM 67 No 9 page 31-32 downloaded from [LLMs’ Data-Control Path Insecurity – Communications of the ACM](https://cacm.acm.org/opinion/llms-data-control-path-insecurity/)

simoneonofri

LTGM thanks @TomCJones and @TallTed

@TomCJones let me know the file you would like to maintain.

It can also be interesting to create (later) a table to map threats and mitigations such as the one made with @KimCerra

w3c/security-request#71 (comment)

simoneonofri · 2024-10-17T16:39:49Z

models/AI in the Browser.md

@@ -0,0 +1,55 @@
+# AI in the Browser
+
+Artificial Intelligence (aka LLM) is getting added to everything, including the Web Browser, which will have some severe unanticipated downside for the user. 


simoneonofri · 2024-10-17T16:40:05Z

models/AI in the Browser.md

+
+## Context
+
+Google on Chromium and others in the W3C have been trying to make web apps that are downloaded from web sites, as attractive and useful as native apps, that are downloaded from the app store. Now that AI access is getting added to the browser it is important to look at the impact on the user. The following is a quote from the introduction of one API into Chromium. We can expect more APIs enabling access to AI soon.


simoneonofri · 2024-10-17T16:40:42Z

models/AI in the Browser.md

+
+Google on Chromium and others in the W3C have been trying to make web apps that are downloaded from web sites, as attractive and useful as native apps, that are downloaded from the app store. Now that AI access is getting added to the browser it is important to look at the impact on the user. The following is a quote from the introduction of one API into Chromium. We can expect more APIs enabling access to AI soon.
+
+Browsers and operating systems are increasingly expected to gain access to a language model. By exposing this built-in model, we avoid every website needing to download their own multi-gigabyte language model, or send input text to third-party APIs. The rewriter API in particular exposes a high-level API for interfacing with a language model in order to transform inputs for a variety of use cases, in a way that does not depend on the specific language model in question. [https://github.com/explainers-by-googlers/writing-assistance-apis/blob/main/README.md\#rewriter-api](https://github.com/explainers-by-googlers/writing-assistance-apis/blob/main/README.md#rewriter-api)


simoneonofri · 2024-10-17T16:40:51Z

models/AI in the Browser.md

+
+## Vulnerabilities
+
+These all arise from providing the website with nearly complete control of what JavaScript runs whenever their page is activated. The above API does include the following language "Finally, we intend to prohibit (in the specification) any use of user-specific information that is not directly supplied through the API. For example, it would not be permissible to fine-tune the language model based on information the user has entered into the browser in the past." The problem here is that the browser does not have control of the LLM  that is provided to the browser or whether the user has provided personal information to that LLM by interactions outside of the browser. The LLM (or other AI) envisioned here is provided in yet another user agent in the user device completely independent of the browser and used by other functions running in the device.


simoneonofri · 2024-10-17T16:40:59Z

models/AI in the Browser.md

+
+**User Profiling**
+
+The web site will be able to ask the AI loaded on the user's device for a UI that would match what the user would see as the local AI is used in that personal user device. Trying different responses to the same user (via the local AI agent) would give the website information about the user's preferences and behavior. Clearly this is a way to avoid asking the user’s consent to share information by trying to extract it from the user's AI without the user's permission or knowledge. 


simoneonofri · 2024-10-17T16:41:36Z

models/AI in the Browser.md

+
+**Provisioning Malware**
+
+The supply chain that can be attacked includes the AI (LLM) module within the device. It is assumed that there may be multiple AI modules in the future, some of uncertain provenance.  It is not at all clear why the browser API should trust the LLM provided.


simoneonofri · 2024-10-17T16:41:51Z

models/AI in the Browser.md

+There is a current set of vulnerabilities for caching today that are being addressed by mitigations described in the feature listed below. Any cross-site vulnerability found there could equally apply to shared use of a user’s local AI not only within the browser but by any other app on the user’s device.
+
+See the Feature: [Incorporating navigation initiator into the HTTP cache partition key](https://chromestatus.com/feature/5190577638080512) 
+and [the slide deck](https://docs.google.com/presentation/d/1StMrI1hNSw_QSmR7bg0w3WcIoYnYIt5K8G2fG01O0IA/edit#slide=id.g2f87bb2d5eb_0_4)
+## Mitigations


simoneonofri · 2024-10-17T16:41:58Z

models/AI in the Browser.md

+
+**AI Isolation**
+
+Only AI that has no interaction with the device holder may be accessed by any user agent that hosts pages from a web site that is not fully trusted by the holder or device owner. Specifically, the impact of the prompts entered by an origin site should not be able to impact either the holder or other origin site’s interactions with the holder.


simoneonofri · 2024-10-17T16:42:03Z

models/AI in the Browser.md

+
+**Throttling**
+
+Particularly for battery operated devices, the amount of power allocated to any one origin must be limited. This could be part of a setting that the holder or device owner was permitted to change based on trusted origins.


simoneonofri · 2024-10-17T16:42:11Z

models/AI in the Browser.md

+
+## References
+
+  Bruce Schneier, LLM's Data-Control Path Insecurity CACM 67 No 9 page 31-32 downloaded from [LLMs’ Data-Control Path Insecurity – Communications of the ACM](https://cacm.acm.org/opinion/llms-data-control-path-insecurity/)


simoneonofri · 2024-10-27T12:41:37Z

@TallTed thank you for your commits

@TomCJones thank you in general for the analysis

TomCJones added 5 commits October 9, 2024 14:48

Add files via upload

4e9bfa8

Rename AI in the Browser 10-09.md to AI in the Browser.md

96a90ef

add slide deck link

43535cf

Merge branch 'w3c-cg:main' into main

91c3cb2

add cross-site attack

3d5b8f0

TallTed suggested changes Oct 17, 2024

View reviewed changes

simoneonofri self-requested a review October 17, 2024 16:37

simoneonofri approved these changes Oct 17, 2024

View reviewed changes

Delete models/AI in the Browser.md

d8fe074

simoneonofri approved these changes Oct 27, 2024

View reviewed changes

simoneonofri merged commit 6b93b00 into w3c-cg:main Oct 27, 2024
1 check passed

		@@ -0,0 +1,55 @@
		# AI in the Browser

		Artificial Intelligence (aka LLM) is getting added to everything, including the Web Browser, which will have some severe unanticipated downside for the user.

	Artificial Intelligence (aka LLM) is getting added to everything, including the Web Browser, which will have some severe unanticipated downside for the user.
	Artificial Intelligence (AI), today in the form of Large Language Models (LLMs), is getting added to everything, including the Web Browser, which will have some severe unanticipated downsides for the user.


		## Context

		Google on Chromium and others in the W3C have been trying to make web apps that are downloaded from web sites, as attractive and useful as native apps, that are downloaded from the app store. Now that AI access is getting added to the browser it is important to look at the impact on the user. The following is a quote from the introduction of one API into Chromium. We can expect more APIs enabling access to AI soon.


		Google on Chromium and others in the W3C have been trying to make web apps that are downloaded from web sites, as attractive and useful as native apps, that are downloaded from the app store. Now that AI access is getting added to the browser it is important to look at the impact on the user. The following is a quote from the introduction of one API into Chromium. We can expect more APIs enabling access to AI soon.

		Browsers and operating systems are increasingly expected to gain access to a language model. By exposing this built-in model, we avoid every website needing to download their own multi-gigabyte language model, or send input text to third-party APIs. The rewriter API in particular exposes a high-level API for interfacing with a language model in order to transform inputs for a variety of use cases, in a way that does not depend on the specific language model in question. [https://github.com/explainers-by-googlers/writing-assistance-apis/blob/main/README.md\#rewriter-api](https://github.com/explainers-by-googlers/writing-assistance-apis/blob/main/README.md#rewriter-api)


		## Vulnerabilities

		These all arise from providing the website with nearly complete control of what JavaScript runs whenever their page is activated. The above API does include the following language "Finally, we intend to prohibit (in the specification) any use of user-specific information that is not directly supplied through the API. For example, it would not be permissible to fine-tune the language model based on information the user has entered into the browser in the past." The problem here is that the browser does not have control of the LLM that is provided to the browser or whether the user has provided personal information to that LLM by interactions outside of the browser. The LLM (or other AI) envisioned here is provided in yet another user agent in the user device completely independent of the browser and used by other functions running in the device.


		User Profiling

		The web site will be able to ask the AI loaded on the user's device for a UI that would match what the user would see as the local AI is used in that personal user device. Trying different responses to the same user (via the local AI agent) would give the website information about the user's preferences and behavior. Clearly this is a way to avoid asking the user’s consent to share information by trying to extract it from the user's AI without the user's permission or knowledge.


		Provisioning Malware

		The supply chain that can be attacked includes the AI (LLM) module within the device. It is assumed that there may be multiple AI modules in the future, some of uncertain provenance. It is not at all clear why the browser API should trust the LLM provided.


		AI Isolation

		Only AI that has no interaction with the device holder may be accessed by any user agent that hosts pages from a web site that is not fully trusted by the holder or device owner. Specifically, the impact of the prompts entered by an origin site should not be able to impact either the holder or other origin site’s interactions with the holder.


		Throttling

		Particularly for battery operated devices, the amount of power allocated to any one origin must be limited. This could be part of a setting that the holder or device owner was permitted to change based on trusted origins.


		## References

		Bruce Schneier, LLM's Data-Control Path Insecurity CACM 67 No 9 page 31-32 downloaded from [LLMs’ Data-Control Path Insecurity – Communications of the ACM](https://cacm.acm.org/opinion/llms-data-control-path-insecurity/)

	Bruce Schneier, LLM's Data-Control Path Insecurity CACM 67 No 9 page 31-32 downloaded from [LLMs’ Data-Control Path Insecurity – Communications of the ACM](https://cacm.acm.org/opinion/llms-data-control-path-insecurity/)
	* Bruce Schneier, LLMs' Data-Control Path Insecurity CACM 67 No 9 page 31-32 downloaded from [LLMs’ Data-Control Path Insecurity – Communications of the ACM](https://cacm.acm.org/opinion/llms-data-control-path-insecurity/)

add cross site attack #3

add cross site attack #3

Uh oh!

Conversation

TomCJones commented Oct 16, 2024

Uh oh!

TallTed left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

simoneonofri left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

simoneonofri commented Oct 27, 2024

Uh oh!

Uh oh!