update ai_security_overview.md #82
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| @@ -175,9 +175,9 @@ The AI security matrix below (click to enlarge) shows all threats and risks, ord | |||
| The below diagram puts the controls in the AI Exchange into groups and places these groups in the right lifecycle with the corresponding threats. | |||
| [](/images/threatscontrols.png) | |||
| The groups of controls form a summary of how to address AI security (controls are in capitals): | |||
| 1. **AI Governance**: implement governance processes for AI risk, and include AI into your processes for information security and software lifecycle: | |||
There was a problem hiding this comment.
It's not just adding AI risks, it's adding the whole of AI in your lifecycle
There was a problem hiding this comment.
Addressed now by rewording the sentence.
| @@ -187,27 +187,34 @@ The groups of controls form a summary of how to address AI security (controls ar | |||
| >([MONITORUSE](/goto/monitoruse/), [MODELACCESSCONTROL](/goto/modelaccesscontrol/), [RATELIMIT](/goto/ratelimit/)) | |||
| - 2c Adopt **new** IT security controls: | |||
| >([CONFCOMPUTE](/goto/confcompute/), [MODELOBFUSCATION](/goto/modelobfuscation/), [PROMPTINPUTVALIDATION](/goto/promptinputvalidation/), [INPUTSEGREGATION](/goto/inputsegregation/)) | |||
| 3. Data scientists apply **data science security controls** risk-based : | |||
There was a problem hiding this comment.
Please revert. The actors are important and the risk-based is about the applying, not about the controls
There was a problem hiding this comment.
This comment isn't so clear. Could you please elaborate more on what changes you'd love to see?
| 1. **Identifying Risks**: Recognizing potential risks (Threats) that could impact the organization. See “Threat through use” section to identify potential risks (Threats). | ||
| 2. **Evaluating Risks by Estimating Likelihood and Impact**: To determine the severity of a risk, it is necessary to assess the probability of the risk occurring and evaluating the potential consequences should the risk materialize. Combining likelihood and impact to gauge the risk's overall severity. This is typically presented in the form of a heatmap. See below for further details. | ||
| 1. **Identifying Risks**: Recognizing potential risks that could impact the organization. See “Threat through use” section to identify potential risks. | ||
| 2. **Evaluating Risks by Estimating Likelihood and Impact**: To determine the severity of a risk, it is necessary to assess the probability of the risk occurring and evaluating the potential consequences should the risk materialize. Combining likelihood and impact to gauge the risk's overall severity. This is typically presented in the form of a heatmap. This is discuused in more detail in the sections that follow. |
| 3. **Deciding What to Do (Risk Treatment)**: Choosing an appropriate strategy to address the risk. These strategies include: Risk Mitigation, Transfer, Avoidance, or Acceptance. See below for further details. | ||
| 4. **Risk Communication and Monitoring**: Regularly sharing risk information with stakeholders to ensure awareness and support for risk management activities. Ensuring effective Risk Treatments are applied. This requires a Risk Register, a comprehensive list of risks and their attributes (e.g. severity, treatment plan, ownership, status, etc). See below for further details. | ||
| 4. **Risk Communication and Monitoring**: Regularly sharing risk information with stakeholders to ensure awareness and continuous support for risk management activities. Ensuring effective Risk Treatments are applied. This requires a Risk Register, a comprehensive list of risks and their attributes (e.g. severity, treatment plan, ownership, status, etc). This is discuused in more detail in the sections that follow. |
|
|
||
| Sometimes model training and running the model is deferred to a supplier. For generative AI, training is mostly performed by an external supplier given the cost of typically millions of dollars. Finetuning of generative AI is also not often performed by organizations given the cost of compute and the complexity involved. Some GenAI models can be obtained and run at your own premises. The reasons to do this can be lower cost (if is is an open source model), and the fact that sensitive input information does not have to be sent externally. A reason to use an externally hosted GenAI model can be the quality of the model. |
There was a problem hiding this comment.
Your're missing the point here. the training is performed by somebody else BECAUSE it is expensive
There was a problem hiding this comment.
The correction is in the edited portion. I have furthermore added the fact that training by a third party is expensive.
|
|
||
| If you use RAG: apply the above to your repository data, as if it was part of the training set: as the repository data feeds into the model and can therefore be part of the output as well. |
There was a problem hiding this comment.
please revert of improve differently. we want to make the point to treat RAG data just like training data
There was a problem hiding this comment.
I reverted the change to what it was originally per your comment. The text above is the original text so it has been reverted to that.
| @@ -336,15 +343,15 @@ Selecting potential risks (Threats) that could impact the organization requires | |||
| **Leaking input data** | |||
|
|
|||
| Is your input data sensitive? | |||
| - Prevent [leaking input data](/goto/leakinput/). Especially if the model is run by a supplier, proper care needs to be taken that this data is transferred or stored in a protected way and as little as possible. Study the security level that the supplier provides and the options you have to for example disable logging or monitoring at the supplier side. Note, that if you use RAG, that the data you retrieve and insert into the prompt is also input data. This typically contains company secrets or personal data. | |||
| - Prevent [leaking input data](/goto/leakinput/). If the model is run by a supplier, proper care needs to be taken to ensure that this data is minimized and transferred or stored securely. Review the security measures provided by the supplier, including any options to disable logging or monitoring on their end. If you're using a RAG system, remember that the data you retrieve and inject into the prompt also counts as input data. This often includes sensitive company information or personal data. | |||
There was a problem hiding this comment.
you removed 'especially' but that needs to be put back.
There was a problem hiding this comment.
I have added especially back now.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR covers editing and rephrasing for the rest of the document. It's multiple changes in one PR to avoid creating too many PRs. Overall, typos, sentence structure and phrasal structures were changed.