You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* To learn how to use the built-in detectors with `trustyai_fms` orchestrator server external provider for Llama Stack to detect PII, see link:{rhoaidocshome}{default-format-url}/monitoring_data_science_models#detecting-pii-by-using-guardrails-with-llama-stack[Adding a model server to your data science project].
40
+
* To learn how to use the built-in detectors with `trustyai_fms` orchestrator server external provider for Llama Stack to detect PII, see link:{rhoaidocshome}{default-format-url}/monitoring_data_science_models#detecting-pii-by-using-guardrails-with-llama-stack[Detecting personally identifiable information (PII) by using Guardrails with Llama Stack].
Copy file name to clipboardExpand all lines: modules/guardrails-auto-config.adoc
+4-8Lines changed: 4 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -43,16 +43,12 @@ spec:
43
43
+
44
44
* `inferenceServiceToGuardrail`: Specifies the name of the vLLM inference service to protect with Guardrails.
45
45
* `detectorServiceLabelToMatch`: Specifies the label that you applied to each of your detector servers in the `metadata.labels` specification for the detector. The Guardrails Orchestrator `ConfigMap` automatically updates to reflect detectors in your namespace that match the label set in the `detectorServiceLabelToMatch` field.
46
-
46
+
+
47
47
If `enableGuardrailsGateway` is true, a template Guardrails gateway config called `<ORCHESTRATOR_NAME>-gateway-auto-config` is generated. You can modify this file to tailor
48
48
your Guardrails Gateway setup as desired. The Guardrails Orchestrator automatically deploys when changes are detected. Once modified, the label `trustyai/has-diverged-from-auto-config` is applied. To revert the file back to the auto-generated starting point, simply delete it and the original auto-generated file is recreated.
49
-
49
+
+
50
50
If `enableBuiltInDetectors` is true, the built-in detector server is automatically added to your orchestrator configuration under the same `built-in-detector`, and a sample configuration is included in the auto-generated guardrails gateway config if desired.
51
51
52
-
53
-
54
-
55
-
56
52
. Deploy the orchestrator custom resource. This step creates a service account, deployment, service, and route object in your namespace.
You can verify that the `GuardrailsOrchestrator` CR and corresponding automaticallygenerated configuration objects were successfully created in your namespace by running the following commands:
60
+
You can verify that the `GuardrailsOrchestrator` CR and corresponding automatically-generated configuration objects were successfully created in your namespace by running the following commands:
65
61
66
62
. Confirm that the `GuardrailsOrchestrator` CR was created:
67
63
+
@@ -77,7 +73,7 @@ $ oc get guardrailsorchestrator -n <your_namespace>
77
73
$ oc get configmap -n <your_namespace> | grep auto-config
78
74
----
79
75
80
-
. You can then view the automatically generated config:
76
+
. You can then view the automatically generated configmap:
Copy file name to clipboardExpand all lines: modules/guardrails-configuring-the-hugging-face-detector-serving-runtime.adoc
+4-8Lines changed: 4 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -57,14 +57,12 @@ spec:
57
57
- containerPort: 8000
58
58
protocol: TCP
59
59
----
60
+
+
60
61
The above serving runtime example matches the default template used with {productname-long}, and should suffice
61
-
for the majority of use-cases. The main relevant configuration parameter is the `SAFE_LABELS` environment variable,
62
-
which specifies which prediction label(s) from the `AutoModelForSequenceClassification` constitute a "safe" response and thus
63
-
should not trigger guardrailing. For example, if `[0, 1]` is specified as `SAFE_LABELS`
64
-
for a four-class model, a predicted label of `0` or `1` will be considered "safe", while a predicted label
65
-
of `2` or `3` will trigger guardrailing. The default value is `[0]`.
62
+
for the majority of use-cases. The main relevant configuration parameter is the `SAFE_LABELS` environment variable. This specifies which prediction label or labels from the `AutoModelForSequenceClassification` constitute a "safe" response and therefore should not trigger guardrailing. For example, if `[0, 1]` is specified as `SAFE_LABELS`
63
+
for a four-class model, a predicted label of `0` or `1` is considered "safe", while a predicted label of `2` or `3` triggers guardrailing. The default value is `[0]`.
66
64
67
-
The following tables describe configuration values for the Guardrails Detector Hugging Face serving runtime:
65
+
== Guardrails Detector Hugging Face serving runtime configuration values
68
66
69
67
.Template configuration
70
68
[cols="2,5"]
@@ -148,8 +146,6 @@ The following tables describe configuration values for the Guardrails Detector H
. Create a Guardrails Orchestrator custom resource. Make sure that `orchestratorConfig` and `guardrailsGatewayConfig` matches the names of the resources you created in steps 1 and 2.
48
+
. Create a Guardrails Orchestrator custom resource. Make sure that the `orchestratorConfig` and `guardrailsGatewayConfig` match the names of the resources you created in steps 1 and 2.
49
49
+
50
50
.Example `orchestrator_cr.yaml` CR
51
51
[source,yaml]
@@ -55,39 +55,40 @@ kind: GuardrailsOrchestrator
55
55
metadata:
56
56
name: guardrails-orchestrator-sample
57
57
spec:
58
-
orchestratorConfig: <NAME OF ORCHESTRATOR CONFIGMAP>
59
-
guardrailsGatewayConfig: <NAME OF GUARDRAILS GATEWAY CONFIGMAP>
60
-
customDetectorsConfig: <NAME OF CUSTOM DETECTORS CONFIG>
If desired, the TrustyAI controller can automatically generate an orchestratorConfig and guardrailsGatewayConfig based on the available resources in your namespace.
76
-
To access this, include the `autoConfig` parameter inside your Custom Resource, and consult <<guardrails-auto-config_{context}, Auto Configuring Guardrails>> for documentation on its usage.
75
+
If desired, the TrustyAI controller can automatically generate an `orchestratorConfig` and `guardrailsGatewayConfig` based on the available resources in your namespace.
76
+
To access this, include the `autoConfig` parameter inside your Custom Resource, and see _Auto Configuring Guardrails_ for documentation on its usage.
77
77
+
78
+
.Parameters from example `orchestrator_cr.yaml` CR
78
79
[cols="1,2a", options="header"]
79
80
|===
80
81
| Parameter |Description
81
82
| *`orchestratorConfig`* *(optional)*|The name of the `ConfigMap` object that contains generator, detector, and chunker arguments. If using `autoConfig`, this field can be omitted.
82
-
|`guardrailsGatewayConfig` *(optional)*| The name of the ConfigMap object that specifies gateway configurations. This field can be omitted if you do not wish to use the Guardrails Gateway or are using `autoConfig`.
83
+
|`guardrailsGatewayConfig` *(optional)*| The name of the ConfigMap object that specifies gateway configurations. This field can be omitted if you are not using the Guardrails Gateway or are using `autoConfig`.
83
84
|`customDetectorsConfig` *(optional)*| This feature is in development preview.
84
-
| `autoConfig` *(optional)* | A list of paired name and value arguments to define how the Guardrails AutoConfig. Any manually-specified configuration files in `orchestratorConfig` or `guardrailsGatewayConfig` will take precedence over the automatically-generated configuration files.
85
+
| `autoConfig` *(optional)* | A list of paired name and value arguments to define how the Guardrails AutoConfig. Any manually-specified configuration files in `orchestratorConfig` or `guardrailsGatewayConfig` takes precedence over the automatically-generated configuration files.
85
86
86
-
* `inferenceServiceToGuardrail` - The name of the inference service you wish to guardrail. This should exactly match the model name provided when deploying the model. For a list of valid names, you can run
87
+
* `inferenceServiceToGuardrail` - The name of the inference service you want to guardrail. This should exactly match the model name provided when deploying the model. For a list of valid names, you can run
87
88
`oc get isvc -n $NAMESPACE`
88
-
* `detectorServiceLabelToMatch` - A string label to use when searching for available detector servers. All inference services in your namespace with the label `$detectorServiceLabelToMatch: true` will be automatically configured as a detector.
89
+
* `detectorServiceLabelToMatch` - A string label to use when searching for available detector servers. All inference services in your namespace with the label `$detectorServiceLabelToMatch: true` is automatically configured as a detector.
89
90
+
90
-
See <<guardrails-auto-config_{context}, Auto Configuring Guardrails>>for more information.
91
+
See _Auto Configuring Guardrails_ for more information.
91
92
92
93
|`enableBuiltInDetectors` *(optional)*| A boolean value to inject the built-in detector sidecar container into the orchestrator pod. The built-in detector is a lightweight HTTP server containing a number of available guardrailing algorithms.
93
94
|`enableGuardrailsGateway` *(optional)*| A boolean value to enable controlled interaction with the orchestrator service by enforcing stricter access to its exposed endpoints. It provides a mechanism of configuring detector pipelines, and then provides a unique `/v1/chat/completions` endpoint per configured detector pipeline.
@@ -101,8 +102,8 @@ See <<guardrails-auto-config_{context}, Auto Configuring Guardrails>>for more in
101
102
+
102
103
| `logLevel` *(optional)* | The log level to be used in the Guardrails Orchestrator- available values are `Error`, `Warn`, `Info` (default), `Debug`, and `Trace`.
103
104
[[tlsSecrets-param]]
104
-
| `tlsSecrets` *(optional)* | A list of names of `Secret` objects to mount to the Guardrails Orchestrator container. All secrets provided here will be mounted into the directory `/etc/tls/$SECRET_NAME` for use in your orchestrator config TLS configuration. We recommend that each secret contain a `tls.crt` and a `tls.key` field.
105
-
|*`replicas`*| The number of orchestrator pods to create.
105
+
| `tlsSecrets` *(optional)* | A list of names of `Secret` objects to mount to the Guardrails Orchestrator container. All secrets provided here are mounted into the directory `/etc/tls/$SECRET_NAME` for use in your orchestrator config TLS configuration. Each secret should contain a `tls.crt` and a `tls.key` field.
106
+
|`replicas`| The number of orchestrator pods to create.
106
107
|===
107
108
108
109
. Deploy the orchestrator CR, which creates a service account, deployment, service, and route object in your namespace:
You can use the Guardrails Orchestrator API to send requests to the regex detector. The regex detector filters conversations by flagging content that matches specified regular expression patterns.
9
9
10
10
.Prerequisites
11
-
You have deployed a Guardrails Orchestrator with the built-in-detector server, e.g.:
11
+
You have deployed a Guardrails Orchestrator with the built-in-detector server, such as in the following example:
0 commit comments