-
Notifications
You must be signed in to change notification settings - Fork 429
Description
Describe the bug
The global settings
variable is used to determine whether to use a server for Guards. This means that if you use a remote Guard (i.e. with fetch_guard
), local Guards will try to contact the Guardrails server. Disabling the use_server
setting will make remote Guards try to use local validators, until it's re-enabled, e.g. by fetch_guard
.
It's probably better to have the setting be per-Guard, perhaps with the global variable providing a default for new Guards.
To Reproduce
Does not need a valid Guardrails-AI server to reproduce, since settings.use_server
is set before confirming if the server exists. However, this reproduction assumes a working server with a Guard called "lowercase-fix", which is just a LowerCase Validator with on_fail="fix"
.
- Create a Guard from a local Validator. It will work normally. E.g.
from guardrails import Guard
from guardrails.hub import RegexMatch
local_guard = Guard().use(RegexMatch, regex="hello.*", on_fail="exception")
# Should pass, since it matches the regex
first_response = local_guard.validate("hello there")
- Create a remote Guard. It will work normally. E.g.
remote_guard = Guard.fetch_guard("lowercase-fix")
# It should give "hello" as validated output
remote_response = remote_guard.validate("HeLlO")
- Use
local_guard.validate("hello there")
again. You will see the below error.
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\hub_telemetry\hub_tracing.py", line 150, in wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\guard.py", line 1097, in validate
return self.parse(llm_output=llm_output, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\hub_telemetry\hub_tracing.py", line 150, in wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\guard.py", line 973, in parse
return trace_guard_execution(
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\telemetry\guard_tracing.py", line 206, in trace_guard_execution
raise e
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\telemetry\guard_tracing.py", line 195, in trace_guard_execution
result = _execute_fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\guard.py", line 799, in _execute
return guard_context.run(
^^^^^^^^^^^^^^^^^^
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\telemetry\common.py", line 100, in wrapped_func
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\guard.py", line 763, in __exec
return self._call_server(
^^^^^^^^^^^^^^^^^^
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\guard.py", line 1251, in _call_server
raise ValueError("Guard does not have an api client!")
ValueError: Guard does not have an api client!
- You can re-enable the local Guard with the below code. However, it will then disable the remote Guard, which will now automatically pass and give the warning
C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\validator_base.py:618: UserWarning: Validator with id guardrails/lowercase was not found in the registry! Ignoring...
. This may not appear if the Validator used in the server is also installed locally.
from guardrails.settings import settings
settings.use_server = False
Expected behavior
Local Guards should continue to work after a remote Guard has been created.
Library version:
Version 0.6.6