Skip to content

[bug] Unable to use local Guards alongside remote Guards #1303

@AlexBlumer

Description

@AlexBlumer

Describe the bug
The global settings variable is used to determine whether to use a server for Guards. This means that if you use a remote Guard (i.e. with fetch_guard), local Guards will try to contact the Guardrails server. Disabling the use_server setting will make remote Guards try to use local validators, until it's re-enabled, e.g. by fetch_guard.

It's probably better to have the setting be per-Guard, perhaps with the global variable providing a default for new Guards.

To Reproduce
Does not need a valid Guardrails-AI server to reproduce, since settings.use_server is set before confirming if the server exists. However, this reproduction assumes a working server with a Guard called "lowercase-fix", which is just a LowerCase Validator with on_fail="fix".

  1. Create a Guard from a local Validator. It will work normally. E.g.
from guardrails import Guard
from guardrails.hub import RegexMatch

local_guard = Guard().use(RegexMatch, regex="hello.*", on_fail="exception")
# Should pass, since it matches the regex
first_response = local_guard.validate("hello there")
  1. Create a remote Guard. It will work normally. E.g.
remote_guard = Guard.fetch_guard("lowercase-fix")
# It should give "hello" as validated output
remote_response = remote_guard.validate("HeLlO")
  1. Use local_guard.validate("hello there") again. You will see the below error.

Traceback (most recent call last):
File "", line 1, in
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\hub_telemetry\hub_tracing.py", line 150, in wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\guard.py", line 1097, in validate
return self.parse(llm_output=llm_output, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\hub_telemetry\hub_tracing.py", line 150, in wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\guard.py", line 973, in parse
return trace_guard_execution(
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\telemetry\guard_tracing.py", line 206, in trace_guard_execution
raise e
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\telemetry\guard_tracing.py", line 195, in trace_guard_execution
result = _execute_fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\guard.py", line 799, in _execute
return guard_context.run(
^^^^^^^^^^^^^^^^^^
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\telemetry\common.py", line 100, in wrapped_func
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\guard.py", line 763, in __exec
return self._call_server(
^^^^^^^^^^^^^^^^^^
File "C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\guard.py", line 1251, in _call_server
raise ValueError("Guard does not have an api client!")
ValueError: Guard does not have an api client!

  1. You can re-enable the local Guard with the below code. However, it will then disable the remote Guard, which will now automatically pass and give the warning C:\Users\Alex\AppData\Local\Programs\Python\Python311\Lib\site-packages\guardrails\validator_base.py:618: UserWarning: Validator with id guardrails/lowercase was not found in the registry! Ignoring.... This may not appear if the Validator used in the server is also installed locally.
from guardrails.settings import settings
settings.use_server = False

Expected behavior
Local Guards should continue to work after a remote Guard has been created.

Library version:
Version 0.6.6

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions