feat(tools): run sync tools in a thread to avoid event loop blocking

[deepankarm]

Synchronous tools execution blocks the event loop, preventing other requests and asyncio.Tasks. This PR runs the sync tools in a separate thread using asyncio.to_thread()

import asyncio
import os

from agents import Agent, Runner, function_tool
from pyleak import no_event_loop_blocking, no_task_leaks
from tavily import TavilyClient

@function_tool
def trivily_web_search(query: str) -> str:
    """Search the web for the query."""
    client = TavilyClient(api_key=os.getenv("TAVILY_API_KEY"))
    return str(client.search(query))

hindi_agent = Agent(
    name="Hindi agent",
    instructions="You only speak Hindi.",
)

english_agent = Agent(
    name="English agent",
    instructions="You only speak English",
)

triage_agent = Agent(
    name="Triage agent",
    instructions="Handoff to the appropriate agent based on the language of the request.",
    handoffs=[hindi_agent, english_agent],
    tools=[trivily_web_search],
)

async def main():
    async with (
        no_event_loop_blocking(action="log", threshold=0.1),
        no_task_leaks(action="log"),
    ):
        result = await Runner.run(triage_agent, input="Namaste, aaj ka mausam kaisa hai Bangalore mein?")
        print(result.final_output)

if __name__ == "__main__":
    asyncio.run(main())

Event loop blocked for 0.305s (threshold: 0.100s)
Event loop blocked for 0.305s (threshold: 0.100s)
Namaste! Bangalore mein aaj mausam badliwala hai, taapmaan lagbhag 28 degree Celsius hai aur hawa tez chal rahi hai.
Detected 0 leaked asyncio tasks
Detected 2 event loop blocks
Event Loop Block: block-1
  Duration: 0.305s (threshold: 0.100s)
  Timestamp: 1749305305.440
  Blocking Stack:
    File "/path/to/script.py", line 44, in <module>
        asyncio.run(main())
      File "/path/to/python/lib/python3.9/asyncio/runners.py", line 44, in run
        return loop.run_until_complete(main)
      File "/path/to/python/lib/python3.9/asyncio/base_events.py", line 634, in run_until_complete
        self.run_forever()
      File "/path/to/python/lib/python3.9/asyncio/base_events.py", line 601, in run_forever
        self._run_once()
      File "/path/to/python/lib/python3.9/asyncio/base_events.py", line 1905, in _run_once
        handle._run()
      File "/path/to/python/lib/python3.9/asyncio/events.py", line 80, in _run
        self._context.run(self._callback, *self._args)
      File "/path/to/venv/lib/python3.9/site-packages/agents/tool.py", line 399, in _on_invoke_tool
        return await _on_invoke_tool_impl(ctx, input)
      File "/path/to/venv/lib/python3.9/site-packages/agents/tool.py", line 388, in _on_invoke_tool_impl
        result = the_func(*args, **kwargs_dict)
      File "/path/to/script.py", line 13, in trivily_web_search
        return str(client.search(query))
      File "/path/to/venv/lib/python3.9/site-packages/tavily/tavily.py", line 125, in search
        response_dict = self._search(query,
      File "/path/to/venv/lib/python3.9/site-packages/tavily/tavily.py", line 79, in _search
        response = requests.post(self.base_url + "/search", data=json.dumps(data), headers=self.headers, timeout=timeout, proxies=self.proxies)
      File "/path/to/venv/lib/python3.9/site-packages/requests/api.py", line 115, in post
        return request("post", url, data=data, json=json, **kwargs)
      File "/path/to/venv/lib/python3.9/site-packages/requests/api.py", line 59, in request
        return session.request(method=method, url=url, **kwargs)
      File "/path/to/venv/lib/python3.9/site-packages/requests/sessions.py", line 589, in request
        resp = self.send(prep, **send_kwargs)
      File "/path/to/venv/lib/python3.9/site-packages/requests/sessions.py", line 703, in send
        r = adapter.send(request, **kwargs)
      File "/path/to/venv/lib/python3.9/site-packages/requests/adapters.py", line 667, in send
        resp = conn.urlopen(
      File "/path/to/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 787, in urlopen
        response = self._make_request(
      File "/path/to/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 464, in _make_request
        self._validate_conn(conn)
      File "/path/to/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
        conn.connect()
      File "/path/to/venv/lib/python3.9/site-packages/urllib3/connection.py", line 741, in connect
        sock_and_verified = _ssl_wrap_socket_and_match_hostname(
      File "/path/to/venv/lib/python3.9/site-packages/urllib3/connection.py", line 920, in _ssl_wrap_socket_and_match_hostname
        ssl_sock = ssl_wrap_socket(
      File "/path/to/venv/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 480, in ssl_wrap_socket
        ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
      File "/path/to/venv/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 524, in _ssl_wrap_socket_impl
        return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
      File "/path/to/python/lib/python3.9/ssl.py", line 501, in wrap_socket
        return self.sslsocket_class._create(
      File "/path/to/python/lib/python3.9/ssl.py", line 1074, in _create
        self.do_handshake()
      File "/path/to/python/lib/python3.9/ssl.py", line 1343, in do_handshake
        self._sslobj.do_handshake()
Event Loop Block: block-2
  Duration: 2.430s (threshold: 0.100s)
  Timestamp: 1749305308.301
  Blocking Stack:
    File "/path/to/script.py", line 44, in <module>
        asyncio.run(main())
      File "/path/to/python/lib/python3.9/asyncio/runners.py", line 44, in run
        return loop.run_until_complete(main)
      File "/path/to/python/lib/python3.9/asyncio/base_events.py", line 634, in run_until_complete
        self.run_forever()
      File "/path/to/python/lib/python3.9/asyncio/base_events.py", line 601, in run_forever
        self._run_once()
      File "/path/to/python/lib/python3.9/asyncio/base_events.py", line 1905, in _run_once
        handle._run()
      File "/path/to/python/lib/python3.9/asyncio/events.py", line 80, in _run
        self._context.run(self._callback, *self._args)
      File "/path/to/venv/lib/python3.9/site-packages/agents/tool.py", line 399, in _on_invoke_tool
        return await _on_invoke_tool_impl(ctx, input)
      File "/path/to/venv/lib/python3.9/site-packages/agents/tool.py", line 388, in _on_invoke_tool_impl
        result = the_func(*args, **kwargs_dict)
      File "/path/to/script.py", line 13, in trivily_web_search
        return str(client.search(query))
      File "/path/to/venv/lib/python3.9/site-packages/tavily/tavily.py", line 125, in search
        response_dict = self._search(query,
      File "/path/to/venv/lib/python3.9/site-packages/tavily/tavily.py", line 79, in _search
        response = requests.post(self.base_url + "/search", data=json.dumps(data), headers=self.headers, timeout=timeout, proxies=self.proxies)
      File "/path/to/venv/lib/python3.9/site-packages/requests/api.py", line 115, in post
        return request("post", url, data=data, json=json, **kwargs)
      File "/path/to/venv/lib/python3.9/site-packages/requests/api.py", line 59, in request
        return session.request(method=method, url=url, **kwargs)
      File "/path/to/venv/lib/python3.9/site-packages/requests/sessions.py", line 589, in request
        resp = self.send(prep, **send_kwargs)
      File "/path/to/venv/lib/python3.9/site-packages/requests/sessions.py", line 703, in send
        r = adapter.send(request, **kwargs)
      File "/path/to/venv/lib/python3.9/site-packages/requests/adapters.py", line 667, in send
        resp = conn.urlopen(
      File "/path/to/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 787, in urlopen
        response = self._make_request(
      File "/path/to/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 534, in _make_request
        response = conn.getresponse()
      File "/path/to/venv/lib/python3.9/site-packages/urllib3/connection.py", line 516, in getresponse
        httplib_response = super().getresponse()
      File "/path/to/python/lib/python3.9/http/client.py", line 1377, in getresponse
        response.begin()
      File "/path/to/python/lib/python3.9/http/client.py", line 320, in begin
        version, status, reason = self._read_status()
      File "/path/to/python/lib/python3.9/http/client.py", line 281, in _read_status
        line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
      File "/path/to/python/lib/python3.9/socket.py", line 716, in readinto
        return self._sock.recv_into(b)
      File "/path/to/python/lib/python3.9/ssl.py", line 1275, in recv_into
        return self.read(nbytes, buffer)
      File "/path/to/python/lib/python3.9/ssl.py", line 1133, in read
        return self._sslobj.read(len, buffer)
Event loop monitoring summary: 2 block(s), 2.74s total blocked time

Note: This is an MWE showing sync tools blocking the event loop. Tavily does have an async client which fixes this particular problem, but the core issue affects any sync tool.

Consider adding pyleak in CI to catch asyncio task leaks and event loop blocking (disclaimer: I'm the author).

[deepankarm]

Here's a sample test that fails on the main branch - deepankarm#1. We can also add a test to the stream functions to ensure there are task leaks or event loop blocks. Happy to do a PR to the origin.

[deepankarm]

Requesting for reviews 🙇 @rm-openai

[github-actions]

This PR is stale because it has been open for 10 days with no activity.

[seratch]

These changes look good to me; @rm-openai thoughts?

[rm-openai]

hmm i'd definitely not want to move things to a separate thread by default. That would cause issues for anyone who ignores thread safety today based on the existing implementation.

Thoughts on making this configurable?

[deepankarm]

hmm i'd definitely not want to move things to a separate thread by default. That would cause issues for anyone who ignores thread safety today based on the existing implementation.

Thoughts on making this configurable?

@rm-openai sure, it makes sense to make it configurable. I've added a run_in_thread flag to function_tool decorator with a default false.

I'm curious about your thoughts on how to add a unit test for it - I had added a test here using pyleak, but I'm not sure if we want to add a new test dependency.