Skip to content

Conversation

GuillermoBlasco
Copy link
Contributor

Added minor change to documentation so specify the requirements for tools to be concurrently executed.

Slack conversation reference: https://pydanticlogfire.slack.com/archives/C083V7PMHHA/p1753301883710229

Copy link
Contributor

hyperlint-ai bot commented Jul 24, 2025

PR Change Summary

Specified the requirements for concurrent tool execution in the documentation.

  • Clarified the need for tools to be defined as async functions for concurrent execution.
  • Emphasized the use of non-blocking clients for effective concurrency.

Modified Files

  • docs/tools.md

How can I customize these reviews?

Check out the Hyperlint AI Reviewer docs for more information on how to customize the review.

If you just want to ignore it on this PR, you can add the hyperlint-ignore label to the PR. Future changes won't trigger a Hyperlint review.

Note specifically for link checks, we only check the first 30 links in a file and we cache the results for several hours (for instance, if you just added a page, you might experience this). Our recommendation is to add hyperlint-ignore to the PR to ignore the link check for this PR.

@DouweM
Copy link
Collaborator

DouweM commented Jul 24, 2025

@Kludex Can you please check this out? You had some related comments in our internal Slack that I think should also be documented.

@DouweM DouweM assigned DouweM and Kludex and unassigned DouweM Jul 24, 2025
@GuillermoBlasco
Copy link
Contributor Author

BTW Alex Hall commented in the public Slack about documenting a bit how to use the baggage to add attributes in the spans of an agent run. I was about to document this also, do you care if I add that bit to this PR?
https://pydanticlogfire.slack.com/archives/C06EDRBSAH3/p1753133127281339

@DouweM
Copy link
Collaborator

DouweM commented Jul 25, 2025

@GuillermoBlasco I'd prefer to have that in a separate PR.


As for the thing I pinged Marcelo for, he's going to be out next week, so I'll quote it here so you can see if you can incorporate it into this PR yourself:

[Other person]

Is it general good practice to have all tools async?

Marcelo Trylesinski

in general, if your tool just returns a value without any blocking io, yes

if you use anything that is blocking io, like non-async http call, file reading, etc, then def is the way to go

underneath it will run in a different thread

so if you have just

def simple():
   return 1

it's better to be async

same concept in FastAPI

[Other person]

Ahh interesting I would have expected the opposite, I guess you don't know if def is quick or slow, hence using the different thread.

Marcelo Trylesinski

yep

[Other person]

Does that also apply for @agent.instructions?

Marcelo Trylesinski

yep

@GuillermoBlasco
Copy link
Contributor Author

Docs updated with the detail Marcelo provided!

Co-authored-by: Douwe Maan <[email protected]>
@DouweM DouweM changed the title chore: specified the requirements for concurrent tool execution Document performance implications of async vs sync tools Aug 4, 2025
@DouweM DouweM merged commit 07f54f9 into pydantic:main Aug 4, 2025
19 checks passed
@@ -724,6 +724,12 @@ def my_flaky_tool(query: str) -> str:

Raising `ModelRetry` also generates a `RetryPromptPart` containing the exception message, which is sent back to the LLM to guide its next attempt. Both `ValidationError` and `ModelRetry` respect the `retries` setting configured on the `Tool` or `Agent`.

### Parallel tool calls & concurrency

When a model returns multiple tool calls in one response, Pydantic AI schedules them concurrently using `asyncio.create_task`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to not have the "using asyncio.create_task" in this sentence.

That's an implementation detail, which I don't think is correct (and if it is, we might need to change it).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Kludex Can you remove/reword it please?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can @claude do it?


When a model returns multiple tool calls in one response, Pydantic AI schedules them concurrently using `asyncio.create_task`.

Async functions are run on the event loop, while sync functions are offloaded to threads. To get the best performance, _always_ use an async function _unless_ you're doing blocking I/O (and there's no way to use a non-blocking library instead) or CPU-bound work (like `numpy` or `scikit-learn` operations), so that simple functions are not offloaded to threads unnecessarily.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is okay.

ethanabrooks added a commit to reflectionai/pydantic-ai that referenced this pull request Aug 20, 2025
* Add `priority` `service_tier` to `OpenAIModelSettings` and respect it in `OpenAIResponsesModel` (pydantic#2368)

* Add an example of using RunContext to pass data among tools (pydantic#2316)

Co-authored-by: Douwe Maan <[email protected]>

* Rename gemini-2.5-flash-lite-preview-06-17 to gemini-2.5-flash-lite as it's out of preview (pydantic#2387)

* Fix toggleable toolset example so toolset state is not shared across agent runs (pydantic#2396)

* Support custom thinking tags specified on the model profile (pydantic#2364)

Co-authored-by: jescudero <[email protected]>
Co-authored-by: Douwe Maan <[email protected]>

* Add convenience functions to handle AG-UI requests with request-specific deps (pydantic#2397)

* docs: add missing optional packages in `install.md` (pydantic#2412)

* Include default values in tool arguments JSON schema (pydantic#2418)

* Fix "test_download_item_no_content_type test fails on macOS" (pydantic#2404)

* Allow string format, pattern and others in OpenAI strict JSON mode (pydantic#2420)

* Let more `BaseModel`s use OpenAI strict JSON mode by defaulting to `additionalProperties=False` (pydantic#2419)

* BREAKING CHANGE: Change type of 'source' field on EvaluationResult (pydantic#2388)

Co-authored-by: Douwe Maan <[email protected]>

* Fix ImageUrl, VideoUrl, AudioUrl and DocumentUrl not being serializable (pydantic#2422)

* BREAKING CHANGE: Support printing reasons in the console output for pydantic-evals (pydantic#2163)

* Document performance implications of async vs sync tools (pydantic#2298)

Co-authored-by: Douwe Maan <[email protected]>

* Mention that tools become toolset internally (pydantic#2395)

Co-authored-by: Douwe Maan <[email protected]>

* Fix tests for Logfire>=3.22.0 (pydantic#2346)

* tests: speed up the test suite (pydantic#2414)

* google: add more information about schema on union (pydantic#2426)

* typo in output docs (pydantic#2427)

* Deprecate `GeminiModel` in favor of `GoogleModel` (pydantic#2416)

* Use `httpx` on `GoogleProvider` (pydantic#2438)

* Remove older deprecated models and add new model of Anthropic (pydantic#2435)

* Remove `next()` method from `Graph` (pydantic#2440)

* BREAKING CHANGE: Remove `data` from `FinalResult` (pydantic#2443)

* BREAKING CHANGE: Remove `get_data` and `validate_structured_result` from `StreamedRunResult` (pydantic#2445)

* docs: add `griffe_warnings_deprecated` (pydantic#2444)

* BREAKING CHANGE: Remove `format_as_xml` module (pydantic#2446)

* BREAKING CHANGE: Remove `result_type` parameter and similar from `Agent` (pydantic#2441)

* Deprecate `GoogleGLAProvider` and `GoogleVertexProvider` (pydantic#2450)

* BREAKING CHANGE: drop 4 months old deprecation warnings (pydantic#2451)

* Automatically use OpenAI strict mode for strict-compatible native output types (pydantic#2447)

* Make `InlineDefsJsonSchemaTransformer` public (pydantic#2455)

* Send `ThinkingPart`s back to Anthropic used through Bedrock (pydantic#2454)

* Bump boto3 to support `AWS_BEARER_TOKEN_BEDROCK` API key env var (pydantic#2456)

* Add new Heroku models (pydantic#2459)

* Add `builtin_tools` to `Agent` (pydantic#2102)

Co-authored-by: Marcelo Trylesinski <[email protected]>
Co-authored-by: Douwe Maan <[email protected]>

* Bump mcp-run-python (pydantic#2470)

* Remove fail_under from top-level coverage config so <100% html-coverage step doesn't end CI run (pydantic#2475)

* Add AbstractAgent, WrapperAgent, Agent.event_stream_handler, Toolset.id, Agent.override(tools=...) in preparation for Temporal (pydantic#2458)

* Let toolsets be built dynamically based on run context (pydantic#2366)

Co-authored-by: Douwe Maan <[email protected]>

* Add ToolsetFunc to API docs (fix CI) (pydantic#2486)

* tests: change time of evals example (pydantic#2501)

* ci: remove html and xml reports (pydantic#2491)

* fix: Add gpt-5 models to reasoning model detection for temperature parameter handling (pydantic#2483)

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Douwe Maan <[email protected]>
Co-authored-by: Marcelo Trylesinski <[email protected]>

* History processor replaces message history (pydantic#2324)

Co-authored-by: Marcelo Trylesinski <[email protected]>

* ci: split test suite (pydantic#2436)

Co-authored-by: Douwe Maan <[email protected]>

* ci: use the right install command (pydantic#2506)

* Update config.yaml (pydantic#2514)

* Skip testing flaky evals example (pydantic#2518)

* Fix error when parsing usage details for video without audio track in Google models (pydantic#2507)

* Make OpenAIResponsesModelSettings.openai_builtin_tools work again (pydantic#2520)

* Let Agent be run in a Temporal workflow by moving model requests, tool calls, and MCP to Temporal activities (pydantic#2225)

* Install only dev in CI (pydantic#2523)

* Improve CLAUDE.md (pydantic#2524)

* Add best practices regarding to coverage to CLAUDE.md (pydantic#2527)

* Add support for `"openai-responses"` model inference string (pydantic#2528)

Co-authored-by: Claude <[email protected]>

* docs: Confident AI (pydantic#2529)

* chore: mention what to do with the documentation when deprecating a class (pydantic#2530)

* chore: drop hyperlint (pydantic#2531)

* ci: improve matrix readability (pydantic#2532)

* Add pip to dev deps for PyCharm (pydantic#2533)

Co-authored-by: Marcelo Trylesinski <[email protected]>

* Add genai-prices to dev deps and a basic test (pydantic#2537)

* Add `--durations=100` to all pytest calls in CI (pydantic#2534)

* Cleanup snapshot in test_evaluate_async_logfire (pydantic#2538)

* Make some minor tweaks to the temporal docs (pydantic#2522)

Co-authored-by: Douwe Maan <[email protected]>

* Add new OpenAI GPT-5 models (pydantic#2503)

* Fix `FallbackModel` to respect each model's model settings (pydantic#2540)

* Add support for OpenAI verbosity parameter in Responses API (pydantic#2493)

Co-authored-by: Claude <[email protected]>
Co-authored-by: Douwe Maan <[email protected]>

* Add `UsageLimits.count_tokens_before_request` using Gemini `count_tokens` API (pydantic#2137)

Co-authored-by: Douwe Maan <[email protected]>

* chore: Fix uv.lock (pydantic#2546)

* Stop calling MCP server `get_tools` ahead of `agent run` span (pydantic#2545)

* Disable instrumentation by default in tests (pydantic#2535)

Co-authored-by: Marcelo Trylesinski <[email protected]>

* Only wrap necessary parts of type aliases in forward annotations (pydantic#2548)

* Remove anthropic-beta default header set in `AnthropicModel` (pydantic#2544)

Co-authored-by: Marcelo Trylesinski <[email protected]>

* docs: Clarify why AG-UI example links are on localhost (pydantic#2549)

* chore: Fix path to agent class in CLAUDE.md (pydantic#2550)

* Ignore leading whitespace when streaming from Qwen or DeepSeek (pydantic#2554)

* Ask model to try again if it produced a response without text or tool calls, only thinking (pydantic#2556)

Co-authored-by: Douwe Maan <[email protected]>

* chore: Improve Temporal test to check trace as tree instead of list (pydantic#2559)

* Fix: Forward max_uses parameter to Anthropic WebSearchTool (pydantic#2561)

* Let message history end on ModelResponse and execute pending tool calls (pydantic#2562)

* Fix type issues

* skip tests requiring API keys

* add `google-genai` dependency

* add other provider deps

* add pragma: no cover for untested logic

---------

Co-authored-by: akenar <[email protected]>
Co-authored-by: Tony Woland <[email protected]>
Co-authored-by: Douwe Maan <[email protected]>
Co-authored-by: Yi-Chen Lin <[email protected]>
Co-authored-by: José I. Escudero <[email protected]>
Co-authored-by: jescudero <[email protected]>
Co-authored-by: Marcelo Trylesinski <[email protected]>
Co-authored-by: William Easton <[email protected]>
Co-authored-by: David Montague <[email protected]>
Co-authored-by: Guillermo <[email protected]>
Co-authored-by: Hamza Farhan <[email protected]>
Co-authored-by: Mohamed Amine Zghal <[email protected]>
Co-authored-by: Yinon Ehrlich <[email protected]>
Co-authored-by: Matthew Brandman <[email protected]>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Douwe Maan <[email protected]>
Co-authored-by: Alex Enrique <[email protected]>
Co-authored-by: Jerry Yan <[email protected]>
Co-authored-by: Claude <[email protected]>
Co-authored-by: Mayank <[email protected]>
Co-authored-by: Alex Hall <[email protected]>
Co-authored-by: Jerry Lin <[email protected]>
Co-authored-by: Raymond Xu <[email protected]>
Co-authored-by: kauabh <[email protected]>
Co-authored-by: Victorien <[email protected]>
Co-authored-by: Ethan Brooks <[email protected]>
Co-authored-by: eballesteros <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants