[genai, vertexai]: Add anyOf support for Union types in tool calling

[msaadg]

PR Description

This PR adds support for Union types in tool arguments for Gemini tool calling in both langchain_google_genai and langchain_google_vertexai. Previously, Union types caused InvalidArgument errors in ChatVertexAI and incomplete schema handling in ChatGoogleGenerativeAI. The fix introduces anyOf schema support to represent all Union type variants, ensuring compatibility with Gemini’s API. Additionally, bind_tools docstrings are updated to document this new feature.

Relevant issues

Fixes #463 #986

Type

🆕 New Feature
🐛 Bug Fix
📖 Documentation
✅ Test

Changes

• langchain_google_genai:
  • Modified _get_properties_from_schema in _function_utils.py to support anyOf for non-nullable Union types.
  • Updated bind_tools in chat_models.py to document Union type support in the docstring.
  • Added unit tests in test_function_utils.py for Union types (simple, primitive, nested, Pydantic Field).
• langchain_google_vertexai:
  • Updated _format_json_schema_to_gapic in functions_utils.py to handle arbitrary anyOf types, resolving InvalidArgument errors.
  • Updated bind_tools in chat_models.py to document Union type support in the docstring.
  • Added unit tests in test_function_utils.py to validate schema conversion and gapic.Schema compatibility.
• Preserved nullable type handling for backward compatibility with Optional fields.

Testing

• Test Procedure:
  • Added unit tests for both packages covering:
    • Simple Union types (e.g., Union[Helper1, Helper2])
    • Union types with primitives (e.g., Union[str, Helper])
    • Nested Union types (e.g., List[Union[str, Contact]])
    • Pydantic Field with Union types and metadata
    • Nullable type handling for Optional fields
  • Tests validate schema structure, type correctness, and gapic.Schema compatibility (for Vertex AI).
• Test Result:
  • All tests pass locally with make test.
  • Verified schema compatibility with Google’s SDK for Vertex AI.

Note

• The PR touches both genai and vertexai packages due to the shared nature of tool calling functionality with Gemini.

[msaadg]

@ccurme @lkuligin can I please have a review on this. Thanks!

[lkuligin]

@msaadg integration tests are failing

[msaadg]

@lkuligin It seems that the failing integration tests in vertexai are unrelated to the changes I made. There were no new updates on this PR, and the integration tests transitioned from passing to failing automatically. Could you please confirm?

[yotamgabi]

@lkuligin It seems that the failing integration tests in vertexai are unrelated to the changes I made. There were no new updates on this PR, and the integration tests transitioned from passing to failing automatically. Could you please confirm?

you got - "can't create service account key on gcp", yet its important to check IT with gemini imho

[msaadg]

@lkuligin the integration tests are now passing!

[ronfogel]

Hey, any updates on the merge? We’re eager to get it through since it’s key to resolving an urgent issue on our side. 🙏

[lkuligin]

@msaadg thanks a lot for your contribution. approved, next time, please, split different libraries in different PRs since it would be easier to review and release then.