Error Using Gemini API Key with Agent-as-Tool Pattern in openai-agent-sdk

[shirazkk]

Issue: Using Agent-as-Tool Pattern with Gemini API

I am trying to use the agent-as-tool pattern from the openai-agents-sdk, but I encounter an error when using a Gemini API key with the OpenAIChatCompletionsModel and AsyncOpenAI. The error message I get is:

Error:

Orchestrator result: I am sorry, I cannot fulfill this request. The tool returned an error. Please try again in sometime.

Code:

import asyncio
from agents import Agent, ItemHelpers, MessageOutputItem, Runner, trace
from agents import Agent, Runner, set_tracing_disabled, OpenAIChatCompletionsModel, RunConfig, ModelProvider, ModelSettings
from openai import AsyncOpenAI
from typing import cast
import os
from dotenv import load_dotenv
import traceback


load_dotenv()

set_tracing_disabled(disabled=True)

gemini_api_key = os.getenv("GEMINI_API_KEY")

if not gemini_api_key:
    raise ValueError("GEMINI_API_KEY is not set. Please ensure it is defined in your .env file.")

external_client = AsyncOpenAI(
    api_key= gemini_api_key,
    base_url="https://generativelanguage.googleapis.com/v1beta/openai/",
)

model = OpenAIChatCompletionsModel(
    model="gemini-2.0-flash",
    openai_client=external_client,
)

config = RunConfig(
    model=model,
    model_provider=cast(ModelProvider, external_client),
    tracing_disabled=True,
)


spanish_agent = Agent(
    name="spanish_agent",
    instructions="You translate the user's message to Spanish",
)

french_agent = Agent(
    name="french_agent",
    instructions="You translate the user's message to French",
)

italian_agent = Agent(
    name="italian_agent",
    instructions="You translate the user's message to Italian",
)

orchestrator_agent = Agent(
    name="orchestrator_agent",
    instructions=(
        "You are a translation agent. You use the tools given to you to translate."
        "If asked for multiple translations, you call the relevant tools in order."
        "You never translate on your own, you always use the provided tools."
    ),
    tools=[
        spanish_agent.as_tool(
            tool_name="translate_to_spanish",
            tool_description="Translate the user's message to Spanish",
        ),
        french_agent.as_tool(
            tool_name="translate_to_french",
            tool_description="Translate the user's message to French",
        ),
        italian_agent.as_tool(
            tool_name="translate_to_italian",
            tool_description="Translate the user's message to Italian",
        ),
    ],
)


result = Runner.run_sync(
    orchestrator_agent,
    "Translate 'Hello, how are you?' to Spanish",
    run_config=config,
)

print("Orchestrator result: ", result.final_output)

What I’ve Tried:

• Confirmed the Gemini API key is loaded and valid
• Confirmed the code pattern works with OpenAI models and API keys
• Changing model name, base URL, and other settings

What’s Not Working:

• When the orchestrator agent tries to call the translation tools, the tool returns a generic error and translation does not occur.
• No detailed traceback about the API error is provided.

Questions:

1. Is the agent-as-tool pattern expected to work with the Gemini API (using AsyncOpenAI)?
2. Are there any compatibility limitations between the OpenAI client and Gemini endpoints for tool use/function calling?

This markdown will retain the structure and clarity of the original content, formatted for use in a markdown-supported environment like GitHub or documentation.