How do we make the prompt explicit rather than implicit?

[GuillermoGaete]

Hello! How are you doing?

As an early adopter of LangChain at the time, I'd like to share a concern.

Today, I spent some time playing around with the ADK, and I really like some of the abstractions introduced.

My concern, however, is that we're currently prompting behind the scenes, especially with the LLMAgent, as if it were a black box. It's well-known that the prompt and the way it's structured are crucial for achieving good results.

As a user, I would feel much more comfortable if we could reach an approach that allows us to explicitly define the prompt instead of delegating it entirely to the ADK. The ADK should help me develop agents but shouldn't hide such important details as prompting.

This is simply my opinion, and I hope we continue developing these kinds of tools that enable us to build amazing products!

[hangfei]

Thanks for the feedback!

Could you share some code example to explain your concern(what's hidden from developers)?

[GuillermoGaete]

Sure! Take a look 👇

I'm testing a simple coordinator scenario. Here is my agent code:

# Conceptual Code: Coordinator using LLM Transfer
from google.adk.agents import LlmAgent
from google.adk.models.lite_llm import LiteLlm

billing_agent = LlmAgent(name="Billing", description="Handles billing inquiries.")
support_agent = LlmAgent(name="Support", description="Handles technical support requests.")
chat_agent = LlmAgent(model=LiteLlm(model="openai/gpt-4o-mini"),name="Chat", description="Handles regular chat inquiries.")

root_agent = LlmAgent(
    name="HelpDeskCoordinator",
    model=LiteLlm(model="openai/gpt-4o"), # LiteLLM model string format
    instruction="Route user requests: Use Billing agent for payment issues, Support agent for technical problems.",
    description="Main help desk router.",
    # allow_transfer=True is often implicit with sub_agents in AutoFlow
    sub_agents=[chat_agent, billing_agent, support_agent]
    )
# User asks "My payment failed" -> Coordinator's LLM should call transfer_to_agent(agent_name='Billing')
# User asks "I can't log in" -> Coordinator's LLM should call transfer_to_agent(agent_name='Support')

Here's my interaction:

If you look at the prompt sent to the model, it appears as follows:

"Route user requests: Use Billing agent for payment issues, Support agent for technical problems. You are an agent. Your internal name is "HelpDeskCoordinator". The description about you is "Main help desk router." You have a list of other agents to transfer to: Agent name: Chat Agent description: Handles regular chat inquiries. Agent name: Billing Agent description: Handles billing inquiries. Agent name: Support Agent description: Handles technical support requests. If you are the best to answer the question according to your description, you can answer it. If another agent is better for answering the question according to its description, call `transfer_to_agent` function to transfer the question to that agent. When transfering, do not generate any text other than the function call. "

The prompt is built dynamically, as I understand, with the properties used to construct the sub-agents and our coordinator. For example, while navigating the codebase, I found this class:

class _IdentityLlmRequestProcessor(BaseLlmRequestProcessor):
  """Gives the agent identity from the framework."""

  @override
  async def run_async(
      self, invocation_context: InvocationContext, llm_request: LlmRequest
  ) -> AsyncGenerator[Event, None]:
    agent = invocation_context.agent
    si = [f'You are an agent. Your internal name is "{agent.name}".']
    if agent.description:
      si.append(f' The description about you is "{agent.description}"')
    llm_request.append_instructions(si)

    # Maintain async generator behavior
    if False:  # Ensures it behaves as a generator
      yield  # This is a no-op but maintains generator structure


request_processor = _IdentityLlmRequestProcessor()

To me as a user, at first, this seems like something magical (and awesome), but it will quickly become a problem if I want to define my own prompt to do the routing with more complexity, or even if I want to structure it for few-shot prompting.

I’m not sure what the best solution would be, maybe we could think of a higher-level abstraction that allows us to model a prompt for routing agents and gives the user control over the parts of that prompt.

I think the responsibilities of a coordinator should be:

1. Correctly delegate to a sub-agent --> We would need to know when and how to do this for each one.
2. If none is appropriate --> Reject the request --> Maybe delegate to a fallback? (which should be required by design)

Thinking about that, perhaps we could have a default prompt (or a meta-prompt), but it should be over-writable and modifiable in one place or by overriding a single dependency.

[rcleveng]

I think I have the same problem.

I really want to be able to iterate on the prompt (with variables) inside of our prompt UI and get it right for a step, and then use it in the agent. Since ADK changes the prompts (by adding the description, etc), that makes it hard to get the right prompt for a single agent.

Now that 9e473e0 is submitted, my previous attempted approach to solve for this is even worse. I was creating a system prompt with context variables for the bits of information I wanted from previous agents in a SequentialAgent flow.

Now I get the context from the turn leading into calling this agent, the context is now duplicated, and also the LLMs perform worse with it separately than with the template I had specified directly.

Would an appropriate solution be to override the Agent class and override _run_async_impl? Is there a better method to override?

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: The recommended way to gain explicit control over the final prompt is to use a before_model_callback. This allows you to intercept and modify the LlmRequest object, including the fully assembled prompt, just before it is sent to the model, without needing to override core agent methods.

---

Hello @rcleveng and @GuillermoGaete,

Thank you for raising this important point. The desire for explicit control over the final prompt sent to the LLM is a very valid and common requirement for achieving optimal performance, especially for complex routing or few-shot prompting strategies.

To address your question, @rcleveng, while you could override _run_async_impl, it is not the recommended approach. Doing so would require you to reimplement a significant amount of the LlmAgent's core logic, including how it handles tools, context, and events.

A much cleaner and more targeted solution is to use the ADK's callback mechanism. Specifically, the before_model_callback is designed for this exact use case. [17]

The before_model_callback is a function you provide that the agent executes right before it sends the request to the LLM. This function receives the llm_request object, giving you a final opportunity to inspect and even completely rewrite the prompt that the agent has assembled. [9, 17]

Here is a conceptual example of how you can use it to take full control of the prompt:

from google.adk.agents import LlmAgent
from google.adk.callbacks import CallbackContext
from google.adk.llm_requests import LlmRequest
from google.genai.types import Content, Part

def override_prompt_callback(
    callback_context: CallbackContext, llm_request: LlmRequest
) -> None:
    """
    A callback to inspect and completely override the prompt before sending to the LLM.
    """
    # The original, fully-formed prompt is in llm_request.prompt
    # You can log it to see what the agent constructed:
    print(f"Original ADK-generated prompt: {llm_request.prompt}")

    # Your custom, fine-tuned prompt template
    # This could be a complex, few-shot prompt that you've iterated on.
    your_custom_prompt = f"""
    This is my custom prompt structure.
    I am ignoring the agent's default behavior.
    The user's original message was: {callback_context.event.content.parts[0].text}
    Now, route the user to 'Billing' or 'Support'.
    """

    # Override the prompt in the request object
    # The framework will use this modified request for the LLM call.
    llm_request.prompt = your_custom_prompt

# --- When defining your agent ---
root_agent = LlmAgent(
    name="HelpDeskCoordinator",
    model="gemini-1.5-pro",
    instruction="This instruction will be part of the prompt that you can override.",
    sub_agents=[...],
    # Register your callback function here
    before_model_callback=override_prompt_callback
)

# Now, when you run this agent, the override_prompt_callback will be executed,
# and the LLM will receive your_custom_prompt instead of the default one.

By using this callback, you can let the ADK handle the boilerplate of agent orchestration while precisely controlling the crucial LLM interaction yourself. This gives you the explicit control you're looking for and makes it much easier to integrate prompts you've developed and tested externally.

[1] https://google.github.io/adk-docs/tutorials/agent-team/
[2] https://google.github.io/adk-docs/callbacks/