Question: Best way to pass an image to Agent with a tool

[JoshuaSiraj]

I'm looking to have a tool do something like this:

def load_image(path: str) -> str:
    return Path(path).read_bytes()

What is the best way to do this?

[Yung-Chih-Lo]

same question

[zhangp365]

I usually use two methods:

1. Use a URL, the agent will take this url as a parameter to the tool.
2. Set google/adk/agents/ run_comfig.py file
   save_input_blobs_as_artifacts: bool = True
   Then the user can upload the image in the Adk web, and the agent can take the artifact file name as a parameter to the tool.

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: The recommended way to handle files and binary data like images in ADK is to use the artifact management system. Instead of returning the image bytes directly, your tool should save the image as an artifact using tool_context.save_artifact().

---

Hello! As an AI agent, I'd be happy to help with your question.

ADK provides an artifact management system that is specifically designed for handling files and binary data, such as images. [1] The best practice is to use this system within your tools.

Here’s how you can modify your load_image tool to use the artifact management system. This example assumes your tool has access to the tool_context.

from pathlib import Path
from google.genai import types
from adk.tools import ToolContext

def load_image(path: str, tool_context: ToolContext) -> str:
    """Loads an image from a path and saves it as an artifact."""
    image_bytes = Path(path).read_bytes()
    # Create a Part object from the image bytes
    image_part = types.Part.from_bytes(image_bytes, mime_type='image/png') # Or other appropriate mime_type

    # Save the image as an artifact
    artifact_filename = "loaded_image.png"
    tool_context.save_artifact(artifact_filename, image_part)

    return f"Image loaded from {path} and saved as artifact: {artifact_filename}"

In this example:

1. The image data is read from the specified path.
2. A google.genai.types.Part object is created from the image bytes.
3. The tool_context.save_artifact() method is used to save the Part object as an artifact with a specified filename. [2]
4. The tool then returns a string confirming that the artifact was saved.

You can later use the tool_context.load_artifact() method in another tool to load the image data for processing. [3]

I hope this helps you build your agent!

---

[1] About ADK
[2] agent.py example
[3] Tools documentation