Tool Calling - access to ToolResponseMessage

[dolukhanov]

When using tools with ChatModel and internalToolExecutionEnabled set to true, the tool calls are made internally, and the results are passed directly to the AI model.

In many cases, it would be helpful to have access to the results of those tool calls for subsequent interactions. For example, if a tool returns a list of entities with both an ID and a name/label, the Assistant Message typically only exposes the name/label. If a later interaction requires the ID, it is no longer available.

Disabling internalToolExecutionEnabled and manually managing tool calls via ToolCallingManager solves this problem, since you can access toolExecutionResult.conversationHistory(), which includes all User, Assistant, and Tool messages.

However, is there a way to access the full conversation history — including ToolResponseMessages — without having to manually manage tool calls?

I tried using InMemoryChatMemory and MessageChatMemoryAdvisor, but they did not persist ToolResponseMessages.

If this functionality is not currently available, I believe it would be a valuable addition to the framework.

⸻

[myifeng]

+1

[Indra1806]

Step 1: Understand the Limitation
With internalToolExecutionEnabled = true, tool calls are executed internally and only the Assistant’s interpreted output is exposed.

This means ToolResponseMessages (raw tool outputs like IDs, metadata, etc.) are not persisted in memory by default.

As you noticed, InMemoryChatMemory and MessageChatMemoryAdvisor only store User and Assistant messages, not tool responses.

Step 2: Use Manual Tool Management (Current Workaround)

• Disabling internalToolExecutionEnabled and using ToolCallingManager allows you to capture the full conversation history, including ToolResponseMessages.

• Example:

java

ToolCallingManager manager = new ToolCallingManager();
ChatModel chatModel = new ChatModel(...);
manager.executeToolCall(toolRequest);

List<Message> history = manager.toolExecutionResult().conversationHistory();

Why this works:
Because you’re explicitly managing tool calls, the framework persists all message types (User, Assistant, Tool) in the conversation history.

Step 3: Extend Memory to Capture Tool Responses

• If you want to keep internalToolExecutionEnabled = true but still capture tool responses, you’ll need to extend the memory implementation.

• For example, subclass InMemoryChatMemory and override how messages are persisted:

java

public class ToolAwareChatMemory extends InMemoryChatMemory {

@Override

public void addMessage(Message message) {

super.addMessage(message);

if (message instanceof ToolResponseMessage) {

// Custom persistence logic

storeToolResponse((ToolResponseMessage) message);

}

}

}

Why this works:
By customizing memory, you ensure ToolResponseMessages are not discarded and can be retrieved later.

Step 4: Track Options in a Checklist

Here’s a quick comparison of approaches:

Approach	Captures ToolResponseMessages?	Effort	Notes
internalToolExecutionEnabled = true (default)	No	Low	Simplifies tool execution but hides raw responses
Disable internalToolExecutionEnabled + ToolCallingManager	Yes	Medium	Full control, but more boilerplate
Custom Memory (e.g., ToolAwareChatMemory)	Yes	High	Requires extending framework classes

Step 5: Future Suggestion

• As you mentioned, adding native support for persisting ToolResponseMessages would be a valuable enhancement.

• Until then, the manual manager or custom memory extension are the only reliable solutions.

Final Note

Right now, there’s no built-in way to access ToolResponseMessages with internalToolExecutionEnabled = true. If you want them, either:

1. Disable internal execution and use ToolCallingManager, or

2. Extend memory classes to persist tool responses yourself.

Both approaches ensure you retain IDs and metadata for subsequent interactions.