Automatically use Responses API previous_response_id instead of sending message history when possible

[GDaamn]

Closes #2663

• Add openai_previous_response_id field to store response id (provider_response_id) from the latest response sent by Responses API, which can then be reused as previous_response_id in subsequent requests.
• Update logic to trim message history: instead of sending the entire history, pass the provider_response_id as previous_response_id in the next request, allowing the Responses API to access prior reasoning on the server side. This is used when message history is passed instead of openai_previous_response_id, or during single agent.run with multiple request/response.
• Supporting docs and unit tests.

[DouweM]

If the matched ModelResponse was not the most recent one, we should include all the messages that came after it, not just the last one

[GDaamn]

Here, I was also thinking, if the user sends message_history with results from different models combined, it becomes tricky to handle. Something like

[
  ModelResponse(..., model_name='claude-3-5-sonnet-20241022',....),
  ModelResponse(..., model_name='gpt-5-2025-08-07',...., provider_response_id=resp_123),
  ModelResponse(..., model_name='claude-3-5-sonnet-20241022',....)
]

The first claude response will be excluded. wdyt ?

[DouweM]

That should be OK since the model_name='gpt-5-2025-08-07',...., provider_response_id=resp_123 indicates that that response (and the history before it) has already been sent to the Responses API, right? So it will have that model response in the server-side history, even though it originally came from another model.

[GDaamn]

Indeed that is true.
I was thinking of a scenario like (not sure how common this is):

agent = Agent('anthropic:claude-3-5-sonnet-latest')
r1 = await agent.run(user_prompt="tell me something about rockets")

model = OpenAIResponsesModel('gpt-5')
agent = Agent(model)
r2 = await agent.run('what are rockets used for')

agent = Agent('anthropic:claude-3-5-sonnet-latest')
r3 = await agent.run(user_prompt="which countries have sent rockets to space")

history = r1.all_messages() + r2.all_messages() + r3.all_messages()

model = OpenAIResponsesModel('gpt-5')
agent = Agent(model)
result = await agent.run('summarize the conversation so far', message_history=history)

[DouweM]

@GDaamn Hmm, I don't think that's common but it is indeed technically supported. Then maybe we should only send previous_response_id if all responses came from the matching model.

[DouweM]

This can be == right? And be merged with the previous if statement?

[GDaamn]

Even when the model is set:

model = OpenAIResponsesModel('gpt-5')
agent = Agent(model)
result = await agent.run('what is the capital of france')
print(result.all_messages())

#> [ModelRequest(parts=[UserPromptPart(content='what is the capital of france', timestamp=datetime.datetime(2025, 9, 3, 14, 12, 42, 548683, tzinfo=datetime.timezone.utc))]),
 ModelResponse(parts=[TextPart(content='Paris.')], usage=RequestUsage(input_tokens=12, output_tokens=8, details={'reasoning_tokens': 0}), model_name='gpt-5-2025-08-07', timestamp=datetime.datetime(2025, 9, 3, 14, 12, 42, tzinfo=TzInfo(UTC)), provider_name='openai', provider_response_id='resp_68b84cdac694819493f5b822d10a152c0946e65b063355c0')]

OpenAI returns with a dated version(gpt-5-2025-08-07) of the given model

[DouweM]

Ah OK, that's worth adding a comment then

[DouweM]

If a openai_previous_response_id was explicitly set, I think we should not do the stuff we do above, and assume the user only passed in the messages in message history that they actually meant to send

[DouweM]

If we find a ModelResponse with out a model_name, we also want to reset response_id + break right? I think having the third branch made sense. We can change the if isinstance(m, ModelResponse) and m.model_name line to if isinstance(m, ModelResponse) and m.model_name and self.model_name in m.model_name

[DouweM]

I think we should account for https://ai.pydantic.dev/message-history/#processing-message-history. It would be unexpected if that feature is ignored entirely when using the Responses API. Maybe we should not automatically use previous_response_id if a history processor was configured?

[DouweM]

Additionally, https://platform.openai.com/docs/guides/reasoning?api-mode=responses#encrypted-reasoning-items mentions the Responses API "stateless mode (either with store set to false, or when an organization is enrolled in zero data retention)", in which case previous_response_id wouldn't work. So there needs to be a way to turn it off.

Perhaps it's better to make it opt-in by setting OpenAIModelSettings(openai_previous_response_id='auto') or something?

[GDaamn]

1. for history_processor, I did some quick experiments and it seems it isn't intefering. Also checking the code I can see that the the history is processed inside _prepare_request, a step before we pass the messages to model request where the previous_response_id logic happens.
   

   pydantic-ai/pydantic_ai_slim/pydantic_ai/_agent_graph.py

   Lines 422 to 442 in 3c2f1cf

   	model_settings, model_request_parameters, message_history, _ = await self._prepare_request(ctx)
   	model_response = await ctx.deps.model.request(message_history, model_settings, model_request_parameters)
   	ctx.state.usage.requests += 1
   	return self._finish_handling(ctx, model_response)
   	async def _prepare_request(
   	self, ctx: GraphRunContext[GraphAgentState, GraphAgentDeps[DepsT, NodeRunEndT]]
   	) -> tuple[ModelSettings | None, models.ModelRequestParameters, list[_messages.ModelMessage], RunContext[DepsT]]:
   	ctx.state.message_history.append(self.request)
   	ctx.state.run_step += 1
   	run_context = build_run_context(ctx)
   	# This will raise errors for any tool name conflicts
   	ctx.deps.tool_manager = await ctx.deps.tool_manager.for_run_step(run_context)
   	message_history = await _process_message_history(ctx.state, ctx.deps.history_processors, run_context)

Please lmk if there are other scenarios where this is not the case or I misunderstood the flow

[GDaamn]

2. Regarding "stateless mode", thanks for pointing out. In that case we can modify this:

 previous_response_id = model_settings.get('openai_previous_response_id')
        if not previous_response_id:
            messages, previous_response_id = self._get_response_id_and_trim(messages)

to this right ?

 previous_response_id = model_settings.get('openai_previous_response_id')
        if previous_response_id == 'auto':
            messages, previous_response_id = self._get_response_id_and_trim(messages)

-> when user passes message_history, it will be trimmed only whenopenai_previous_response_id = 'auto' and the ModelResponses are from the same model.
But in this case, we'll still have to explicitly set openai_previous_response_id = 'auto', when mutliple requests are made within the same agent.run (like part of a tool call etc).