Skip to content

Conversation

DayuanJiang
Copy link

@DayuanJiang DayuanJiang commented Oct 14, 2025

Description

Introduced the cache_messages configuration parameter to BedrockModel to enable message caching for intermediate steps.

When cache_messages = "default", the agent will add a cache point at the end of the message list each time it calls a Bedrock model. This allows the agent to cache intermediate steps effectively while preserving any existing cache points users may have set.

Key implementation details:

  • Adds a new cache point at the end of the last message
  • Creates copies of message structures to avoid modifying the original messages passed to format_request

Related Issues

#404 #1015

Documentation PR

None

Type of Change

New feature

Testing

  • test_format_request_cache_messages(): Validates that existing cache points are preserved and a new cache point is added at the end

  • test_format_request_cache_messages_does_not_modify_original(): Verifies that the original messages passed to format_request are not mutated

    • Tests preservation of existing cache points from multiple messages
    • Verifies correct addition of new cache point at the end of the last message
    • Confirms complete request structure matches expectations
    • Ensures no mutation of original message objects
  • I ran hatch run prepare

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

cleaned_messages.append({"role": message["role"], "content": cleaned_content})
else:
cleaned_messages.append(message)
return cleaned_messages
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain the purpose of clearing cachePoints from previously processed messages?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing that out. Originally, the implementation would add a cache point to the last message on each call. To avoid accumulating too many cache points across multiple calls, I decided to remove all existing cache points before adding the new one.

However, I realized that this approach was problematic because it modified the original messages passed to the function. So I modified the code to create a copy of the messages and add the cache point to the copied version, which means clearing existing cachePoints is no longer necessary.

I have also updated the PR description to reflect this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants