Python: Fix Bedrock non-ASCII escaping in JSON content blocks#6628
Open
kimnamu wants to merge 1 commit into
Open
Python: Fix Bedrock non-ASCII escaping in JSON content blocks#6628kimnamu wants to merge 1 commit into
kimnamu wants to merge 1 commit into
Conversation
The Bedrock Converse `json` content block was serialized with `json.dumps(json_value)`, whose default `ensure_ascii=True` escapes CJK/emoji/accented characters to `\uXXXX` and surfaces garbled text. Add `ensure_ascii=False` to match the sibling OpenAI client and the 16+ other call sites across the repo. Includes a regression test. Closes microsoft#6627 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes a Bedrock Converse response parsing edge case where structured json content blocks were being serialized with json.dumps(..., ensure_ascii=True) (default), causing non‑ASCII characters to be escaped into \uXXXX sequences in user-visible text.
Changes:
- Update Bedrock
jsoncontent block serialization to useensure_ascii=Falseso non‑ASCII characters are preserved. - Add a regression test ensuring CJK text and emoji remain unescaped and the emitted text still round-trips via
json.loads.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
python/packages/bedrock/agent_framework_bedrock/_chat_client.py |
Preserves non‑ASCII characters when converting Bedrock json content blocks into text. |
python/packages/bedrock/tests/test_bedrock_client.py |
Adds a unit test verifying non‑ASCII preservation and preventing regression. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thanks for Agent Framework and the Bedrock integration — it's a pleasure to build on.
Closes #6627
Problem
When the Bedrock Converse API returns a structured
jsoncontent block,BedrockChatClient._parse_message_contentsserializes it to text withjson.dumps(json_value). Sincejson.dumpsdefaults toensure_ascii=True, non-ASCII characters (CJK, emoji, accented Latin, etc.) are escaped to\uXXXXand reach the user garbled.Cause
This single call is the outlier. The sibling OpenAI client (
python/packages/openai/agent_framework_openai/_chat_client.py) and 16+ other call sites across the repo already serialize user-facing / span data withensure_ascii=False. PR #3894 ("Python: Fix non-ascii chars in span attributes") fixed the same class of issue in observability — this is the matching fix the Bedrock client was missing.Change
One line: add
ensure_ascii=Falseto thejson.dumpsfor the Bedrockjsoncontent block, plus a regression test.Before / After
jsonblock{"greeting": "你好世界"}→ text{"greeting": "你好世界"}{"greeting": "你好世界"}"🎉"injsonblock"🎉""🎉"block.get("text"))json.loads)Test (red → green)
New test
test_process_converse_response_preserves_non_ascii_in_json_blockuses the existing key-free_StubBedrockRuntimepattern.Against the unpatched source (bug present):
With the fix:
Full bedrock suite (no regressions) + lint/format clean:
This contribution was prepared with the help of an AI agent (Claude Code); a human reviewed the change, rationale, and test results before submitting.