CLI broken (functionality does not work) on Windows: "'charmap' codec can't encode character"

[garretwilson]

Describe the bug

Running the following on Windows (in PowerShell or in Git Bash for Windows) produces a Python character encoding error. This implies that the Python library is assuming a charset rather than specifying wone.

aws agent-toolkit list-available-skills

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

The command aws agent-toolkit list-available-skills lists skills with no error.

Current Behavior

{
    "skills": [
…
        {
            "name": "amazon-documentdb",
            "description":

aws: [ERROR]: 'charmap' codec can't encode character '\u2192' in position 249: character maps to <undefined>

Reproduction Steps

1. Open PowerShell or Git Bash for Windows on Windows 11.
2. Issue aws agent-toolkit list-available-skills

Possible Solution

There is no known solution at the time. The CLI is completely broken.

Additional Information/Context

See UnicodeEncodeError: 'charmap' codec can't encode characters for possible background research.

This comes up in the AgentCore CLI as well; see aws/agentcore-cli#2010. The stack trace there indicated …\.venv\Lib\site-packages\bedrock_agentcore\runtime\app.py:_sync_stream_with_error_handling:945. There's no way to know without further investigation whether this is some shared AWS Python library, or bad Python Code AWS has duplicated across toolkits and/or CLIs, or what.

CLI version used

aws-cli/2.36.23 Python/3.14.6 Windows/11 exe/AMD64

Environment details (OS name and version, etc.)

Windows 11 Pro 25H2 26200.9168

[RyanFitzSimmonsAK]

Hi @garretwilson, thanks for reaching out. I was able to reproduce this behavior, and used the same workaround as in #8283. Setting the environment variable PYTHONUTF8=1 solved the issue.

Please let me know if you have any further issues.

[garretwilson]

Setting the environment variable PYTHONUTF8=1 solved the issue.

@RyanFitzSimmonsAK thank you for confirming the bug, and for suggesting a workaround.

However this is only a workaround. The customer should not have to set environment variables just to run the command-line tool. This bug needs to be fixed at its root cause. This is likely a bug in the assumptions made in the Python code. You'll see that this also shows up in Bedrock AgentCore, as I reported in aws/agentcore-cli#2010 . It would appear that AWS teams did not sufficiently test these tools on Windows, because I'm running into errors all over the place, such as aws/agentcore-cli#2036 and aws/agentcore-cli#1849.

In short, AWS Windows customers will hit blocking issues throughout current AWS tools, including AWS CLI and AgentCore CLI. I hope AWS will make Windows customers a priority and add more sufficient testing and fix these bugs.

[garretwilson]

Let me explain this a little more. PYTHONUTF8=1 is a "nuclear" option that sets the default encoding for Python across the entire system. But what if I have other Python code that wants to use the actual default system encoding?

The root problem is that there is likely some AWS code that wants to UTF-8, and won't work with any other encoding, but instead of calling the API and passing UTF-8 as the requested charset the developer didn't read the API and simply said, "I'll just use the default system charset; it works on my machine!" But the problem is that the developer's machine happened to use UTF-8 by default. The correct fix is find those places that do things requiring UTF-8 (such as open files or streams), and make sure that UTF-8 is actually requested rather than relying on the system default and assuming the system default is UTF-8.

This is a common developer mistake for decades. I called it out over a decade ago in my software development course lesson on text I/O, for example, when dealing with the analogous problem in Java:

Both InputStreamReader and OutputStreamWriter provide constructors that do not require a charset to be indicated. These constructors assume the default character encoding of the system. This is extremely dangerous and should be avoided. Different operating systems and configuration may at any time have different default character encodings, and relying on the default not only prevents your code from running consistently across systems, but likely ignores the actual encoding used by some byte stream which may be completely independent of the system. Always specify a charset when constructing an InputStreamReader or OutputStreamWriter.