AsyncClient does not detect length of file-like object

[rubenbaer]

This is related to issue #1490. The blocking httpx.Client detects Content-Length for file like objects. But this is not implemented for httpx.AsyncClient (see

httpx/httpx/_content.py

Line 110 in 9baf3a6

def encode_content(

).

I am using httpx to upload files with presigned URLs to a MinIO AWS S3 target. Since a blocking upload is not cancellable by Trio, I prefer to use httpx.AsyncClient. The current workaround is to set the Content-Length header explicitly. However, the peek_filelike_length function also works in the async context and changing the encode_content function accordingly should also work.

import httpx, trio
import tempfile


def put_sync(filename: str):
    with httpx.Client() as client:
        with open(filename, 'rb') as f:
            response = client.put("https://httpbin.org/anything", content=f)
            print(response.request.headers)

async def put_async(filename: str):
    async with httpx.AsyncClient() as client:
        async with await trio.open_file(filename, 'rb') as f:
            response = await client.put("https://httpbin.org/anything", content=f)
            print(response.request.headers)

f = tempfile.NamedTemporaryFile()
f.write(b"foobar")
f.flush()

put_sync(f.name)

trio.run(put_async, f.name)
f.close()

Expected

Sync and async code should be similar.

Actual

Headers({'host': 'httpbin.org', 'accept': '*/*', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'user-agent': 'python-httpx/0.23.0', 'content-length': '6'})
Headers({'host': 'httpbin.org', 'accept': '*/*', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'user-agent': 'python-httpx/0.23.0', 'transfer-encoding': 'chunked'})

Context

PUT with presigned AWS S3 URL fails as Transfer-Encoding chunked is not supported.

[lovelydinosaur]

However, the peek_filelike_length function also works in the async context and changing the encode_content function accordingly should also work.

Okay - I reckon we should do that then. Yes it's a blocking operation, which we wouldn't normally want to do in an async context, but in practise making sync file accesses like that is actually perfectly okay performance-wise.

[rubenbaer]

@tomchristie thanks! I can very happily create a pull request.