Error in httpx.post() for application/json

[DDaamm]

Hi all! I have a problem in using new versions of httpx.
Recently I tried to update httpx package to 0.19 and 0.20 but, for both versions I face an issue in sending .json files through httpx.post() function. No changes were made inside my original code. This problem does not exist for version 0.18 (which I am actually using).

The code is pretty simple:

json_data = json.dumps({"key": "Test content"})
# Create the payload to send
json_file = File(payload=json_data, file_name="test.json", mime_type="application/json")
json_to_upload = AddDocumentsMultipartData([json_file])
request_body = json_to_upload .to_multipart()
#where url and client are defined elsewhere
response = httpx.post(url, headers=client.token_headers, files=request_body)

These are, respectively, the definition of File type:

@attr.s(auto_attribs=True)
class File:
    """Contains information for file uploads"""

    payload: Union[BinaryIO, TextIO]
    file_name: Optional[str] = None
    mime_type: Optional[str] = None

    def to_tuple(self) -> FileJsonType:
        """Return a tuple representation that httpx will accept for multipart/form-data"""
        return self.file_name, self.payload, self.mime_type

and the definition of AddDocumentsMultipartData class:

from typing import Any, Dict, List, Union
import attr
from ..types import UNSET, File, FileJsonType, Unset


@attr.s(auto_attribs=True)
class AddDocumentsMultipartData:
    """ """

    files: Union[Unset, List[File]] = UNSET
    additional_properties: Dict[str, Any] = attr.ib(init=False, factory=dict)

    def to_dict(self) -> Dict[str, Any]:
        files: Union[Unset, List[FileJsonType]] = UNSET
        if not isinstance(self.files, Unset):
            files = []
            for files_item_data in self.files:
                files_item = files_item_data.to_tuple()

                files.append(files_item)
        field_dict: Dict[str, Any] = {}
        field_dict.update(self.additional_properties)
        field_dict.update({})
        if files is not UNSET:
            field_dict["files"] = files
        return field_dict

    def to_multipart(self) -> Dict[str, Any]:
        if len(self.files) == 1:
            field_dict: Dict[str, Any] = {}
            field_dict["files"] = self.files[0].to_tuple()
            return field_dict
        else:
            files_list = []
            for item in self.files:
                files_item = item.to_tuple()
                files_list.append(("files", files_item))
            return files_list

When I run the script an error is rised: "TypeError: Expected bytes or bytes-like object got: <class 'str'>".
Here is the output:

Traceback (most recent call last):
  File ".\testing_script.py", line 78, in <module>
    resp = add_documents.add(id=cell_id, client=client, multipart_data=json_to_upload)
  File "D:\git\PyBeeYard\beeyard_sdk\api\document\add_documents.py", line 10, in add
    response = client.post(url, headers=client.token_headers, files=request_body)
  File "D:\git\PyBeeYard\beeyard_sdk\client.py", line 51, in post
    response = httpx.post(*args, **kwargs)
  File "C:\Users\tadeas.lejsek\AppData\Local\pypoetry\Cache\virtualenvs\beeyard-sdk-0U4yFQqS-py3.8\lib\site-packages\httpx\_api.py", line 304, in post
    return request(
  File "C:\Users\tadeas.lejsek\AppData\Local\pypoetry\Cache\virtualenvs\beeyard-sdk-0U4yFQqS-py3.8\lib\site-packages\httpx\_api.py", line 100, in request
    return client.request(
  File "C:\Users\tadeas.lejsek\AppData\Local\pypoetry\Cache\virtualenvs\beeyard-sdk-0U4yFQqS-py3.8\lib\site-packages\httpx\_client.py", line 776, in request
    request = self.build_request(
  File "C:\Users\tadeas.lejsek\AppData\Local\pypoetry\Cache\virtualenvs\beeyard-sdk-0U4yFQqS-py3.8\lib\site-packages\httpx\_client.py", line 346, in build_request
    return Request(
  File "C:\Users\tadeas.lejsek\AppData\Local\pypoetry\Cache\virtualenvs\beeyard-sdk-0U4yFQqS-py3.8\lib\site-packages\httpx\_models.py", line 1097, in __init__
    headers, stream = encode_request(content, data, files, json)
  File "C:\Users\tadeas.lejsek\AppData\Local\pypoetry\Cache\virtualenvs\beeyard-sdk-0U4yFQqS-py3.8\lib\site-packages\httpx\_content.py", line 179, in encode_request
    return encode_multipart_data(data or {}, files, boundary)
  File "C:\Users\tadeas.lejsek\AppData\Local\pypoetry\Cache\virtualenvs\beeyard-sdk-0U4yFQqS-py3.8\lib\site-packages\httpx\_content.py", line 124, in encode_multipart_data
    multipart = MultipartStream(data=data, files=files, boundary=boundary)
  File "C:\Users\tadeas.lejsek\AppData\Local\pypoetry\Cache\virtualenvs\beeyard-sdk-0U4yFQqS-py3.8\lib\site-packages\httpx\_multipart.py", line 161, in __init__
    self.fields = list(self._iter_fields(data, files))
  File "C:\Users\tadeas.lejsek\AppData\Local\pypoetry\Cache\virtualenvs\beeyard-sdk-0U4yFQqS-py3.8\lib\site-packages\httpx\_multipart.py", line 175, in _iter_fields
    yield FileField(name=name, value=value)
  File "C:\Users\tadeas.lejsek\AppData\Local\pypoetry\Cache\virtualenvs\beeyard-sdk-0U4yFQqS-py3.8\lib\site-packages\httpx\_multipart.py", line 86, in __init__
    raise TypeError(f"Expected bytes or bytes-like object got: {type(fileobj)}")
TypeError: Expected bytes or bytes-like object got: <class 'str'>

Am I doing something wrong or something is changed from 0.18 to 0.19 in the way the package is handling POST requests?

Thanks in advance!

[lovelydinosaur]

I face an issue in sending .json files through httpx.post() function

Could you describe what the issue is that you're seeing? Is an exception being raised? Is the response being returned okay, and what status code are you seeing?

[DDaamm]

You are right, I forgot to post the error. I've updated my post. Thanks!

[lovelydinosaur]

Gotcha - so uploads need to be bytes, rather than bytes-or-str, so...

json_data = json.dumps({"key": "Test content"}).encode("utf-8")

See also #1897

[DDaamm]

@tomchristie, first of all thanks for your fast reply. I have been doing many tries in these days. Also in the way you suggested to me:

json_data = json.dumps({"key": "Test content"}).encode("utf-8")
resp = httpx.post(url, headers=client.token_headers, files=json_data)

But then I get the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\damiano.peruzzi\PyBeeYard\.venv\lib\site-packages\httpx\_api.py", line 304, in post
    return request(
  File "C:\Users\damiano.peruzzi\PyBeeYard\.venv\lib\site-packages\httpx\_api.py", line 100, in request
    return client.request(
  File "C:\Users\damiano.peruzzi\PyBeeYard\.venv\lib\site-packages\httpx\_client.py", line 780, in request
    request = self.build_request(
  File "C:\Users\damiano.peruzzi\PyBeeYard\.venv\lib\site-packages\httpx\_client.py", line 347, in build_request
    return Request(
  File "C:\Users\damiano.peruzzi\PyBeeYard\.venv\lib\site-packages\httpx\_models.py", line 1102, in __init__
    headers, stream = encode_request(content, data, files, json)
  File "C:\Users\damiano.peruzzi\PyBeeYard\.venv\lib\site-packages\httpx\_content.py", line 185, in encode_request
    return encode_multipart_data(data or {}, files, boundary)
  File "C:\Users\damiano.peruzzi\PyBeeYard\.venv\lib\site-packages\httpx\_content.py", line 130, in encode_multipart_data
    multipart = MultipartStream(data=data, files=files, boundary=boundary)
  File "C:\Users\damiano.peruzzi\PyBeeYard\.venv\lib\site-packages\httpx\_multipart.py", line 166, in __init__
    self.fields = list(self._iter_fields(data, files))
  File "C:\Users\damiano.peruzzi\PyBeeYard\.venv\lib\site-packages\httpx\_multipart.py", line 179, in _iter_fields
    for name, value in file_items:
TypeError: cannot unpack non-iterable int object