Server-side requests/responses

[lovelydinosaur]

So, not something that we need to support as a primary use-case, but something we ought to just think about a little before 1.0, is making sure that our Request and Response models can be used on the server side as well.

For instance...

• It'd be beneficial if libraries like starlette were able to lean heavily on httpx.
• It'd make for easier custom transport implementations, if users were able to eg. instantiate a request instance, given the incoming raw request info. (Since that's a bit easier to worth with that the raw byte-wise API.)

A couple of API/implementation notes to go along with this...

• URL() should accept the raw bytes URL 4-tuple format that the httpcore API uses as one of it's options.
• Request.prepare(...) shouldn't be called automatically on __init__. The Client should call it explicitly in build_request(). This ensures that we'd be able to instantiate a Request instance from the raw HTTP core method/url/headers/stream` information, without having additional headers automatically generated on the model.
• We might want Request.raw, Response.raw, Headers.raw properties, to match up with the existing URL.raw property.

[florimondmanca]

Hey hey,

Interesting stuff, but what would be some possible use cases for this?

I'm thinking a use case of a Starlette server side app where we proxy requests from ASGI through HTTPX and responses back again. Is this discussion related to making this kind of things easier?

[lovelydinosaur]

So heres an example of something you might use this for...

class MockTransport():
    """
    A transport class that simply always returns a single given response.
    """
    def __init__(self, response: Response):
        self.response = response

    def request(self, ...):
        return self.response.raw

[jcugat]

Could this be used to build the httpx-equivalent of responses / aioresponses?

[florimondmanca]

@jcugat There's already RESPX by @lundberg which I think was intended to be such an equivalent (although I believe the API has parted ways a bit). :-) But yes this MockTransport example totally rings a bell and might be a way to make RESPX even simpler in design and implementation…

[lundberg]

True @florimondmanca , the API has parted somewhat, it's hard to keep up with the speed httpx evolves and improves, awesome work!

About the transports...latest version of respx is now implemented using the httpx transport API, and one can choose to use the respx sync/async mock transports directly if wanted. I have a small example in the docs.

I really like the idea of making the request, response and URL from httpx "easier" to use for an external lib @tomchristie . Currently I needed to have some wrappers in respx to fully support http_core "alone".

[johtso]

What's left to be done on this? I had a lot of awkward stuff going on in my custom transport that was alleviated by using a httpx.Request internally, very welcome change. I'm planning to do the same with Responses. Am I right in thinking this is already good to go?

[lovelydinosaur]

Yup you're good to go here. There are some bits & pieces that we might still pursue later.

For example... for accessing JSON or Form encoded data on a request instance, we might want to add .json() and .form() methods.

Aside to self: If we were going all the way with supporting server-side requests, then we'd also want to think about supporting ext containing http_version/client_addr/mount_path in order to give us parity with WSGI/ASGI interfaces.

[florimondmanca]

Is there anything left related to building requests/responses from "raw" data?

Like…

request = httpx.Request(raw=(method, url, headers, stream, ext))
response = httpx.Response(raw=(status_code, headers, stream, ext))

I think I saw this discussed on another issue or PR, but can't remember where. I think @johtso's experiments with caching hint that there might be a use case there…? (Maybe we already support it, btw, and I'm lacking behind?)

https://github.com/johtso/httpx-caching/blob/54251ea1297c6e50deec1e6570d3da62e847cc18/httpx_caching/_models.py#L19-L24