My TCP connection is always established

[yuan-ze]

When I use AsyncClient to make concurrent requests, my TCP connection is always established. I suspect that the underlying connection is not completely disconnected because I receive a lot of timeout responses.
I don't have this problem when I use aiohttp, maybe you can check it

[lovelydinosaur]

Could you provide an example, and show me what behaviour you're seeing?

[yuan-ze]

this is my demo code

import asyncio
import httpx


async def fetch(client: httpx.AsyncClient) -> None:
    # 6. The address I am requesting currently has a high possibility of a connection timeout
    url = ''
    resp = await client.post(url)
    resp.raise_for_status()


async def handle(sem: asyncio.Semaphore, client: httpx.AsyncClient) -> None:
    try:
        # 5. use Semaphore
        async with sem:
            await fetch(client)
    except:
        pass


async def main():
    # 1. I use Semaphore to control the number of coroutines that send http requests
    sem = asyncio.Semaphore(15)
    # 2. I have configured the timeout and limits,
    #    I hope the number of his connections is not more than 10
    timeout = httpx.Timeout(0.1)
    limits = httpx.Limits(max_connections=10)
    client = httpx.AsyncClient(timeout=timeout, limits=limits, verify=False)
    while True:
        # 3.Simulate concurrent processing
        try:
            asyncio.create_task(handle(sem, client))  # 4 to my handle
            await asyncio.sleep(0.01)
        except KeyboardInterrupt:
            exit(0)


if __name__ == '__main__':
    asyncio.run(main())

After a while, I use the linux command to see the current number of connections

$ ss | grep ESTAB | grep ip:port | wc -l
19
$ ss | grep ESTAB | grep ip:port | wc -l
19
$ ss | grep ESTAB | grep ip:port | wc -l
15

You can see that the number of connections is not only greater than the coroutine limit, but also the httpx limit

In my production environment this is more severe.

But this doesn't happen when I use httpx.Client() , but this way it reduces the efficiency of my program

Then, I tried to use aiohttp and it didn't happen