Unrecoverable httpx.PoolTimeout error after multiple httpcore.ReadTimeout errors when using the same AsyncClient

[RogerThomas]

I encountered the following issue when an upstream service became unavailable for approximately 5 minutes. This caused downstream requests to fail, and, even after the upstream service recovered, the downstream service remained in an unrecoverable state due to repeated httpx.PoolTimeout errors.

httpx version:

0.28.1

Current Behavior:

When using a single httpx.AsyncClient, after some amount of repeated httpcore.ReadTimeout errors, the client becomes unusable when it starts exhibiting httpx.PoolTimeout errors.
I.e. once the client gets in to this state, it becomes unable to make any further requests.

Expected Behavior:

After repeated httpcore.ReadTimeout errors, the AsyncClient should be able to still make requests to healthy endpoints.

Steps To Reproduce:

1. Clone this repo
2. Install go-task, uv and hey
3. In one termainl, start upstream service in a 'broken' state (returns 408s after 30s) - task serve-upstream -- true
4. In another terminal, start main service task serve-main
5. In another terminal, hit main service with multiple requests task hey-httpx
6. Switch to terminal from step 4 and watch logs until repeated httpx.PoolTimeout errors can be seen
7. Stop the hey-httpx task from step 5 (ctrl-C)
8. Switch to terminal from step 3, stop service (ctrl-c), restart service in 'fixed' state (returns 200s) - task serve-upstream -- false
9. The main service (step 4) should now be completely broken and no more requests can now be made, despite the fact that the upstream service has 'recovered'. This can be confirmed by sending a single request to upstream (to show that this is working) task test-upstream and sending a request to main service (to show that it is now completely broken) task test-main (should return 500s)

Anything else:

Can confirm the issue is with httpx as the same steps to reproduce can be run as above but with task hey-niquests instead of task hey-httpx. This causes niquests to be used as the http client to fetch the upstream service.

When using niquests, even after running the 'broken' upstream service for an abitrary amount of time, once you stop it and switch it to the 'fixed' service, the main service will always recover and will then continue to accept and return successful requests/responses

There may be easier/more-concise ways of reproducing this issue, but I wanted to share the exact scenario we encountered the issue in.