Recommended way to handle 503 responses from API?

[sbaack]

Hi everyone! Thanks for this wonderful tool. I was hoping to get some feedback and suggestions for something I wrote. I've build a simple tool that turns URLs in Markdown files into snapshots from archive.org. I use httpx's AsyncClient to call the API. The problem is that the API can be slow or unresponsive, in which case it returns a 503 reponse. I've tried to work around this by letting the individual task (each task is a call to the API) wait for five seconds and then retry for a maximum of five times (here is the code in context):

tries: int = 0
while tries < 5:
    try:
        response: httpx.Response = await client.get(api_call)
        response.raise_for_status()
        return response.json()
    except httpx.HTTPError:
        tries += 1
        print("Encountered HTTP error, waiting 5 seconds.")
        await asyncio.sleep(5)
        continue
else:
    sys.exit("API unresponsive, try again later.")

It kinda works, but I'm really unsure about the 'proper' way of doing this. Would be happy for any feedback and suggestions!

[lovelydinosaur]

The good first-pass answer for anything like this would be to figure out what folks are typically doing for requests?

(Since we're broadly API compatible)

[sbaack]

Thanks for the response! With requests it seems to be most common to use urllib3 for that. See for example this discussion. I also found this article that recommends the same and says that utilizing while-loops similar to how I do is a 'terrible solution' :-/ Given that urllib3 is synchronous, I assume I can't use it in conjuction with AsyncClient.

Edit: Oh you can also adjust requests' HTTPAdapter class for that

[lovelydinosaur]

I assume just setting max_retries on it's own will just enable connection retries.

I think the only support requests has for this is through configuring a urllib3 Retries instance.

We haven't implemented the same set of functionality in httpx yet, because it's kind of not in the core of requests either, and that's been our cut-off for helping figure out what stuff ought to be prioritised for a 1.0 release, and what stuff could wait until later down the road.

[sbaack]

Thanks for looking into this. Regardless of whether httpx implements this at some point in the future, I still wonder how to best handle client.get retries with AsyncClient at the moment. Is the while loop I'm using sensible? For aiohttp, there is aiohttp-retry (haven't tested it). I guess I could try tenacity

[StephenBrown2]

I guess I could try tenacity

I think tenacity is the recommended way forward at this current time.

[ugtar]

Tenacity is simple but leaves a lot to be desired since it doesn't understand the context of the request. For example, there is no point in retrying 404 errors, but it's generally expected that the client will retry 503 errors. As Tom mentioned, requests does have slightly more support for for complex retry setups, but generally requests punts on this and relies on urllib3's Retry class. It might be prudent for httpx to take the same tack

[sbaack]

Following @StephenBrown2's reply I now implemented this using tenacity. For the record, this is how the updated function currently looks like (I removed the docstring for this preview):

@tenacity.retry(stop=tenacity.stop_after_attempt(5), wait=tenacity.wait_fixed(2))
async def call_api(client: httpx.AsyncClient, api_call: str) -> dict[str, Any]:
    response: httpx.Response = await client.get(api_call)
    response.raise_for_status()
    return response.json()

To exit the program when all retries failed (again, code is truncated here):

async with httpx.AsyncClient(timeout=None) as client:
        # Building asyncio task list here, with each task calling call_api() shown above...
        try:
            api_responses: list[dict[str, Any]] = await asyncio.gather(*tasks)
        except tenacity.RetryError:
            sys.exit("API appears unresponsive, please try again later.")

Overall, the code is simplified and more readable. I didn't test this a lot, but it seems to work well. If you have any suggestions for improvements, please let me know :-)

Maybe consider adding a section about this to the docs, perhaps with a recommendation for tenacity? And as someone who is not familar with requests' more advanced options, it wasn't obvious to me what 'connection retries' means exactly in the section describing custom transports. Some claification would be helpful there as well.

[stevecj]

Of course, ideally, HTTPX would use exponential back-off and respect Retry-After headers as urllib3 does. Many services (e.g. Atlassian) expect the client to honor Retry-After.