Incomplete message body received when streaming response

[kpfleming]

(This appears to be related to #201, where this exception was added)

I've got some code (below) running in Python 3.8.3 (as packaged in RHEL 8), using httpx 0.17.1. I'm seeing random cases where an exception is thrown claiming that the connection was closed before the complete message body was sent, but I suspect the actual situation is that the connection was closed by the server after it had sent the entire body but before the entire body had been read from the response object.

Code:

async def blob_getter(client: httpx.AsyncClient, blob_dir: pathlib.Path, blob: BlobDetails) -> None:
    blob_path = blob_dir / blob.digest
    if blob_path.exists():
        logger.debug("%s blob %s already present", blob.purpose, blob.name)
        return
    logger.debug("Getting %s blob %s", blob.purpose, blob.name)
    try:
        blob_url = blob.job.name + "/blobs/" + blob.digest
        async with client.stream("GET", blob_url) as response:
            if response.is_error:
                logger.error(
                    "Failed %s blob %s - %s - %s",
                    blob.purpose,
                    blob.name,
                    response.status_code,
                    response.reason_phrase,
                )
                return
            async with aiofile.async_open(blob_path, "wb") as blob_file:
                async for chunk in response.aiter_bytes():
                    await blob_file.write(chunk)
        logger.debug("Completed %s blob %s", blob.purpose, blob.name)
    except asyncio.CancelledError:
        blob_path.unlink(missing_ok=True)
        logger.debug("Cancelled %s blob %s", blob.purpose, blob.name)
    except Exception as exc:
        blob_path.unlink(missing_ok=True)
        logger.error("Failed %s blob %s - %s", blob.purpose, blob.name, str(exc))
        raise

I'm seeing exceptions caught by the except Exception clause, and I'm 100% positive that the server did send the full body, but the socket-closure was noticed before the response had been fully ready.

For now, I'm going to catch this particular exception and 'absorb' it so that I can continue reading, but I'm wondering if there's something else I should be doing.

[lovelydinosaur]

Would you be able to reduce this down to an easily reproducible stand-alone example?