[Tracing] Provide more subspans #875
Description
Similar to #868, but represented as spans.
Right now, requests appear to be taking way longer than I would expect, and I suspect that's because of the connection pool warming up, not the actual request. This should be visible in the emitted spans.
From #868:
- How long did this request spend in the connection pool queue
- How long did it take to make a connection (if one didn't already exist)
- How long did it take to send the request
- DNS timings
Mapped to spans, one way this could look that'd allow me to better understand where time is being spent:
| ---------------------------------------------- HTTPClient.execute ----------------------------------------------------- |
| ----------------------- Request 1 ---------------------- | | --------- Request 2 (redirect to the same host) -------- |
| ------ New connection ------ | | ----- HTTP req/res ---- | | --- Queued/reused connection --- | | -- HTTP req/res -- |
And zooming in:
| ------------------------ New connection ------------------------- |
| --- DNS lookup --- | | --- TCP est --- | | --- TLS handshake --- |
And (below is an example of bidi streaming, when the request and response bodies overlap; in a simple non-streaming example, the request and response wouldn't overlap - important bit is that all 4 subspans would have "HTTP req/res" as the parent span)
| ------------------------------------------- HTTP req/res ------------------------------------------ |
| --- HTTP req head --- | | ------------ HTTP req body ------ |
| --- HTTP res head --- | | ------- HTTP res body ---- |
The goal is for you to be able to gland at the trace and quickly identify where the bulk of time was spent - mainly: pre-request or inside the request.