feat: queue time capture for Rack#2838
Conversation
|
sentry-ruby/lib/sentry/span.rb
Outdated
| MESSAGING_MESSAGE_RETRY_COUNT = "messaging.message.retry.count" | ||
|
|
||
| # Time in ms the request spent in the server queue before processing began. | ||
| HTTP_QUEUE_TIME_MS = "http.queue_time_ms" |
There was a problem hiding this comment.
@AbhiPrasad @lcian is there an attribute convention for time spent in queue waiting for the http request to be picked up in an HTTP server?
The closest I could find is http.client.request.time_in_queue but that's for clients.
There was a problem hiding this comment.
Hmmm I cannot find anything like that. Maybe we introduce our own http.server.request.time_in_queue counterpart?
There was a problem hiding this comment.
sounds good; shall I add to sentry-conventions too?
sl0thentr0py
left a comment
There was a problem hiding this comment.
some small stuff, and mainly want to wait for feedback on the attribute convention name
Co-authored-by: Neel Shah <neel.shah@sentry.io>
solnic
left a comment
There was a problem hiding this comment.
✅ but I left a bunch of minor 💅🏻 suggestions 🙃
Co-authored-by: Peter Solnica <peter@solnica.online>
Co-authored-by: Peter Solnica <peter@solnica.online>
Non-numeric t= values like "t=invalid" or "t=" were silently converted to 0.0 by Ruby's String#to_f, resulting in a Unix epoch timestamp and a queue time of ~56 years instead of nil. Apply the same numeric regex guard already used for raw timestamps to the t= branch before calling to_f. Co-Authored-By: Claude <noreply@anthropic.com>
Co-authored-by: Peter Solnica <peter@solnica.online>
Co-authored-by: Peter Solnica <peter@solnica.online>
Co-authored-by: Peter Solnica <peter@solnica.online>
Co-authored-by: Peter Solnica <peter@solnica.online>
Co-authored-by: Peter Solnica <peter@solnica.online>
Co-authored-by: Peter Solnica <peter@solnica.online>
parse_request_start_header returned nil for headers that arrived with leading/trailing whitespace or as comma-separated values (multiple header occurrences collapsed by a proxy). Strip whitespace from the token and split on commas, taking the first (earliest) timestamp. Co-Authored-By: Claude <noreply@anthropic.com>
When env["puma.request_body_wait"] is a String (e.g. serialized by
middleware), the `> 0` comparison raised ArgumentError, which was
silently swallowed by the broad rescue, causing queue time to be
dropped even with a valid X-Request-Start header.
Call to_f on String values before the comparison. Numeric strings
("40") convert correctly; non-numeric strings ("N/A") become 0.0
and are treated as absent, so total_time_ms is returned unchanged.
Co-Authored-By: Claude <noreply@anthropic.com>
…-ruby into puma-queue-time
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| queue_time_ms >= 0 ? queue_time_ms : 0.0 # more sanity check | ||
| else | ||
| total_time_ms | ||
| end |
There was a problem hiding this comment.
Queue time spoofable via client header
Medium Severity
extract_queue_time trusts env["HTTP_X_REQUEST_START"] without verifying it was set/overwritten by a trusted proxy and without any upper-bound sanity check, so a client-supplied X-Request-Start can inject arbitrarily large http.server.request.time_in_queue values and skew transaction data.
Adds the `http.server.request.time_in_queue` semantic convention attribute to capture the time (in milliseconds) a request spent waiting in the server queue before processing began. This attribute is populated from the `X-Request-Start` header set by reverse proxies (Nginx, HAProxy, Heroku router, etc.) and first introduced in sentry-ruby: getsentry/sentry-ruby#2838 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(http): add http.server.request.time_in_queue attribute Adds the `http.server.request.time_in_queue` semantic convention attribute to capture the time (in milliseconds) a request spent waiting in the server queue before processing began. This attribute is populated from the `X-Request-Start` header set by reverse proxies (Nginx, HAProxy, Heroku router, etc.) and first introduced in sentry-ruby: getsentry/sentry-ruby#2838 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(http): add generated attribute definitions for http.server.request.time_in_queue Generate TypeScript and Python attribute definitions including type exports, metadata entries, and TypedDict annotations. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>


Captures how long requests wait for a e.g. Puma thread from
X-Request-Startheaders and attaches it to transactions ashttp.server.request.time_in_queue. Sentry only measures after Puma picks up the request. Under load, requests might wait long in the queue but only take a fraction to process. Similar to what judoscale and scout do.works with all major reverse proxies:
X-Request-Start: t=1234567890.123(seconds)X-Request-Start: t=1234567890123456(microseconds)X-Request-Start: t=1234567890123456(microseconds)X-Request-Start: t=1234567890(seconds)subtracts
puma.request_body_waitfor accuracy (excludes slow client uploads from queue time).Usage
works 🤞 if your reverse proxy sets the header:
opt-out