Investigations: an alert that keeps firing after the report opens a new request on each re-fire #6778

Claude posting for Erik Darling.

Problem

A reported investigation request takes no more alerts (#5632). So when an alert keeps firing after the agent reports, each re-fire opens a new request.

On the larger production store on 26 Oct, one server had a High CPU episode from 06:02 to 07:56 UTC. The engine evaluated the alert about every 5 minutes. The agent reported each request in about 12 seconds. So each re-fire found only a reported request, and it opened a new one. The episode made 22 requests with one alert each. The later reports added nothing new, except a rising session count. All 22 requests also counted against the route's hourly and daily limits.

Rule (agreed by Erik)

An alert joins a reported request quietly when all of these are true:

• The server and the family are the same, and the alert's metric is already on the request.
• The alert's severity is the same as the request's max_severity, or lower.
• The request has no resolved journal row (Request events: journal event log, resolved detector, fallback delivery, MCP cursor and wake (#5632 Phase C C7) #6003, D7).
• The alert is inside the request's grouping window. That is, it fires within 30 minutes of the request's latest alert, and no later than 4 hours after its first alert.

These alerts still open a new request:

• an alert with a higher severity
• an alert with a new metric
• an alert after the request resolved
• an alert after the 4-hour cap

A closed request stays final.

The join is journaled as joined_quiet, so no reader delivers it. A join never counts against a limit. The report does not change. The request's alert rows and its alert count show how long the condition kept firing.

Why the grouping window, and not the throttle

The deliverer opens or joins the request before the send core applies the cooldown. The order is necessary, because the generic webhook's {{investigation_json}} needs the request (#5821). So at that point the store cannot know if the page will be throttled. The grouping window is the store's existing bound for one incident. With it, the episode above opens one request, not 22. An episode that continues for more than 4 hours opens one request for each 4 hours.

Scope

• Darling: InvestigationRequestStore.FindJoinSql and OpenOrJoinAsync.
• Lite: DuckDbInvestigationStore, the same change under the two-store parity rule.
• The expires_at > now clause is the claim deadline. It applies to open and claimed requests only. A reported request is past its claim, and the 4-hour cap bounds it.
• Update the class remarks and the tests that say "a reported request is final".