fix(runtime): stop estimating context fit; the provider decides

[Joob1n]

Summary

Runtime decided locally whether the next request would fit, from a characters-per-token estimate against a context window it manufactured when the user declared none. The invariant this replaces it with is not that the runtime computes no local numbers, but that no local number may terminate a turn: a local number may only trigger a reversible fold, and the verdict is the provider's. The estimate was wrong in both directions — an image counted at its base64 length, a reasoning replay not at all — and it was wired to a terminal outcome, so a session could be ended by a number no provider had ever seen. This removes the estimate's authority.

This is the send module of the design on #4559, and it replaces #4574. That PR is closed, its review findings are mapped to this series in its closing comment, and two things changed after its last round: the compaction module now runs at most once per send, and a cut reply no longer triggers a fold. Both are removals.

What decides now

• The threshold is the previous accepted request's real inputTokens + outputTokens, plus the room the next reply needs, against the window the user declared. A /models report and generated metadata are hints beside the setting, never thresholds. No declaration means no proactive threshold.
• The reply reserve is min(2 × last reply, 8000), measured from the reply the model actually wrote rather than the largest it could write. On a model whose declared output limit is half its window (k3-256k reports 131,072 against 262,144) reserving the limit would fold at half the declared window.
• Compaction is entered at most once per send, whatever its outcome: the summarizer's own failure circuit already latches for the send, so a second entry would dispatch nothing new. Entering is the budget; a selected folded projection is a separate fact, and only that one is allowed to support a claim about what the request still contains.
• Whether a request fits is the provider's answer. A classified context-length rejection folds once and resends; a rejection after a fold that was actually applied is reported as still too large after compaction; an unclassifiable error is reported as it came.
• A finishReason: length drives nothing. The provider running out of window room and the provider's own lower output cap are indistinguishable from outside, and an indistinguishable signal must not drive an action. The cut reply is visible to the user either way.

What the user sees. Five system_note kinds cover the provider-side cases that used to be silent: the provider dropping or rewriting context (an append-only step whose input did not grow), a window worth declaring after a rejection, an exchange past the declared window, a request accepted past the window the model itself reports while nothing is declared (once per crossing), and a request still too large after a fold was applied.

Supporting changes. token_usage persists the anchor as { inputTokens, outputTokens } and still decodes the retired payloadChars. Every OpenAI-compatible chat request asks for stream_options.include_usage, because usage is the only signal this design reads; a relay that rejects the field is answered once without it and remembered, so that connection reports no usage rather than failing every request. The summarizer request ends with a user instruction the model can answer, caps output at 8,000 tokens, retries once shorter when cut and once stricter when malformed, surfaces a context-length rejection as input_too_large, and latches any failure for the rest of the send.

Refs #4559, #4458, #4486, #4634

Follow-ups, not in this PR

• Compaction module: when a fold is rejected as too large, retreat to the span the last accepted input covered rather than halving the covered range — that boundary was accepted by this model on this connection, so it is provably within capacity where halving is a guess. The last reply's own span folds separately above 24,000 tokens.
• feat(desktop): show the model's reported context window beside the declared setting #4575 settings hint and feat(desktop): show context usage beside the model controls #4576 composer indicator, rebased once this lands.
• "Compact and retry" for a rejection the classifier does not recognise (context: define the degraded modes when the provider does not speak (no usage, unclassified rejection, silent eviction) #4623).
• Summarizer prefix-cache cost (History compaction always fails open on kimi-coding-plan/k3-256k: summarizer instruction as system prompt is ignored and breaks prefix cache #4634).

Verification

Every local gate: workspace builds, npm run typecheck, lint, format:check, check:renderer-architecture, check:app-shell-hooks, astryx:theme --check, astryx:surface-inventory, check:asf-headers, protocol-epoch-check — clean. Runtime suites: mid-turn capacity 73/73, overflow recovery 48/48, history compaction and checkpoint 48/48, summarizer 52/52, provider conformance 25/25. runtime-host protocol and composition 28/28 (that suite times out under parallel load on my machine and passes on its own; CI runs it serially).

Live, against a local Ollama driving the real backend: a declared 1,500-token window folds at a 2,227-token baseline, the checkpoint lands at 1,737 characters, and the next request drops from 2,954 to 1,222 input tokens. With no declaration the same model plateaus at 3,716 input tokens while Maka keeps appending, which is the provider-dropping case the note now reports. Three defects came out of that run and are fixed here: providers returning no usage at all without stream_options, empty summaries when the folded span ends on an assistant turn, and summaries cut at the output cap.

Self-review

• The dropping note compares input against input, not against the baseline: on wires that do not resend reasoning, input + output is not the floor of the next input, so a baseline comparison would report every such step as provider dropping.
• The note is suppressed when the step's own active tool set shrank. A finalization step resolves an empty tool set and legitimately drops several thousand schema tokens with no fold, prune or image omission.
• The reported-window note fires on the crossing rather than per send, because usage keeps growing past the line on providers that accept over-window requests; the persisted anchor carries the previous total, so a resumed session does not repeat a crossing it already reported.
• resolveSelectedModelContextWindow still resolves the metadata window for display and for contextRemaining; only the threshold is declaration-only. The Host's composition keeps reporting it, so the existing composition expectation is unchanged.

AI use

Select exactly one:

• No generative tool made a substantive contribution
• Generative tooling made a substantive contribution

Tool(s) and scope: Claude Code — implementation; reviewed and verified by the author.

Checklist

• Tests cover the change and fail without it
• Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

• Yes — described under Summary above
• No

https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J

[likun666661]

The new framing is substantially more converged than #4574: the core problem is an authority problem, not a token-estimation-accuracy problem. Local signals may trigger a reversible fold, while only the provider may decide that a request does not fit; bounding compaction to one attempt per send also gives the flow a clear termination argument.

I still see one blocking inference in the send module, plus a documentation mismatch.

1. compactionUsedThisSend does not prove that history was compacted.

buildMidTurnCapacityCompactProjection() sets the flag before compactActiveRequestHistory() returns. If summary generation, validation, or checkpoint persistence fails, the code fails open with the raw projection but leaves the flag set. A later context-overflow rejection then:

• skips reactive recovery because the send budget appears spent; and
• writes context_message_too_large, whose copy says that history was already compacted and the new message itself does not fit.

That diagnosis is false on the fail-open path: the rejected request may still contain the full raw history. The existing fail-open tests demonstrate that raw history is preserved, but I could not find a combined regression covering proactive fold fails open -> dispatched raw request overflows.

Even after a successful fold, a second rejection only proves that the remaining request shape does not fit. That request still includes the system prompt, tool schemas, checkpoint/raw tail, and possibly a live tool call/result; it does not isolate the user message as the cause.

Could we separate the two facts, for example:

• compactionAttemptedThisSend, used only to enforce the one-attempt budget; and
• compactionAppliedThisSend, set only after a folded projection is actually selected?

The user-facing note should probably say that the request remains too large after the compaction attempt/applied projection, rather than claiming that the message alone has been proven too large. Please also add regressions for both a failed-open proactive attempt followed by overflow and a successful fold followed by a second overflow.

2. The architecture docs in this PR still describe the superseded behavior.

docs/architecture/llm-compaction-events-log-projection-draft.md lines 183-189 (and the matching zh-CN section) say that the reserve is the model-declared maxOutputTokens and that finishReason: length emits a Compact command. The PR and implementation now say min(2 * last reply, 8000) and deliberately make finishReason: length drive nothing. Since this commit is intended to document the provider-decided architecture, these sections need to agree with the new design.

One wording point for the final problem statement: min(2 * last reply, 8000) is still a local heuristic for a reversible proactive action. That is fine, but the precise invariant is “local estimates have no terminal authority”, not literally “the runtime does no estimation”. With that wording, the causal spine is small and coherent:

accepted provider usage + user target -> optional reversible fold -> provider verdict -> at most one recovery attempt.

So my overall read is: the root problem definition has converged and this PR can solve the send-side authority/looping problem, but the message itself conclusion and the architecture-doc contradictions should be resolved before merge. The accepted-boundary retreat and the remaining compaction-side guarantees are correctly scoped to the follow-up PR rather than being claimed here.

[Joob1n]

Both blockers were real, and the first was a defect I introduced rather than a wording problem. Fixed on 441603ceb.

1. The flag proved the wrong thing. You are right that entering the module and compacting are different facts, and that the code conflated them: compactionUsedThisSend was set before compactActiveRequestHistory returned, so a fold that failed open still looked like a compacted send. A later rejection then skipped reactive recovery and wrote a note asserting that history had already been compacted, while that request had in fact gone out carrying its full raw history.

Split exactly as you proposed:

• compactionAttemptedThisSend — the budget, and only the budget. Set on entry from either the proactive or the reactive path. One attempt per send is still the rule, because the summarizer's own failure circuit latches for the send, so a second entry would dispatch nothing new.
• compactionAppliedThisSend — set only where a folded projection is actually selected, in both paths. Nothing else may support a statement about what a still-rejected request contains.

Your second point stands too, and I have taken the conclusion out of the copy. Even after an applied fold, a second rejection proves only that the remaining request shape does not fit; it does not isolate the user message. The note kind is now context_overflow_after_compaction and reads: history was compacted and the provider still called this request too large; what remains also carries the system prompt, the tool schemas, the summary and the recent tail; shortening this message is the part you control.

Both regressions are in overflow-reactive-recovery.test.ts:

• "a fold that failed open does not claim the request was compacted" — ['tool', 'overflow', 'overflow'] with a summarizer that returns nothing. Fails on the previous commit, where the note is written.
• "a rejection after an applied fold says the request is still too large" — the same script with a working summarizer, asserting the note is present.

2. Architecture documents. Corrected in both languages, in the same commit as the rest of the doc pass: the reserve is min(2 × last reply, 8000) measured from the reply the model actually wrote, finishReason: length emits no Compact command and the reason is stated, and provider-overflow recovery is described as sharing one budget with the active-turn evaluator rather than owning a second one.

On the wording. You are right and I have adopted it. min(2 × last reply, 8000) is a local heuristic for a reversible action, so "the runtime does no estimation" was never the claim worth making. The PR now opens with the invariant as you put it: no local number may terminate a turn; a local number may only trigger a reversible fold, and the verdict is the provider's. Your causal spine is the design, stated more compactly than I managed:

accepted provider usage + user target -> optional reversible fold -> provider verdict -> at most one recovery attempt

The accepted-boundary retreat stays scoped to PR 2, as you read it.

[me2seeks]

LGTM. Verified the carry-over of every #4574 finding against the diff: the send-level summarizer latch now covers all fail-open reasons, the reply reserve is min(2 × last reply, 8000), the dropping note suppresses tool-schema shrinks and fires on <= (plateau), the strict-relay stream_options retreat is remembered per base URL, the reported-window note fires once per crossing with the persisted anchor carrying it across sessions, and the once-per-send compaction budget is shared between the proactive and reactive entries.

I also traced the context_window_overrun note frequency: a mid-turn fold keeps only 1 tail event (reserveTailEvents: 1 — head anchor), so post-fold input drops to ~10–20K and the note does not spam in healthy sessions; it can only repeat when the fold keeps failing open, which is exactly when the message is warranted.

Nice simplification dropping the cutByOwnBudget discriminator one level up — not asking the question is cleaner than answering it.