Skip to content

fix(runtime): retreat a rejected fold to a span the provider has accepted #4667

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
likun666661 merged 1 commit into from
Sep 3, 2026
Merged

fix(runtime): retreat a rejected fold to a span the provider has accepted #4667

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@
owner, immutable request snapshots remain enforced at AgentRun acceptance and backend dispatch,
and SessionEvent-to-RuntimeEvent conversion remains a pure mapper.
- Retired the Task Ledger domain: SessionTodo is now the sole authority for in-session work items, and the operational-state schema drops the `workflow_task_ledger_events` table on first open. **Unfinished Tasks are not migrated and are permanently deleted.** This affects workspaces last opened by `v0.1.0` through `v0.1.11`, `cli-v0.1.0-beta.1`, `v0.2.0-incubating-rc1`, or a `v0.2.0-dev` build; those releases wrote Tasks to a table that no shipped build ever bridged into SessionTodo. Before opening such a workspace with this build, finish or export the Tasks you still need, or copy the workspace's `runtime.sqlite` aside — the migration removes the only live copy, so afterwards recovery requires a backup made in advance.
- A compaction rejected as too large for the summarizer's own window now retreats to the span the last accepted request's input covered, instead of halving the covered range. That span is the newest reply this route produced, found through the run headers, so it was accepted by this model on this connection and is provably within capacity; halving can overshoot (discarding verbatim history for nothing) or undershoot (paying another round trip), and a span another route accepted proves nothing at all. One retreat, then the fold fails open and the provider decides.

@Astro-Han Astro-Han Sep 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 This is true for step-0 recovery and not for /compact, supervisor-wake compaction, or the pre-turn fallback, where there are zero retreats. Land the wiring rather than rewording the line, since rewording would document the gap.

- `token_usage` anchors now record the model and connection that produced them. A token count is a number in one model's tokenizer against one connection; carrying the route on the record lets any reader apply the rule the runtime already enforces, instead of pairing one model's usage with another model's window. The record decodes against a closed allowlist, so sessions written with these keys do not open in earlier releases, and the Runtime Host compatibility epoch moves to 107.
- Let the provider decide whether a request fits. Proactive compaction now uses only a user-declared Maka window and the previous accepted request's provider-reported `inputTokens + outputTokens`; no declaration means no proactive capacity threshold. `/models` and generated model metadata are display hints, not limits. `token_usage` records persist the last-request anchor under `lastRequestAnchor`; its new `{ inputTokens, outputTokens }` shape still decodes the retired `payloadChars` key from older sessions. Requests that are too large are compacted and retried once after a real provider rejection, then reported as a `context_overflow` provider error. Compaction is entered at most once per send, and a request rejected after a fold was actually applied is reported as still too large after compaction. A fold that failed open makes no such claim: that request went out with its full raw history. A reply cut at `finishReason: length` no longer triggers a fold, because the provider running out of window room and the provider's own lower output cap are indistinguishable from outside. Five system notes explain the provider-side cases: dropping context, a window worth declaring, an exchange past the declared window, a request accepted past the window the model reports (once per crossing, while nothing is declared), and a request still too large after compaction. The reply reserve that arms the proactive threshold is twice the last real reply, bounded at 8,000 tokens, rather than the model's maximum output. **Sessions this build writes do not open in earlier releases:** those decode `token_usage` against a closed allowlist, so the reshaped `lastRequestAnchor` key fails the record and, with it, the Session that contains it; downgrading therefore needs a copy of the workspace's `runtime.sqlite` taken before the upgrade. Nothing produces the `context_budget_exhausted` stop reason any more — a request that really is too large is compacted and retried once, then reported as a `context_overflow` provider error — though sessions that already recorded it still decode and present. The Runtime Host compatibility epoch moves to 106.
- Unified context management under one Runtime-owned policy. `MAKA_CONTEXT_*` environment overrides no longer tune or disable compaction and Tool Result pruning; model-visible archive placeholders are read on demand through bounded `ArchiveRead` calls instead of eager hydration. Previously supported overrides are ignored on upgrade: if Tool Result pruning was set to `off`, pruning is re-enabled, and there is currently no supported replacement opt-out.
Expand Down
214 changes: 207 additions & 7 deletions packages/runtime/src/__tests__/history-compaction.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@

import assert from 'node:assert/strict';
import { describe, test } from 'node:test';
import type { AgentRunHeader } from '@maka/core/agent-run';
import type { RuntimeEvent } from '@maka/core/runtime-event';
import {
applyRuntimeEventHistoryCompact,
Expand Down Expand Up @@ -176,7 +177,10 @@ describe('plan context compaction', () => {
assert.deepEqual(result.replacementEvents[1], events[2]);
});

test('retreats the safe prefix by half for each input-too-large rejection', async () => {
test('retreats to the span the last accepted input covered', async () => {
// The newest reply's events end that span: everything before the first of
// them was in the request the provider accepted, so it is provably within
// capacity. Halving would be a guess in either direction (#4559).
const events = [
user('old-user', 'old-turn'),
model('old-model', 'old-turn', 'old result'),
Expand All @@ -188,10 +192,12 @@ describe('plan context compaction', () => {
planInput({
phase: 'standalone',
orderedEvents: events,
runHeaders: HEADERS_A,
acceptedRoute: ROUTE_A,
reserveTailEvents: 0,
summarize: ({ coveredRuntimeEvents }) => {
attemptedCoverage.push(coveredRuntimeEvents.map((event) => event.id));
if (attemptedCoverage.length <= 2) {
if (attemptedCoverage.length === 1) {
throw new HistoryCompactSummarizerError('input_too_large');
}
return structuredSummary('A bounded automatic summary.');
Expand All @@ -203,24 +209,195 @@ describe('plan context compaction', () => {
if (result.decision !== 'compacted') return;
assert.deepEqual(attemptedCoverage, [
['old-user', 'old-model', 'recent-user', 'recent-model'],
['old-user', 'old-model'],
['old-user'],
['old-user', 'old-model', 'recent-user'],
]);
assert.deepEqual(
result.tailRuntimeEvents.map((event) => event.id),
['old-model', 'recent-user', 'recent-model'],
['recent-model'],
);
});

test('a later fold rolls over the reply the retreat left verbatim', async () => {
// The retreat keeps the newest reply out of the fold, so it stays in the
// request as raw text. It does not stay there: the next fold covers it,
// rolling the checkpoint forward, because by then a newer reply ends the
// proven span. This bounds how long a retreat's leftover survives.
const first = [
user('old-user', 'old-turn'),
model('old-model', 'old-turn', 'old result'),
user('recent-user', 'recent-turn'),
model('recent-model', 'recent-turn', 'recent result'),
];
let attempts = 0;
const retreated = await planHistoryCompaction(
planInput({
phase: 'standalone',

@Astro-Han Astro-Han Sep 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 These cases pair phase: 'standalone' with a hand-supplied acceptedRoute, and no production caller produces that combination: the standalone site passes neither field, and the site that passes them is mid-turn or pre-turn. So they prove the boundary arithmetic but not one production obligation, and they would stay green if both call sites were deleted. That is what let the wiring gap through.

One case driving AiSdkCompaction.compactHistory rather than planHistoryCompaction directly, asserting a second summarizer call after one input_too_large, closes it. The overflow-reactive-recovery.test.ts assertion is the only production-wired one today, and it covers the one phase where the wiring happens to work.

orderedEvents: first,
runHeaders: HEADERS_A,
acceptedRoute: ROUTE_A,
reserveTailEvents: 0,
summarize: () => {
attempts += 1;
if (attempts === 1) throw new HistoryCompactSummarizerError('input_too_large');
return structuredSummary('A bounded automatic summary.');
},
}),
);
assert.equal(retreated.decision, 'compacted');
if (retreated.decision !== 'compacted') return;
assert.deepEqual(
retreated.tailRuntimeEvents.map((event) => event.id),
['recent-model'],
);

// The turn continues: a newer reply arrives, so the proven span now ends
// after the one the retreat spared.
const later = [...first, user('next-user', 'next-turn'), model('next-model', 'next-turn')];
const rolled = await planHistoryCompaction(
planInput({
phase: 'standalone',
orderedEvents: later,
runHeaders: HEADERS_A,
acceptedRoute: ROUTE_A,
reserveTailEvents: 0,
previousCheckpoint: retreated.checkpoint,
summarize: () => structuredSummary('A bounded automatic summary.'),
}),
);
assert.equal(rolled.decision, 'compacted');
if (rolled.decision !== 'compacted') return;
assert.equal(
rolled.coveredRuntimeEvents.some((event) => event.id === 'recent-model'),
true,
);
});

test('fails open when the proven boundary is refused too', async () => {
// One retreat, because there is one proven boundary. A rejection of that
// span is the provider saying this fold cannot be made.
const events = [
user('old-user', 'old-turn'),
model('old-model', 'old-turn', 'old result'),
user('recent-user', 'recent-turn'),
model('recent-model', 'recent-turn', 'recent result'),
];
let attempts = 0;
const result = await planHistoryCompaction(
planInput({
phase: 'standalone',
orderedEvents: events,
runHeaders: HEADERS_A,
acceptedRoute: ROUTE_A,
reserveTailEvents: 0,
summarize: () => {
attempts += 1;
throw new HistoryCompactSummarizerError('input_too_large');
},
}),
);

assert.equal(attempts, 2);
assert.equal(result.decision, 'fail_open');
if (result.decision !== 'fail_open') return;
assert.equal(result.diagnosticReason, 'input_too_large');
});

test("a mixed-route session retreats to this route's own newest reply", async () => {
// History can span runs on several routes. A span another model accepted
// proves nothing about this summarizer's window, so the retreat targets the
// newest reply THIS route produced, found through the run headers.
const events = [
user('old-user', 'old-turn'),
modelOnRun('mine', 'old-turn', 'run-1', 'accepted by this route'),
user('switch-user', 'switch-turn'),
modelOnRun('theirs', 'switch-turn', 'run-2', 'accepted by another route'),
];
const attemptedCoverage: string[][] = [];
const result = await planHistoryCompaction(
planInput({
phase: 'standalone',
orderedEvents: events,
runHeaders: [
runHeader('run-1', 'model-a', 'conn-a'),
runHeader('run-2', 'model-b', 'conn-b'),
],
acceptedRoute: ROUTE_A,
reserveTailEvents: 0,
summarize: ({ coveredRuntimeEvents }) => {
attemptedCoverage.push(coveredRuntimeEvents.map((event) => event.id));
if (attemptedCoverage.length === 1) {
throw new HistoryCompactSummarizerError('input_too_large');
}
return structuredSummary('A bounded automatic summary.');
},
}),
);

assert.equal(result.decision, 'compacted');
// Not ['old-user', 'mine', 'switch-user'], which stops at the other route's
// reply: that span is proven only for the model that accepted it.
assert.deepEqual(attemptedCoverage, [
['old-user', 'mine', 'switch-user', 'theirs'],
['old-user'],
]);
});

test('fails open after repeated input-too-large retreat reaches no safe span', async () => {
test('fails open when only another route has ever been accepted', async () => {
let attempts = 0;
const result = await planHistoryCompaction(
planInput({
phase: 'standalone',
orderedEvents: [user('u1', 't1'), modelOnRun('theirs', 't1', 'run-2')],
runHeaders: [runHeader('run-2', 'model-b', 'conn-b')],
acceptedRoute: ROUTE_A,
reserveTailEvents: 0,
summarize: () => {
attempts += 1;
throw new HistoryCompactSummarizerError('input_too_large');
},
}),
);
assert.deepEqual(result, { decision: 'fail_open', reason: 'no_safe_completed_span' });

assert.equal(attempts, 1);
assert.equal(result.decision, 'fail_open');
});

test('fails open without retrying when no accepted reply proves a boundary', async () => {
// No model reply anywhere, so no request has ever been accepted on this
// ledger; inventing a boundary would be the guess this change removes.
let attempts = 0;
const result = await planHistoryCompaction(
planInput({
phase: 'standalone',
orderedEvents: [user('u1', 't1'), user('u2', 't1'), user('u3', 't2')],
runHeaders: HEADERS_A,
acceptedRoute: ROUTE_A,
reserveTailEvents: 0,
summarize: () => {
attempts += 1;
throw new HistoryCompactSummarizerError('input_too_large');
},
}),
);

assert.equal(attempts, 1);
assert.equal(result.decision, 'fail_open');
});

test('fails open on an input-too-large rejection with the summarizer reason', async () => {
// The retreat is bounded by what a provider has already accepted, so a
// rejection that outlives it fails open carrying the summarizer's own
// reason rather than a span-selection one.
const result = await planHistoryCompaction(
planInput({
summarize: () => {
throw new HistoryCompactSummarizerError('input_too_large');
},
}),
);
assert.equal(result.decision, 'fail_open');
if (result.decision !== 'fail_open') return;
assert.equal(result.diagnosticReason, 'input_too_large');
});

test('persisted checkpoint replay-validates against the same ledger prefix (recovery)', async () => {
Expand Down Expand Up @@ -379,6 +556,29 @@ function user(id: string, turnId: string): RuntimeEvent {
function model(id: string, turnId: string, text: string = id): RuntimeEvent {
return { ...base(id, turnId), role: 'model', author: 'agent', content: { kind: 'text', text } };
}
/** A model reply produced by a named run, so a route can be attached to it. */
function modelOnRun(id: string, turnId: string, runId: string, text: string = id): RuntimeEvent {
return { ...model(id, turnId, text), runId, invocationId: runId };
}
function runHeader(runId: string, modelId: string, llmConnectionId: string): AgentRunHeader {
return {
runId,
sessionId: 'session-1',
turnId: 'turn-1',
status: 'completed',
backendKind: 'ai-sdk',
llmConnectionId,
llmConnectionSlug: llmConnectionId,
modelId,
cwd: '/tmp/maka',
permissionMode: 'ask',
createdAt: 1_800_000_000_000,
updatedAt: 1_800_000_000_000,
};
}
const ROUTE_A = { modelId: 'model-a', connectionId: 'conn-a' };
const HEADERS_A = [runHeader('run-1', 'model-a', 'conn-a')];

function call(id: string, callId: string, turnId: string): RuntimeEvent {
return {
...base(id, turnId),
Expand Down
12 changes: 8 additions & 4 deletions packages/runtime/src/__tests__/overflow-reactive-recovery.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1175,23 +1175,27 @@ describe('reactive overflow recovery in the streaming backend', () => {
});

test('step-0 overflow recovery gates reasoning on retry and durable reload', async () => {
// The subject here is reasoning gating across the retry and the durable
// reload, not how many times a fold may retreat: the retreat is bounded by
// the one span a provider has already accepted, so the summarizer answers
// on its first call.
let summarizeCalls = 0;
const fixture = buildReactiveFixture({
script: ['overflow', 'tool', 'done'],
bigPriors: true,
reasoningReplayTail: true,
summarize: (input) => {
summarizeCalls += 1;
if (summarizeCalls <= 2) {
throw new HistoryCompactSummarizerError('input_too_large');
}
if (summarizeCalls === 1) throw new HistoryCompactSummarizerError('input_too_large');
return reactiveStructuredSummary(input.source.foldedRuntimeEvents);
},
});
await runTurn(fixture);

assert.equal(fixture.model.doStreamCalls.length, 3);
assert.equal(fixture.summarizerCalls(), 3);
// One retreat, to the span the last accepted input covered, which leaves
// the reasoning tail verbatim.
assert.equal(fixture.summarizerCalls(), 2);
for (const call of fixture.model.doStreamCalls.slice(1)) {
const prompt = JSON.stringify(call.prompt);
assert.match(prompt, /REACTIVE_SUMMARY_SENTINEL/);
Expand Down
5 changes: 5 additions & 0 deletions packages/runtime/src/ai-sdk-compaction.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1069,6 +1069,11 @@ export class AiSdkCompaction {
phase: input.phase ?? 'mid_turn',
orderedEvents,
headAnchor: { runtimeEventId: state.headAnchor.id, turnId },
runHeaders: state.priorRunHeaders,
acceptedRoute: {

@Astro-Han Astro-Han Sep 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 These two lines are the whole wiring, and they only reach the mid-turn and pre-turn call site. The sibling call at :365 (phase: 'standalone') passes neither, so acceptedInputBoundary returns undefined on its first line and the first input_too_large fails open there, where halving previously walked down until a span was accepted (that path's coverage gate is just coveredCount > 0, so every halving step passed it).

Four entries reach it: CLI /compact, Desktop sessions:compact, supervisor-wake sub-agent compaction (agent-graph-supervisor-wake.ts:111), and the pre-turn fallback at ai-sdk-backend.ts:3426. The first three end failed with a context_compaction_failed_open note; the fourth sends the oversized history and the turn dies with context_overflow. No flag or fallback. And the standalone first attempt covers the entire prior session (reserveTailEvents: 0), the span most likely to be rejected, so it is the ordinary long-session case.

Both values are in hand at :365, where input.runtimeContextRunHeaders is already used ten lines below:

...(input.runtimeContextRunHeaders ? { runHeaders: input.runtimeContextRunHeaders } : {}),
acceptedRoute: {
  modelId: this.input.modelId,
  ...(this.targetConnectionId !== undefined ? { connectionId: this.targetConnectionId } : {}),
},

P3, separately: state.priorRunHeaders excludes the current turn by construction (prior-run-context.ts:66-71 filters run.turnId !== currentTurnId), so no current-turn reply can be on route and the proven index always lands at or below headAnchorIndex, while the mid_turn gate wants strictly above it. So mid-turn cannot reach a retreat either. That gate is unchanged from before and halving usually undershot it too, so this is a pre-existing limit rather than something this PR broke. A follow-up, not a fix under time pressure. If you do pursue it, the current run is the route by construction, so a synthetic header for input.origin.runId or a currentRunId field treated as on-route would make the comment at history-compaction.ts:239-246 true.

modelId: this.input.modelId,
...(this.targetConnectionId !== undefined ? { connectionId: this.targetConnectionId } : {}),
},
reserveTailEvents: 1,
charsPerToken,
now: this.now(),
Expand Down
Loading