Repository navigation
Does chat history reduction preserve the goal verbatim? Handoffs are lossy compactions #14484
behrnt-slatgng
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Disclosure up front: I build Grunz, a coding agent on open-weight models. Single-agent, not multi-agent, so treat this as a view from outside the paradigm. Nothing to sell — I would genuinely like to know whether this is solved here and I have missed how.
Long runs fail at compaction boundaries, not at function calls
When the context window fills, something decides what to discard. If the goal is not preserved verbatim through that step, it gets summarised into vagueness. The model reads back its own compacted notes, no longer knows precisely what it was doing, and quietly restarts the plan — frequently re-deriving completed work.
From outside it reads as "it forgot my instructions." They were not buried in the middle of a long context. They were compressed out. Nothing errors, so it never surfaces as a failure — it surfaces as a run that took three times as long and landed adjacent to what was asked.
Why I would expect agent orchestration to make this harder, not easier
The intuitive pitch is that decomposition solves context pressure: each agent holds less, so nobody fills their window. My experience is that it relocates the problem.
Every handoff is a lossy compaction that also crosses a boundary. When agent A summarises state for agent B, that is the same operation as a compaction — something decides what survives — except B has no access to what A dropped and no way to know it is missing. A compaction at least leaves the raw history recoverable in principle. A handoff generally does not.
The boundaries do not line up. An agent's context can fill mid-subtask, so it compacts, loses fidelity on its own assignment, then hands off a summary of a plan it had already partly forgotten. The orchestrator sees a completed handoff and a plausible summary, so nothing looks wrong at the supervision layer.
Net effect: the group can be confidently, collectively wrong in a way a single agent with a full trace usually is not, because every participant's view is locally coherent.
Three questions for people running SK in anger
Does chat history reduction preserve the original goal and acceptance criteria verbatim, or is that something you implement per-workflow? Pinning the goal verbatim through every reduction was the single highest-leverage change I made, and I would expect it to matter more in an orchestrated setting, not less.
Is the reduction step observable? Teams log function calls and messages, but I rarely see anyone logging what got dropped at a summarisation step — which is usually where the run silently went wrong. Without it you can explain every individual action and still not explain the run.
Is there a pattern for a handoff that carries a pointer to full prior context rather than only a summary, so a receiving agent can pull back detail it discovers it needs?
Entirely possible the answer to all three is "yes, here is the doc," in which case I would be glad to read it. I would rather find out I am wrong about this than keep telling people orchestration makes it worse.
All reactions