Stream Bash/edit eval requests through long reasoning turns

[samsja]

Long Bash/edit reasoning turns can exceed the bundled client's read timeout while a healthy model is still generating. The shared Bash/null chat helper now always requests streaming, so model deltas and interception-server SSE keepalives reach the sandbox throughout the turn. There is no stream CLI flag, harness setting, or helper/compactor parameter; the shared launcher and compaction plumbing remain unchanged.

The SDK assembles complete responses before tools execute, preserving fragmented arguments, reasoning, usage, and finish_reason="length". Assistant role metadata is normalized because some providers repeat it in each delta. Empty and unfinished streams fail before tool execution.

Training clients continue generating complete responses through the normal graph-commit path, retaining token IDs and log probabilities. The interception server frames the completed training response as SSE for the harness; this does not add token-by-token generation to training. The chat dialect includes tool-call indexes when framing complete responses. Browser-use retains its existing launch arguments.

Validation:

• Repository checks: 80 tests passed, 76 credential-dependent tests skipped; all-file Ruff and pre-commit passed.
• Three live CI-model tests passed: colocated tools, repeated tool use, and Bash execution.
• Temporary HTTP regressions cover repeated role metadata, keepalives, interleaved Bash/edit arguments, reasoning/usage, length termination, and empty/truncated streams.
• A real interception-server test with a stubbed training generator verifies two tool calls and a follow-up, committed token IDs/log probabilities, usage, and length termination. It passes against both the PR checkout and the installed runtime backport.
• The previous revision's full GitHub CI passed. CI is rerunning for this revision; Docker/browser live coverage runs there because Docker is unavailable on this host.

This addresses idle request deadlines; it does not establish that all observed gateway 504s have the same cause.

---

Note

Medium Risk
Eval completion transport changes for bash/edit runs; truncated streams now raise RuntimeError instead of proceeding with partial output, though training paths remain non-streaming.

Overview
Bash/edit eval harnesses can now use streaming chat completions so long model turns stay alive past idle read timeouts (e.g. 600s) via SSE keepalives, while the loop still waits for a full completion before running tools.

chat() gains an optional stream flag: when enabled it requests stream_options.include_usage, accumulates chunks with AsyncChatCompletionStream (no auto tool-arg parsing), normalizes assistant role, and fails if the stream ends without a finish_reason. Compactor forwards the same flag on initial, retry, and checkpoint-summary calls. The bundled program exposes --stream; launch_chat_program adds --stream only for the shared null/bash program when the model context uses EvalClientConfig (training/non-eval clients keep non-streaming).

^{Reviewed by Cursor Bugbot for commit c40ca9b. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add streaming mode for bash/edit eval completion requests in harness

• Adds a stream keyword option to Compactor that is forwarded to initial, retry, and checkpoint-summary completion requests in compaction.py
• Extends the chat helper in core.py with a streaming path that requests usage data, collects the async stream, and returns the accumulated snapshot; raises RuntimeError when the result has no choices or any choice lacks a finish reason
• Adds a --stream CLI flag to parse_args and wires it into the standalone Compactor setup in main
• Appends --stream to generated program arguments in launch.py when the model context uses EvalClientConfig; other client configurations are unaffected
• Risk: the new streaming path in core.chat raises RuntimeError on empty or unfinished choices — callers that previously tolerated such responses will now see failures

📊 Macroscope summarized 8627469. 3 files reviewed, 2 issues evaluated, 0 issues filtered, 2 comments posted

🗂️ Filtered Issues

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 8627469. Configure here.}

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — The PR is a focused but cross-layer runtime transport change: shared chat calls switch to streamed completions, and training responses gain synthetic SSE framing while preserving full-response tool execution. Because existing request behavior changes across the harness and interception server, human review is appropriate.

^{You can add or adjust custom eligibility rules. Learn more.}