Skip to content

Typed operations, engines, workspace, query & MCP (#689)#690

Open
tony wants to merge 170 commits into
masterfrom
engine-ops
Open

Typed operations, engines, workspace, query & MCP (#689)#690
tony wants to merge 170 commits into
masterfrom
engine-ops

Conversation

@tony

@tony tony commented Jun 21, 2026

Copy link
Copy Markdown
Member

Summary

Implements the typed operations + engines architecture under libtmux.experimental — an inert, statically-typed operation spine; a family of interchangeable engines (subprocess, concrete, control-mode, their async variants, and the native imsg easter-egg); lazy/async plans with ;-folding chainability; pure object-graph snapshots; a typed read surface; engine-typed facades; a declarative tmuxp-style workspace builder; a live pane query DSL; tmux 3.7 floating panes; an optional Model Context Protocol server; and a docs catalog generated from the registry.

Operationalizes #688 (architecture) per the plan in #689. Touches no existing public API — everything is additive under libtmux.experimental (explicitly outside the versioning policy). Nothing is generated at runtime; everything is statically typed and checker-clean.

What's delivered

The spine — libtmux.experimental.ops (pure, no tmux):

  • Operation[ResultT]: frozen, keyword-only, class-vars as the single source of truth (kind/command/scope/result_cls/effects/safety/chainable/version gates). Pure render() with declarative version gating; build_result() adapts raw output to a typed result (version-threaded so read parsing matches the gated render).
  • Typed Result hierarchy with opt-in raise_for_status(): AckResult, SplitWindowResult/CreateResult (captured ids), CapturePaneResult, ListPanes/Windows/Sessions/ClientsResult (snapshot-deriving rows), plus HasSessionResult, DisplayMessageResult, ShowOptionsResult, ShowBufferResult.
  • Closed Target sum, fail-closed OperationRegistry, stdlib serialization, and catalog() (registry-derived docs data).
  • LazyPlan (record → resolve SlotRef forward refs → execute). Folding is planner-based: pass a Planner — sequential (one tmux call per op), FoldingPlanner (;-chained tmux a ; b), or MarkedPlanner ({marked}-fold). All yield an identical PlanResult; only the dispatch count differs, and failure attribution matches tmux's cmdq_remove_group (first failed, rest skipped).
  • Read seam: ListPanes/ListWindows/ListSessions/ListClients render the same -F template neo uses (imported, not copied) and parse into models snapshots — a typed read surface parallel to neo, leaving the ORM untouched.
  • 58 operations across client/pane/window/session/server scopes.

Engines — libtmux.experimental.engines (all behind TmuxEngine/AsyncTmuxEngine, all returning the same CommandResult):

Family Sync Async
Subprocess (classic) SubprocessEngine AsyncSubprocessEngine
Concrete (in-memory) ConcreteEngine AsyncConcreteEngine
Control mode (tmux -C) ControlModeEngine AsyncControlModeEngine (event stream via subscribe())
Native imsg (binary protocol) ImsgEngine (opt-in easter egg)

Control engines use an I/O-free bytes ControlModeParser with FIFO/skip correlation (startup-ACK consumed up front; unsolicited hook blocks skipped), report their tmux version for runtime gating, reap the throwaway session a bare tmux -C implies, and end event subscribers cleanly on engine death. The imsg engine speaks tmux's binary peer protocol directly (AF_UNIX + SCM_RIGHTS, PROTOCOL_VERSION 8) with a live parity test vs the subprocess engine.

Models — libtmux.experimental.models: frozen Pane/Window/Session/Client/ServerSnapshot (typed core + raw field tail), from_pane_rows() builds the whole tree from one list-panes -a query, round-trips to plain dicts.

Facades — libtmux.experimental.facade ("mode lives in the type"): the full eager / lazy / async × Server/Session/Window/Pane/Client matrix over the same ops; control mode is just an engine choice.

Declarative workspace — libtmux.experimental.workspace: a tmuxp-style Workspace declares a session as a tree of windows and panes and lowers to a Core LazyPlan. build/abuild fold the dispatches by default (a multi-pane window collapses to a handful of ;-chained + {marked} calls, identical result to an unfolded build); host steps (per-command sleeps, the opt-in wait_pane anti-race) stay hard fold boundaries. Threads env/shell/options through the IR, round-trips via to_dict, emits a build-event stream, supports floating panes (including cross-window overlays wired through a graphlib symbol table), and ships a load CLI for .tmuxp.yaml.

Live pane query — libtmux.experimental.query: panes() opens a lazy, chainable query over the panes a running server has (filter/order_by/limit/map, including filter(floating=True)), reading nothing until a terminal call. commands() records per-pane ops (send keys, resize, select, respawn, clear history, kill) into a LazyPlan that folds to one tmux dispatch. Resolves against a live engine or a plain sequence of snapshots (offline in tests).

Floating panes (tmux 3.7): available from the operation layer (a new-pane op with absolute geometry and zoom), the pane facades (new_pane()), the MCP surface (a curated tool), and workspace specs (Workspace float declarations).

MCP server — libtmux.experimental.mcp (optional libtmux[mcp] extra): a framework-agnostic tool projection (no hard MCP/pydantic dependency) plus a thin FastMCP adapter, launched as libtmux-engine-mcp. Three tiers: a curated vocabulary of intuitive verbs (~40) that mirror the ORM, a per-operation tool for every operation (hidden by default), and plan tools that preview or build a whole workspace. It is caller-aware (discovers the launching pane and refuses to kill or respawn its own pane/window/session), gates mutating and destructive tools behind a tiered LIBTMUX_SAFETY level until opted in, adds middleware (audit logging, read-only retry, tail-preserving output limits), serves a needle-free wait_for_output monitor and tmux:// resources, and ships recipe prompts.

Docs: an in-repo tmuxop-catalog Sphinx directive renders catalog() into the operation reference (exercised by the docs gate), so the reference can't drift from the code.

Testing

  • A large experimental suite (~2,260 repo tests) + doctests. The pure spine/models/concrete/query tests need no tmux; the classic/control/async/imsg engines, the facades, workspace builds, and the MCP surface are validated against a real tmux server via the libtmux fixtures.
  • Cross-engine contract suite: same typed result across engines; serialization round-trips.
  • Full repo gate green: ruff, ruff format, ty, pytest, build-docs.

Design notes

  • Revises Design typed operations and engines #688: execution mode lives in the facade type, not a runtime-bound engine attribute (return types differ by mode).
  • Per-engine error policy: classic reproduces today's behavior; newer engines return typed results with opt-in raise_for_status(). Same result shape across engines.
  • Core is stdlib-dataclass-only; the MCP surface sits behind the optional mcp extra, with no MCP/pydantic dependency in the core projection.
  • imsg is opt-in and non-default: it depends on tmux's internal protocol (v8), is POSIX-only, and cannot host attach (which falls back to a local spawn).

Refs #688, #689.

@codecov

codecov Bot commented Jun 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.22611% with 1260 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.87%. Comparing base (d8b7127) to head (ee6903e).

Files with missing lines Patch % Lines
scripts/mcp_swap.py 26.72% 314 Missing and 15 partials ⚠️
src/libtmux/experimental/engines/imsg/base.py 51.60% 163 Missing and 33 partials ⚠️
src/libtmux/experimental/engines/control_mode.py 67.95% 77 Missing and 31 partials ⚠️
...libtmux/experimental/engines/async_control_mode.py 78.62% 60 Missing and 21 partials ⚠️
...rc/libtmux/experimental/mcp/vocabulary/_resolve.py 60.95% 46 Missing and 11 partials ⚠️
src/libtmux/experimental/engines/imsg/v8.py 76.17% 39 Missing and 17 partials ⚠️
src/libtmux/experimental/mcp/middleware.py 77.15% 32 Missing and 13 partials ⚠️
docs/_ext/tmuxop.py 18.18% 36 Missing ⚠️
src/libtmux/experimental/mcp/vocabulary/pane.py 76.15% 32 Missing and 4 partials ⚠️
src/libtmux/experimental/mcp/__init__.py 47.61% 28 Missing and 5 partials ⚠️
... and 49 more
Additional details and impacted files
@@             Coverage Diff             @@
##           master     #690       +/-   ##
===========================================
+ Coverage   52.44%   75.87%   +23.42%     
===========================================
  Files          26      229      +203     
  Lines        3726    13726    +10000     
  Branches      747     1769     +1022     
===========================================
+ Hits         1954    10414     +8460     
- Misses       1468     2654     +1186     
- Partials      304      658      +354     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tony tony changed the title Typed operations and engines: inert op spine (#689) Typed operations and engines: spine + 4 engines + facades (#689) Jun 21, 2026
@tony tony changed the title Typed operations and engines: spine + 4 engines + facades (#689) Typed operations and engines Jun 21, 2026
@tony tony changed the title Typed operations and engines Typed operations & engines: spine, 6 engines, plans, models, facades (#689) Jun 21, 2026
tony added a commit that referenced this pull request Jun 21, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
@tony

tony commented Jun 21, 2026

Copy link
Copy Markdown
Member Author

Code review

Found 2 issues:

  1. LazyPlan resolves a forward SlotRef only for target, never for src_target, so a dual-target op (JoinPane, SwapPane, MovePane, BreakPane, SwapWindow, MoveWindow, LinkWindow) whose src_target comes from an earlier plan.add(...) reaches render() with the slot unresolved and raises TypeError: cannot render an unresolved SlotRef. (bug: _resolve() substitutes operation.target but not operation.src_target, even though serialize.py already handles both)

def _resolve(
operation: Operation[t.Any],
bindings: dict[int, str],
) -> Operation[t.Any]:
"""Substitute a :class:`SlotRef` target with a captured concrete id."""
target = operation.target
if not isinstance(target, SlotRef):
return operation
try:
concrete = bindings[target.slot] + target.suffix
except KeyError as error:
msg = (
f"slot {target.slot} has no captured id yet; a plan step can only "
f"target an earlier step that creates an object"
)
raise OperationError(msg) from error
return dataclasses.replace(operation, target=_target_from_id(concrete))

  1. SaveBuffer declares contradictory metadata: safety = "mutating" together with effects = Effects(read_only=True), where read_only is documented as "does not change tmux state". Its read peer ShowBuffer uses safety = "readonly", and a consumer filtering registry.select(lambda s: s.safety == "readonly") would omit save_buffer despite effects.read_only=True. (bug: inconsistent safety/effects declarations)

result_cls = AckResult
safety = "mutating"
effects = Effects(read_only=True)

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@tony

tony commented Jun 21, 2026

Copy link
Copy Markdown
Member Author

Code review

Found 1 issue:

  1. LazyPlan resolves forward SlotRefs on every dispatch path except the {marked} fold's decorates. _drive resolves the create op but builds decorates raw, so a chainable dual-target decorate (SwapPane/JoinPane/MovePane) whose src_target is a forward slot reaches render_marked unresolved and raises TypeError: cannot render an unresolved SlotRef. The single-op and chain paths both call _resolve; this one does not. (bug: decorates = [self._operations[i] for i in decorate_idx] skips _resolve, so src_target SlotRefs survive into render under MarkedPlanner)

create_idx, *decorate_idx = step.indices
create = _resolve(self._operations[create_idx], bindings)
decorates = [self._operations[i] for i in decorate_idx]
merged: CommandResult = yield _Chain(
render_marked(create, decorates, version),
)

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@tony

tony commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

🤖 Generated with Claude Code

1 similar comment
@tony

tony commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

🤖 Generated with Claude Code

tony added a commit that referenced this pull request Jun 27, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
tony added a commit that referenced this pull request Jun 28, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
tony added a commit that referenced this pull request Jul 4, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
tony added a commit that referenced this pull request Jul 4, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
@tony tony changed the title Typed operations & engines: spine, 6 engines, plans, models, facades (#689) Typed operations, engines, workspace, query & MCP (#689) Jul 4, 2026
@tony

tony commented Jul 5, 2026

Copy link
Copy Markdown
Member Author

Code review

No issues found. Fresh review of the current HEAD (4393b02a) — the ops / engines / facade / workspace / query / MCP surface was checked for bugs and AGENTS.md compliance and is clean. (The prior review comments predate ~12 days of commits.)

🤖 Generated with Claude Code

tony added a commit that referenced this pull request Jul 5, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
tony added a commit that referenced this pull request Jul 11, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
tony added a commit that referenced this pull request Jul 12, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
tony added 17 commits July 18, 2026 05:25
why: AGENTS.md shipped-vs-branch keeps never-shipped, branch-internal
narrative out of docstrings; round 1 missed a few.

what:
- control_mode.py: drop the "chainable-commands runner /
  libtmux-protocol-engines parser" lineage sentence
- concrete.py: drop "preserving the historical behaviour" (an
  intermediate in-branch state)
- new_pane.py: drop the "first operation gated by min_version" ordinal;
  keep only the current-state VersionUnsupported clause
why: register_plan_tools' docstring said build_workspace is registered
only on the synchronous server, but the async branch registers it too
(dispatching to abuild_workspace) -- misleading a maintainer about tool
exposure.

what:
- State it registers on both servers, per-engine dispatch
why: The session start_directory lands on window 0's first pane, but
confirm() read the window's active_pane. When a non-first pane is
focused (or a split leaves focus off pane 0) with a different cwd,
confirm falsely reported "first pane cwd != declared".

what:
- Check the first window's first pane (index 0), not active_pane
- Live regression: a focused non-first pane at a different cwd still
  confirms ok
why: A tmux restart or socket blip killed the async control-mode
engine permanently -- the reader EOF'd, the engine went dead, and
subscribers had to be torn down. A supervisor that self-heals keeps
the event stream alive across the gap and lets subscribe() rely on
_closing alone instead of a sticky _dead gate.

what:
- Replace the one-shot start()/reader with a supervisor that spawns
  tmux -C, replays desired subscriptions/attach, runs the reader
  inline (one at a time), and reconnects with deterministic jittered
  backoff when the reader returns on EOF.
- Add add_subscription()/set_attach_targets() declarative state,
  replayed on every (re)connect; bump _generation per connect and
  reset parser/pending/attach before the new proc's bytes flow.
- Surface first-connect failure to every start() caller; reset the
  backoff on a healthy connect.
- Gate subscribe() on _closing only: a reconnecting (_dead) engine
  keeps its subscriber so the post-reconnect reader feeds it.
- Cover supervisor reconnect, attach replay, and attach reset;
  drop the now-obsolete _dead-gate sentinel test.
why: The engine's supervisor now reconnects on a tmux restart or
socket blip, but _broadcast_stream_end ends the subscribe stream on
the disconnect, so the pull ring's drain task completes. A bare
``is None`` guard never re-subscribed, silently freezing the cursor
after a reconnect.

what:
- Restart the drain in _EventRing._ensure_started when the prior task
  has completed (not just when unset).
- Clear a prior drain error at the start of _drain (a fresh attempt)
  rather than on restart, so a still-unread failure is not wiped
  before since() surfaces it -- a persistently dead stream must not
  read as empty-but-healthy.
- Cover restart-only-if-done and persistent-error surfacing with
  parametrized tests.
why: "Facade" is not idiomatic Python for these engine-typed classes.
They are the domain-shaped tmux nouns -- server/session/window/pane/
client -- so the package reads as libtmux.experimental.objects, and
an individual engine-bound class is a "wrapper" in prose. Retires the
facade/handle vocabulary.

what:
- Rename src/libtmux/experimental/facade -> objects and its tests,
  keeping the Eager/Lazy/Async class names unchanged.
- Reword package docstrings from "facade"/"handle" to "object".
- Update the three stray package references (ops.results,
  ops.new_pane, docs/experimental) to "wrapper".
why: A proc that connects then instantly dies still consumes its
startup ACK, so the supervisor counted it as a healthy connection and
reset the reconnect backoff to zero every loop. A persistently
flapping proc (fatal config, permission, or resource error) then
fork-stormed tmux at ~10 Hz instead of backing off.

what:
- Reset the backoff only for a connection that survived a minimum
  lifetime; a connect-then-immediately-die counts as a failed attempt,
  so the backoff escalates.
- Add _HEALTHY_CONNECTION_SECONDS and a regression test asserting a
  connect-then-die loop escalates instead of pinning at _backoff(0).
why: A reader that returned via an exception (not a clean EOF) leaves
the tmux -C process alive. The supervisor then reconnected and _spawn
overwrote _proc with the new process without terminating the old one,
orphaning a control client on every such reconnect.

what:
- Terminate a still-alive prior proc at the top of _spawn before
  replacing it; a clean-EOF proc has already exited (no-op).
- Add a regression test that _spawn terminates a live prior proc.
why: Quantify the experimental workspace builder's build cost across
engines and answer "which engine, how fast" reproducibly, without ever
touching the developer's live tmux server.

what:
- Add scripts/bench_engines.py, a self-contained PEP 723 script (uv
  run) that sweeps scenarios x engines x wait-modes and reports
  min/avg/median/p90/p95/p99/max as rich tables + JSON.
- Engines: classic; the builder on subprocess/control_mode/imsg/
  concrete; and a pipelined prototype that batches independent creates
  via run_batch.
- Sandboxed: per-run isolated sockets under a throwaway dir, TMUX
  unset, atexit teardown + orphan backstop -- the default server is
  never contacted.
- Subcommands: run (in-process grid), cell (one build, for hyperfine),
  profile (cProfile).
why: Version the measured build cost across engines (and the pipelining
prototype) alongside the harness so the numbers are reproducible and
reviewable.

what:
- Add scripts/bench-results/ with RESULTS.md (analysis: the engine
  grid, with/without shell-readiness wait, profile, and the ~79x
  reconciliation) plus grid.json / wait.json (raw per-run data from
  scripts/bench_engines.py).
why: The committed benchmark script failed a repo-wide `mypy .` sweep
with 5 errors. Four stem from the `typer` PEP 723 inline dependency,
which the repo's mypy environment cannot resolve (cascading into
untyped-decorator errors); one is a genuine Server|None narrowing gap.
The configured scope (files = [src, tests]) hides them, but a broad
`mypy .` surfaces them.

what:
- Add a file-level disable-error-code for the two typer environment
  artifacts (import-not-found, untyped-decorator), keeping every other
  check strict.
- Accept `Server | None` in ImsgForServer and assert non-None so the
  make_engine callable type narrows correctly.
why: "Concrete" named the in-memory simulator engine, but every real
engine (subprocess, control_mode, imsg) is equally a concrete
implementation. Across the Python/Rust ecosystem "Concrete*" is a
test-only convention for "a minimal instantiable ABC subclass", the
opposite of this docs/doctest workhorse, and the word already carries
its id/type sense throughout ops/query/objects. "Mock" names the engine
by its role: the no-tmux, in-memory stand-in.

what:
- Rename ConcreteEngine -> MockEngine and AsyncConcreteEngine ->
  AsyncMockEngine; module concrete.py -> mock.py
- EngineKind.CONCRETE ("concrete") -> EngineKind.MOCK ("mock");
  EngineSpec.concrete() -> EngineSpec.mock(); registry key becomes
  "mock" (available_engines() re-sorts accordingly)
- Rename the benchmark engine key/label; update RESULTS.md and
  grid.json to match
- Keep docstrings describing the in-memory simulation so the name's
  test-double flavor does not mislead doctest readers
- Leave "concrete" untouched where it means a concrete id/target/pane
  handle (ops, query, objects, workspace)
why: Every engine re-derived the same two things before it could
dispatch: which tmux binary to exec, and which tmux server to point
at. Five copies of the -L/-S/-f/-2/-8 flag construction, three copies
of the memoized shutil.which + `tmux -V` probe, and one drifted copy
in control_mode that called shutil.which unguarded and unmemoized on
every connect.

what:
- Add engines/connection.py with a frozen ServerConnection value
  object owning the connection flags, memoized binary resolution,
  memoized version probe, and argv assembly
- ServerConnection.from_server() is the single mirror of
  Server.cmd()'s flag construction; every engine's for_server() is
  now built on it
- SubprocessEngine, AsyncSubprocessEngine, ControlModeEngine,
  AsyncControlModeEngine and ImsgEngine hold a connection instead of
  re-deriving one; tmux_bin/server_args stay as read-only properties
- Export ServerConnection from libtmux.experimental.engines
why: _chain re-inlined tmux's success rule (`returncode == 0 and not
stderr`) at two attribution sites, so the definitive rule in
ops/results.py could drift out from under them. Worse,
ensure_chainable -- the fail-closed guard against folding an op that
captures output or creates an object -- was dead code: no rendering
path called it, so a capturing op folded into a `;` chain would have
silently mis-attributed its stdout.

what:
- Import status_for from ops/results.py in attribute() and
  attribute_marked() instead of re-inlining the success rule
- Run ensure_chainable over every op in render_chain()
- Run ensure_chainable over render_marked()'s decorates (the create is
  the one non-chainable op the fold tolerates: its captured id is read
  back from stdout[0])
- Document the guard on the module and on both rendering paths, with
  doctests that show the fold failing closed
why: Six helpers had been re-implemented rather than imported, so a
fix to one copy silently left the others behind: the docstring
first-line summary (3 clones), the optional-target resolver, the
workspace on_exists preflight, the caller's socket-path
reconstruction, the plan-outcome projection, and the pane-readiness
poll (2 clones, one of which had already drifted its poll budget).

what:
- Add ops/_docstring.py first_line(); catalog.py, mcp/registry.py and
  mcp/fastmcp_adapter.py import it (the adapter keeps its `| None`
  contract at the call site)
- Add experimental/_wait_pane.py holding the cursor-probe poll;
  fluent.py and workspace/runner.py import wait_pane/await_pane
- mcp/vocabulary/option.py imports _resolve.opt_target
- workspace/sets.py imports runner.py's _preflight_sync/_preflight_async
- mcp/vocabulary/_caller.py extracts socket_path_of() and
  same_socket_path() from the two copies in socket_matches /
  socket_could_match
- mcp/plan_tools.py adds PlanOutcome.to_dict() and _to_outcome(); the
  adapter's plan tools return outcome.to_dict()
why: Six pieces of surface were written and never read. The imsg
protocol's six msg_* accessors and pack_message() were declared in the
ImsgProtocolCodec Protocol and implemented in v8 with no caller, so
the Protocol overstated what a codec must supply. EngineSpec.extra was
never consulted, and the engine registry's docstring advertised a
create_engine(EngineSpec) overload that does not exist. The MCP
descriptor carried effects/version_gates/version_gate that no
projection reads, and middleware kept a tool-batch summarizer for a
shape it no longer sees.

what:
- Drop msg_exit/msg_write_open/msg_write/msg_write_close/msg_read_open/
  msg_exiting and pack_message from the ImsgProtocolCodec Protocol and
  from ProtocolV8Codec
- Drop EngineSpec.extra; correct engines/registry.py's docstring to
  describe the EngineKind it actually accepts
- Drop ToolDescriptor.effects/version_gates and
  ParamDescriptor.version_gate plus the registry lines populating them
- Drop mcp/middleware.py's _summarize_tool_batch_operation_args and the
  _summarize_nested_operation_args indirection over it
- Register the imsg engine under EngineKind.IMSG.value, not a bare string
why: Control mode does not forward pane output verbatim: tmux writes
every non-printable byte -- and the backslash itself -- as a backslash
plus three octal digits. Any reader that scans %output for raw bytes
must undo that first or it can never match. That decoding is transport
knowledge, so it belongs next to render_control_line in the engines
layer rather than in whichever consumer notices the escaping first.

what:
- Add unescape_control_output() to engines/base.py, the inbound
  counterpart to render_control_line: octal escapes decode back to the
  bytes the pane wrote, and bytes tmux left alone pass through, so an
  already-raw payload is unaffected.
- Cover it with doctests over printable text, an OSC escape sequence,
  and multi-byte UTF-8.
tony added 12 commits July 18, 2026 06:27
why: scripts/bench_engines.py has three subcommands, six engines, and
several easy-to-misread gotchas (wait-mode apples-to-oranges, hyperfine
understating the builder). Give agents a tailored reference so they run
and read it correctly.

what:
- Add .agents/skills/benchmarking-engine-builds/SKILL.md
- Cover run/cell/profile, the six engines, and how to read percentiles
- Flag the uv-run requirement and the wait-mode comparison trap
why: Claude Code discovers skills under .claude/skills only, while
.agents/skills is the agent-agnostic source of truth. One symlink gives
both tools the same skills without duplicating content. .claude/ is
globally gitignored, so this is force-added to track it per-repo.

what:
- Add .claude/skills -> ../.agents/skills symlink (force-added)
why: The swap wrote agy's server config to
~/.gemini/antigravity/mcp_config.json, but the Antigravity CLI reads
~/.gemini/config/mcp_config.json, so swapped servers never loaded.
Ports the fix already landed in libtmux-mcp/agentgrep/rampa.

what:
- CLIS["agy"].config_path -> ~/.gemini/config/mcp_config.json
- Drop the stale "may read a different profile" docstring hedge
- Note Grok/agy in the scope docstring (already supported)
why: The flat `run` grid can't isolate which choice drives engine
build cost. Researching bottlenecks needs a factorial over the four
independent axes so each one's effect is readable on its own.

what:
- Add `matrix` subcommand: 5 expression layers (imperative, plan-seq,
  plan-fold, ws-seq, ws-fold) x {subprocess, control_mode} x
  {sync, async}, with a `classic` reference row.
- Layers compose from one sans-I/O op generator (sync/async pumps)
  plus plan/workspace lowering, reusing the existing hermetic harness.
- Fold a mock-parity contract into `matrix --check`: every layer x
  mode must render identical tmux argv to the mock oracle, so the
  benchmark doubles as an ops-language contract test. `mock` is the
  oracle, never a results row.
- Add `concurrency` subcommand: K independent builds sync-serial vs
  async-`gather` over one connection, per transport.
why: CI wants the ops-language parity assertion without paying for
live timed builds, and a green parity result should be provably
non-vacuous.

what:
- Add `contract` subcommand: run the mock-parity check across shapes,
  exit non-zero on any layer x mode argv divergence.
- Guard with a negative control (non-empty, order-sensitive oracle),
  so a passing contract demonstrably catches a dropped op.
why: The module docstring, RESULTS.md, and the skill covered only the
`run` grid, so the new factorial and concurrency subcommands were
undiscoverable.

what:
- Describe the four-axis matrix, the mock-as-oracle contract, and the
  concurrency measurement in the module docstring + Run examples.
- Add matrix + concurrency sections to bench-results/RESULTS.md with
  reproduce commands and an illustrative snapshot.
- List matrix/concurrency/contract in the benchmarking skill.
why: A create that fails captures no id, so its slot never binds. The
next step then raised ForwardCaptureError, naming an unbound forward
reference while discarding the real tmux error. A vanished server
("server exited unexpectedly") surfaced hundreds of lines away as a
confusing reference problem, making the true cause invisible.

what:
- Add FailedCreateError carrying the failing argv, exit code, and
  stderr.
- Thread the step results into _resolve/_resolve_slot; when the
  referenced slot's creator ran and failed, raise FailedCreateError
  rather than ForwardCaptureError.
- Keep ForwardCaptureError for its real case: a creator that succeeded
  but captured no id.
- Cover both branches, so the discrimination is asserted, not assumed.
why: A create rendered with `-P -F` tells tmux exactly how many ids to
print, but nothing checked that they arrived. A short or empty line left
the missing ids None and logged nothing, so the loss only surfaced later
and elsewhere -- an unresolvable plan reference, or an AttributeError
wrapping a None id. The id-parse path had no logging at all.

what:
- Check the capture invariant in build_result: warn when a create did
  not complete, and error when a complete create captured fewer ids than
  its format requested (which is what catches the partial case).
- Warn in status_for when tmux exits 0 but writes to stderr, since the
  result is downgraded to failed on stderr text alone.
- Use the documented tmux_* extra keys; heavy stdout/stderr stay DEBUG.
why: The ops engines had almost no logging, so a command that returned
nothing looked identical to one that worked. Control mode was worse:
a dropped or misattributed reply block is silent, and correlation
desync could only be inferred after the fact.

what:
- Log each subprocess dispatch at DEBUG (argv, exit code, stdout,
  stderr), matching what libtmux.common already provides.
- Add BlockSequenceMonitor: tmux's guard echoes a strictly increasing
  command number, so a solicited block whose number does not advance is
  provable desync -- one integer compare per block.
- Warn on the silent control-mode paths: an unparseable %begin guard,
  a surplus solicited block, a drained solicited reply, and an async
  block arriving with no pending command.
- Heavy stdout/stderr stay DEBUG behind isEnabledFor guards.
why: tmux reports a lost server on the client's stderr, not in an
%error block, so _read_stderr logged that text and dropped it. A caller
saw only how libtmux noticed ("tmux -C closed stdout") and never what
tmux said ("server exited unexpectedly"), leaving control-mode failures
strictly less diagnosable than subprocess ones.

what:
- Keep a bounded tail of the tmux process's stderr as it is read.
- Add _died() to build connection-death errors with that tail appended,
  and raise through it from the exit and closed-stdout sites.
- Leave the message untouched when stderr had nothing to add.
why: Each cell killed its session between builds, dropping the server
to zero sessions. Under tmux's exit-empty default the server then began
exiting, and the next build's new-session could reach the still-bound
socket mid-shutdown and fail with "server exited unexpectedly" -- an
intermittent create failure whose rate rose with machine load. Control
mode never hit it: its tmux -C phantom already pinned the server, which
is why only subprocess cells failed.

what:
- Give every bench server a keepalive session, so a cell's teardown
  never drops it to zero. Setting exit-empty alone cannot work: an empty
  server exits before the option can land.
- Offload the async cells' kill-session to a thread; a blocking
  subprocess issued from inside a running event loop measurably raised
  the failure rate.
why: _cleanup only knows this process's socket dir, so a run killed
before its atexit hook leaves its scratch dir -- and any tmux still
bound to it -- behind for good. Those survivors keep consuming CPU and
descriptors, and machine load is exactly what makes the server-teardown
race fire, so an unreaped leak feeds the failure it came from.

what:
- On import, remove any ltbench-* scratch dir with no live tmux bound
  to it; report how many were reclaimed.
- Leave dirs with a live server alone -- they may belong to a
  concurrent run, and stealing another run's servers is worse than
  leaking a hung client until its server exits.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant