Skip to content
Merged
54 changes: 42 additions & 12 deletions docs/specs/dor-browser.md
Original file line number Diff line number Diff line change
Expand Up @@ -284,18 +284,44 @@ The stream WebSocket provides:
- tab snapshots,
- native `input_mouse` / `input_keyboard` input.

Dormouse does not render the stream JPEG by default. The screencast is
CSS-resolution only — Chromium's `Page.startScreencast` captures in DIP with no
DPR knob, so its frames upscale to mush on HiDPI; this is a Chromium limit, not
agent-browser's, so owning the CDP connection wouldn't change it. So Dormouse
treats frame messages as change pulses, captures a crisp device-resolution
screenshot through the host's `agentBrowserScreenshot`, and draws that to canvas
with latest-only backpressure. A capture whose bytes are identical to the last
displayed frame (a static page the daemon keeps re-pulsing) costs no decode or
draw; a re-attach bumps a draw generation so a fresh blank canvas still repaints.
A host without `agentBrowserScreenshot` renders only the placeholder — stream
frame bytes are discarded by design (the connection reduces every frame message
to a `frame-pulse`), so there is no frame-drawing fallback path.
Dormouse paints a changed stream JPEG immediately as a **provisional frame** for
the first image and for 250ms after pointer input (continuous movement extends
the window), then replaces it with a crisp device-resolution screenshot through
the host's `agentBrowserScreenshot`. This two-stage paint keeps hover and other
pointer feedback aligned with the live cursor instead of waiting for a
screenshot child-process round trip, while an idle animated page does not pay to
decode the stream continuously and the resting image stays sharp on HiDPI. The
provisional frame is CSS-resolution because Chromium's `Page.startScreencast`
captures in DIP with no DPR knob; this is a Chromium limit, not agent-browser's.

Both paths are latest-only. A newer stream pulse cancels an older provisional
decode. A provisional frame painted while a crisp capture is in flight marks that
capture stale, so it cannot overwrite the newer responsive pixels.

Because a capture (~120ms) is slower than the stream (~20Hz), *every* capture
started inside the provisional window is superseded before it resolves. The loop
therefore does not start one at all while the window is open — it defers to the
window's end and takes a single settled shot, so a drag costs one host round trip
instead of one per pulse. This is free of pixels: the deferred shots never drew
anything, and the frame that lands is the first one started after the last
provisional paint either way. Continued pointer input pushes the window out, so
the wait re-arms rather than firing early.

A capture dropped as stale leaves the loop dirty, because nothing else will
necessarily pulse it again — a single pointer move over a static page pulses once,
and that pulse is consumed by the very capture the provisional supersedes. Without
the retry the pane would keep the CSS-resolution frame until the page next changed.
An unpainted pulse alone does not suppress a crisp draw, so idle animated pages
still update.
Byte-identical daemon heartbeat frames are dropped before either path, and
byte-identical crisp captures skip decode/draw. That byte-dedup assumes the crisp
loop is the only writer, so anything else that paints the canvas must bump the
**draw generation** folded into its key: re-attach does (a fresh canvas mounts
blank), and so does every provisional paint (or a resting page whose crisp bytes
match the last crisp draw would dedup to a no-op and strand the pane on the blurry
frame — the sharp-at-rest guarantee above).
A host without `agentBrowserScreenshot` paints every changed provisional stream
frame as its lower-resolution final image rather than showing only the placeholder.

The high-rate `[ab-panel]`/`[agent-browser]` stream and screenshot console
diagnostics sit behind the `dormouse.flags.abDebugLogs` localStorage flag, read
Expand All @@ -304,6 +330,10 @@ always on as the post-hoc tool.

Important input details:

- Canvas pointer coordinates map through one width-derived scale on both axes;
using the frame/device heights would stretch input when a stream frame is
shorter than the viewport. Source of truth: `toDevice` in
`AgentBrowserPanel.tsx`.
- `input_keyboard.text` is always sent; non-text keys use `text: ""`.
- `windowsVirtualKeyCode` comes from a real key map, never `key.charCodeAt(0)`
(`.` is char 46 = VK_DELETE, so periods would otherwise become Delete presses).
Expand Down
2 changes: 1 addition & 1 deletion docs/specs/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ A **Session** is the tuple of its `SessionId` plus one state per layer. `Session
| State | Meaning |
|---|---|
| `Paned` | Rendered as a pane in the content area (a Lath leaf) |
| `Zoomed` | Subset of `Paned` — the selected pane is maximized |
| `Zoomed` | Subset of `Paned` — the passthrough-focused pane is maximized; acquiring zoom gives it focus, and losing that focus returns it to `Paned` |
| `Doored` | Rendered as a door on the baseboard |
| `Hidden` | In neither pane nor door — the webview is closed, the Surface belongs to an inactive Workspace (standalone), or the Surface is mid-transition. Process and Activity are unaffected. |

Expand Down
19 changes: 13 additions & 6 deletions docs/specs/layout.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ Elements from left to right:
- Mouse-reporting override icon (only when the inside program requests mouse reporting; hidden in minimal tier)
- SplitHorizontalIcon `split left/right [|]` (full tier only)
- SplitVerticalIcon `split top/bottom [-]` (full tier only)
- ArrowsOutIcon / ArrowsInIcon `zoom / unzoom [z]` (full tier only)
- ArrowsOutIcon / ArrowsInIcon `zoom [z] / unzoom` (full tier only)
- ArrowLineDownIcon `minimize [m]`
- XIcon `kill [x]` (hover turns error-red)

Expand Down Expand Up @@ -172,6 +172,8 @@ Wall starts in `command` mode by default. Embedders may pass `initialMode="passt
**Enter passthrough mode:**
- Click any pane body or header
- Press `Enter` on a selected pane in command mode
- Create a terminal through a manual split (`|` / `%` / `-` / `"`, a header split button) or a host New Terminal action; the new pane is selected and focused immediately
- Press `z` on a selected pane in command mode (also zooms it)
- Click or press `Enter` on a door (restores session first)
- Focus is deferred via `requestAnimationFrame` so it lands after the click/mousedown event finishes

Expand All @@ -180,28 +182,29 @@ Wall starts in `command` mode by default. Embedders may pass `initialMode="passt
- Detected via capture-phase `keydown` listener on `e.key === 'Meta'` (or `e.key === 'Shift'`) and `e.location` (1 = left, 2 = right). The Meta and Shift tracks are independent, so a Left Cmd followed by a Right Shift does not trigger.
- Works even when xterm has DOM focus because listener uses capture phase
- On keyboards without a right Meta key (common on Windows/Linux laptops), the Shift track is the available gesture; both tracks are always active.
- If the focused pane is zoomed, returning keyboard focus to command mode starts unzoom immediately.

## Keyboard shortcuts (command mode)

All handled in a single capture-phase `keydown` listener on `window`. Every handled key calls `preventDefault()` + `stopPropagation()`. While a rename input is active, all shortcuts are bypassed.

| Key | On pane | On door |
|-----|---------|---------|
| `\|` / `%` | Horizontal split — new pane to the right | — |
| `-` / `"` | Vertical split — new pane below | — |
| `\|` / `%` | Horizontal split — new pane to the right, then enter passthrough | — |
| `-` / `"` | Vertical split — new pane below, then enter passthrough | — |
| Arrow keys | Spatial navigation between panes | Left/Right between doors, Up to panes |
| `Cmd/Ctrl+Arrow` | Swap session content with neighbor | — |
| `Enter` | Enter passthrough mode | Restore session + enter passthrough |
| `,` | Inline rename | — |
| `x` / `k` | Kill with confirmation | Restore session + kill confirmation |
| `m` / `d` | Minimize to door | Restore session (stay in command) |
| `z` | Toggle maximize/restore | — |
| `z` | Zoom pane and enter passthrough | — |
| `t` | Toggle TODO flag | — |
| `a` | Dismiss or toggle alert | — |

### Split cwd inheritance

When a split is initiated from an existing pane (via `|`/`%`/`-`/`"`, the header split buttons, or `Cmd/Ctrl+Click` on a split icon), the new pane spawns with its source pane's last-known cwd as the spawn directory. The source cwd is read from `getTerminalPaneState(sourceId).cwd`; remote cwds (`isRemote === true`, e.g. an OSC 7 path reported over ssh) are ignored because they aren't usable as a local spawn cwd. When no source cwd is known, when the split has no source pane (initial pane creation), or when the source is remote, the host's default cwd applies. The inherited cwd rides through `setPendingShellOpts` alongside the inherited shell selection and is consumed by `getOrCreateTerminal` on the next `platform.spawnPty`.
When a split is initiated from an existing pane (via `|`/`%`/`-`/`"`, the header split buttons, or `Cmd/Ctrl+Click` on a split icon), the new pane spawns with its source pane's last-known cwd as the spawn directory, then becomes selected and enters passthrough immediately. Host New Terminal actions use the same focus tail. Repeated layout construction therefore requires re-entering command mode between manual splits. Focus-neutral control-plane creation (`dor split -- …`, `dor ensure`, `dor iframe`, `dor ab`) retains its documented background behavior. The source cwd is read from `getTerminalPaneState(sourceId).cwd`; remote cwds (`isRemote === true`, e.g. an OSC 7 path reported over ssh) are ignored because they aren't usable as a local spawn cwd. When no source cwd is known, when the split has no source pane (initial pane creation), or when the source is remote, the host's default cwd applies. The inherited cwd rides through `setPendingShellOpts` alongside the inherited shell selection and is consumed by `getOrCreateTerminal` on the next `platform.spawnPty`.

### Kill confirmation

Expand Down Expand Up @@ -327,7 +330,7 @@ Each session also carries `TerminalPaneState` from `docs/specs/terminal-state.md

## Theme

The Lath host styling lives in the `.lath-host` / `.lath-leaf` rules in `lib/src/index.css`: an app-bg host, a 30px header band per leaf, and a terminal-bg body. The content area uses a 6px top/sides inset and 2px bottom inset (`px-1.5 pt-1.5 pb-0.5` on wrapper, `inset-x-1.5 top-1.5 bottom-0.5` on container); the `LATH_LAYOUT_OPTS` gap of 6px is the only visual separator between panes.
The Lath host styling lives in the `.lath-host` / `.lath-leaf` rules in `lib/src/index.css`: an app-bg host and a terminal-bg body. Each leaf has a 30px header band applied by LathHost from the shared `PANE_HEADER_HEIGHT_PX` in `lib/src/components/design.tsx`. The content area uses a 6px top/sides inset and 2px bottom inset (`px-1.5 pt-1.5 pb-0.5` on wrapper, `inset-x-1.5 top-1.5 bottom-0.5` on container); the `LATH_LAYOUT_OPTS` gap of 6px is the only visual separator between panes.

Colors use a two-layer CSS variable strategy: `@theme --color-*` tokens → `var(--vscode-*)`. VSCode provides host theme variables in extension mode; standalone and website mode apply bundled or installed theme variables before rendering. Tailwind v4 `@theme` block registers `--color-*` tokens as Tailwind colors (e.g., `bg-app-bg`, `text-app-fg`, `border-border`). See `theme.css` for the full token map.

Expand All @@ -337,6 +340,10 @@ The Lath host paints `var(--color-app-bg)` so gutters and rounded pane/header co

All pane motion is owned by the Lath **animator** — a pure function of time that turns committed layout changes into interpolated frames, applied imperatively to the leaf divs by LathHost (`docs/specs/tiling-engine.md` → "Animation"). Default motion is 440ms `cubic-bezier(0.22, 1, 0.36, 1)`; under reduced motion the animator runs the same code with a 0 duration (instant). The selection overlay measures the leaf divs, which carry the interpolated inline geometry, so `getBoundingClientRect` tracks the tween frame-accurately. There are no CSS entrance/exit classes. Terminal panes, by contrast, do not refit every frame: `TerminalPane`'s resize observer throttles `refitSession` (leading edge, then at most one per ~150ms while resizes keep arriving, plus a trailing call at rest), so a motion or sash drag reflows the xterm buffer and fires a PTY resize a handful of times instead of once per animated cell-boundary crossing, while the resting geometry still gets an exact fit.

### Zoom (elevated expansion)

Zoom is presentation-only: the split tree and every tiled rect remain unchanged. It is coupled to passthrough focus: acquiring zoom enters passthrough and focuses that pane; exiting passthrough, focusing another pane, or selecting a Door starts unzoom immediately. Only the zoom owner's header shows Unzoom, with its foreground/background header tokens inverted so the escape action stands out; partially exposed panes retain their normal Zoom control. Only the owner's control toggles zoom off — the perimeter leaves other headers reachable, and their Zoom control hands zoom over (focus included) rather than merely unzooming the owner. The chosen pane rises above the tiled/dying bands and sashes before expanding from its tiled rect to the Wall rect inset by half the 30px pane-header height (15px). The perimeter exposes the tiled layout beneath, while a blurred `--color-app-bg` shadow separates the elevated edge from the content below. Unzoom reverses the geometry while keeping the pane elevated and shadowed for the whole return; both elevation effects clear only on the settled frame. Source of truth: focus/zoom orchestration and `ZoomedIdContext` in `lib/src/components/Wall.tsx`; `paneZoomButtonClass` and `PANE_HEADER_HEIGHT_PX` in `lib/src/components/design.tsx`; `presentationTargets`, `LATH_ZOOM_MARGIN`, `LATH_ZOOM_SHADOW`, and the layer-to-z-index mapping in `lib/src/components/wall/LathHost.tsx`; discrete layer lifetime in `lib/src/lib/lath/animator.ts`.

### Spawn (new pane reveal)

A newly added leaf enters by growing from the boundary it was placed against, at opacity 0 → 1 (a split to the right grows from its left boundary, and so on). The store's mutators derive this **enter hint** from the edge they commit; the auto-spawn refill overrides it to `'top-left'` (the killed last pane shrank toward the bottom-right, so the refill grows from the opposite corner). See `docs/specs/tiling-engine.md` → "Animation" → Enter.
Expand Down
6 changes: 3 additions & 3 deletions docs/specs/shortcuts.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ In the VS Code extension host, selected workbench chords are mirrored: the termi

| Key | Action | Description |
|-----|--------|-------------|
| `\|` or `%` | Split left/right | Split the selected pane into two side-by-side panes. |
| `-` or `"` | Split top/bottom | Split the selected pane into two stacked panes. |
| `z` | Toggle zoom | Fullscreen the selected pane, or return to the normal layout. |
| `\|` or `%` | Split left/right | Create a pane to the right, select it, and enter passthrough. |
| `-` or `"` | Split top/bottom | Create a pane below, select it, and enter passthrough. |
| `z` | Zoom and focus | Elevate the selected pane and enter passthrough; leaving passthrough or focusing elsewhere ends zoom. |
| `m` or `d` | Minimize / reattach | Minimize the selected pane to the baseboard, or reattach a minimized door. |
| `k` or `x` | Kill | Kill the selected pane or door. Prompts for a random character to confirm; untouched (never-typed-in) panes and doors are killed immediately without the prompt. |
| `,` | Rename | Enter rename mode for the selected pane's title. |
Expand Down
3 changes: 2 additions & 1 deletion docs/specs/standalone.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,8 @@ shell controls, the theme picker (`docs/specs/theme.md`), and the window
controls (minimize / maximize / close via `@tauri-apps/api/window`, with
window-focus tracking dimming the bar). The shell controls are:

- **`[+]`** — spawns a new terminal with the currently selected shell
- **`[+]`** — spawns a new terminal with the currently selected shell, selects it,
and enters passthrough immediately
(dispatches the `dormouse:new-terminal` CustomEvent that `Wall` listens
for; the VS Code equivalent is the `dormouse:newTerminal` postMessage in
`docs/specs/transport.md`).
Expand Down
Loading
Loading