Simulator devices: T3 parity (Android, 3D/iPhone Duo, SSH hosts, device tabs, power features, phone viewer)

[sambitcreate]

Summary

Brings Aiden's Simulator devices up to T3 Code's Devices panel (T3 a6ec88f7, MIT; ported 1:1 where possible and adapted to Aiden's routes, IPC and design system). It also adds a few capabilities neither app had. Plan: docs/plans/simulator-devices-t3-parity-plan.md.

Workstream	What landed
Android emulators	platform: "ios" | "android" across contracts. serve-emu SEMU-framed H.264 stream, Android keys plus Back/Home/Recents/Power. Typed adb actions: appearance, text size, Reduce Motion, network, orientation, location, permissions, URL/app. Android SDK and JDK discovery. Paired-Mac relay carries platform. Agent device_* tools support Android. Foldable Fold/Unfold via serve-emu /api/fold.
3D	T3's full 3D feel: spring motion with flick coast, rest-view snapping, damped camera framing, log zoom, trackpad pinch, and orbit end on gestureScrollEnd. Hardware-accurate procedural models by device name. iPhone Duo in 3D: hinged two-leaf body, per-display panel streams, touch mapped per leaf, pinch-to-fold with hinge preview, animated fold with hinge re-centring, orientation-following Book/Laptop controls, flip to switch display, Duo snap views. Android 3D fold body. "Save framed screenshot…".
SSH hosts and tool versions	SSH device hosts: ssh -L hub and agent tunnels, bootstrap scripts, local-target skip, per-host agent targets, Settings host editor, Test connection. Per-host installed/running/pinned helper versions, consent-respecting update progress, manual Update and Retry, read-only Check versions, locked reclaim, Host diagnostics.
Workspace UX	One renameable tab per device (only the visible one streams). Floating device mini-player over chat, auto-shown when the agent opens a device. Quick View docks beside chat on narrow windows. Device rows in the timeline (desktop, iOS, Android).
Power features (iOS and Android)	Accessibility-tree overlay, event log (serve-sim feed / logcat), erase (simctl erase / -wipe-data cold boot), clipboard paste (HID Cmd+V / chunked text) and copy (iOS), screen recording (simctl io recordVideo / screenrecord), multi-touch (Option-drag pinch/rotate, Option+Shift pan), screenshot to chat and "Save screenshot…", plus agent device_screenshot saveTo. Controls are gated per platform by deviceFeatureCapabilities.
Phone viewer	Native iOS (SwiftUI) and Android (Compose) full-screen simulator viewer over Aiden Remote. Pinned-TLS MJPEG, input over WebSocket, on-demand controls (shake to toggle; Android Back reveals them first), device button above the composer. New simulators:mobile capability behind an owner consent, "Share with Aiden On The Go" (off by default). Aiden Remote contract revision 26. Main claimed 25 for the Bots rework, so this moved to 26 at merge, with the fixture and the desktop, iOS and Android assertions updated together, with fixtures shared by desktop, iOS and Android.

Review and acceptance

• Two independent review passes. Every confirmed finding is fixed with a behavioral test:
  • SSH stop races
  • phone scope: shutdown refusal and a relay input-tag allowlist that closes with 1008
  • peer device-id rewrite
  • save-folder symlink race
  • AVD-style ids on phones
  • Android in the floating player
  • fresh grants for Duo panel feeds
  • 1006 vs expired grant
  • trusted platform for feature actions
  • mobile touch/lifecycle/retry/frame-memory
• Live acceptance on this Mac against the real expo-device-hub@0.12.0:
  • iPhone Duo: every fold shape and stance, pinch-to-fold, flip, rotate, touch on the inner display.
  • iPhone 17 Pro and Pro Max in 3D.
  • Power features, including HID Cmd+V paste, a playable recording, and erase on throwaway simulators.
  • Tabs and floating player.
  • A Pixel AVD.
  • Six bugs found and fixed: stale serve-sim helper after shutdown, JAVA_HOME for Android boot, paste key pacing, overlay label clamp, local event-log time, and the procedural Duo camera.

Bundled hardware models

Following the owner's decision on 2026-10-08 to match T3, Aiden bundles T3's device GLBs:

• iPhone 18 Pro and iPhone 18 Pro Max
• iPad Pro 13 (M5), with and without the Magic Keyboard
• iPhone Duo, with its real centimetre hinge rig

These are Apple AR-derived assets with no established redistribution licence. Their provenance and hashes are in renderer/assets/devices/models/sources.json and THIRD_PARTY_NOTICES.md.

How they load:

• Models load lazily as separate assets, so initial JS is unchanged and the bundle budget passes.
• A procedural body shows first and is swapped in place once the model loads. On any load or validation failure the procedural body stays.

Coverage:

• A real-asset test parses every model with three's GLTFLoader.
• Headless renders of the Pro Max, the iPad with keyboard, and the Duo at 0°, 90° and 180° were checked.

Also fixed from main

• renderer/main/chat-pane.tsx on main read a bot query that Bots rework: Grok-style single-thread bots + Pi Durable sessions #377 removed after Antigravity UI/UX polish: review fixes and phone support #373 used it. It now uses chat.data?.botId, which is equivalent because Bot chats redirect to their own route.

Decisions to note

• Launcher label. Now "Device", since it covers simulators and emulators.
• Quick View. Shifts or narrows the chat column on narrow windows. This reverses the old "never changes conversation width" rule; docs/chatgpt-desktop-ui-inspiration.md is updated.
• Android copy-from-device. Disabled with a reason, because API 36 images have no clipboard read.
• Android recording. Capped at 3 minutes (one screenrecord segment).
• Phones and Android emulators. Phones list Android emulators as "Open on your Mac to view"; they don't decode H.264 natively yet.
• Known posture (documented follow-up). While an SSH host is connected, its hub forward is reachable by local processes on loopback, the same as the local hub today.

Tests

• npm run type-check, npm run type-check:e2e, eslint .: clean.
• npm run test:devices: 564 pass. npm run test:ci-policy: 56 pass. New test files are registered in test:devices and scripts/ci-test-registry.json.
• npm test (all lanes, after merging main): core-git 2979 pass, runtime-subagents 2887 pass, renderer-other 2245 pass / 1 load-sensitive failure (below) that passes in isolation.
• Playwright: all environment-devices-* specs, settings-simulator-ssh-hosts.spec.ts and workspace-panel.spec.ts pass.
• iOS XCTest (AidenOnTheGo scheme, iPhone 17 Pro simulator): full scheme 666 passed / 0 failed; the simulator-viewer, Phase 0 and remote-client suites pass again after the review fixes.
• Android :app:testDebugUnitTest: 530 tests, 0 failures.

Flaky and pre-existing failures seen locally and in CI (per AGENTS.md)

• Android AidenChatProgressUiTest flaked on main too, and is now fixed here.
  • Root cause: Espresso.pressBack() picks a root by focus. Right after a state restore it can pick the activity window while the recreated bottom-sheet dialog holds focus, then time out (RootViewWithoutFocusException).
  • Main runs that failed the same way: 37831916309 and 37722482284.
  • Fix: back is now pressed on the sheet's dialog root. The class passed 10/10 on a headless emulator.
• Hosted E2E only: floating-player drag test.
  • Root cause: Linux CI runs in a small xvfb window. A phone-shaped player is narrow there, so the test's grab point at handle x+40 landed on a header button, and a press on a button never starts a drag.
  • Reproduced locally at 640×480 and 800×600.
  • Fix: the test now grabs the grip. It passes at 640×480, 800×600 and 1280×1024.
• tests/e2e/workspace-panel.spec.ts:294 and :328 race on main.
  • Symptom: the "LM Studio (local) added." onboarding toast intercepts the "New browser tab" click, and Sonner keeps a hovered toast open.
  • Reproduced on origin/main (4cb409d5).
  • Fixed here with a waitForToastsToClear fixture: the pointer leaves and the test waits for the toast to close. No timeouts or retries were changed.
• main/services/coding-tools.test.ts "foreground scans cap work on wide no-match directories and bound output" failed once.
  • Symptom: list_dir returned 5942 entries, expected 10000.
  • This was during a full local run while live simulator acceptance was saturating the machine. The test passes in isolation and on the other full runs.
  • This PR doesn't touch coding-tools. It looks like a time-budget sensitivity worth a follow-up.

🤖 Generated with Claude Code

[pullfrog]

Important

The new desktop rotation path misplaces taps in width-constrained flat views and desynchronizes accessibility highlights. These two behavioral issues should be addressed before merging.

Reviewed changes: Reviewed the changes since the previous Pullfrog review at 3b333ec, including the review fixes and the additional desktop orientation correction.

• Hardened screenshot saves: Replaced pathname-based truncation with a native held-parent temporary-file and rename operation, with parent-swap and hard-link coverage.
• Invalidated queued install approvals: Added streaming/agent revocation epochs and service-level checks before queued local and SSH install requests.
• Corrected native orientation: Shared the display-rotation predicate with touch encoding on iOS and Android, fitted rotated frame dimensions, and adjusted Android subsampling.
• Rotated desktop media: Turned flat-view and floating-player media and updated their displayed aspect ratios.
• Updated integration checks: Reserved floating-screen typing, updated iOS shipping/notice allowlists and the browser floating-frame test, and changed the drag test to drop inside the chat window.

Validation: TypeScript checking, 52 focused desktop tests, 14 iOS shipping checks, and 30 focused Android viewer/stream tests passed. A browser probe using production CSS and geometry helpers reproduced both inline findings. The full device suite on Linux reported 564 passes, 5 failures, and 3 skips; all five failures encounter the new writer's Saving screenshots to a file needs macOS. refusal, so native screenshot-save execution still requires macOS verification. iOS XCTest and macOS Electron acceptance were not run here.

Scope: Re-read the complete incremental delta and affected full-diff regions/callers, reusing the prior full review for unchanged code rather than repeating it.

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using gpt-6.1-sol ｜ 𝕏}

[pullfrog]

✅ No new issues found.

Reviewed changes: Reviewed the display-geometry fixes since the previous Pullfrog review at 94b2e386, excluding changes inherited from the merge of main.

• Fitted flat screens: Sized the full viewer and floating player's input surfaces to the largest displayed-aspect rectangle that fits their parent, keeping rotated media and touch bounds aligned.
• Aligned accessibility overlays: Passed displayed screen geometry to the overlay so interface-space rectangles remain aligned and raw portrait trees map into landscape display space.
• Extended geometry checks: Added focused tests for width/height constraints and displayed accessibility screen configuration.

Validation: npm run type-check and all 30 focused display, accessibility, multi-touch, player-layout, and player-render tests passed. An independent Chromium probe using production helpers and compiled production CSS passed 32 viewer/player layout cases across four sizes and all four orientations, plus accessibility hit tests for interface-space trees, raw portrait trees, and already-rotated iOS/Android frames. macOS Electron acceptance and native suites were not rerun on this Linux runner.

Scope: Read the complete incremental and head-commit diffs, affected full-diff regions, and relevant callers; reused prior full-review coverage for unchanged implementation rather than repeating it.

  ^{｜ View workflow run ｜ Using gpt-6.1-sol ｜ 𝕏}

[pullfrog]

✅ No new issues found.

Reviewed changes: Reviewed the flat-view sizing and accessibility-coordinate fixes in 3580dfd, following the previous Pullfrog review at 94b2e386.

• Fitted flat screens: Sized the viewer and floating player's input surface to the largest displayed-aspect rectangle that fits its parent, keeping rotated media and pointer bounds aligned.
• Aligned accessibility overlays: Passed displayed screen dimensions and orientation into accessibility mapping so highlights and hover hit tests follow the turned image.
• Extended geometry checks: Added width-bound/height-bound sizing cases and displayed-screen mapping tests.

Validation: All 53 focused geometry, accessibility, input, layout, and renderer tests passed, as did npm run type-check. A Chromium probe using production CSS and geometry functions passed 12 combinations of stage size and orientation, checking matching media/input bounds, containment, and off-center accessibility hit testing. This was a focused browser probe, not macOS Electron acceptance.

Scope: Read the complete incremental delta, the four-file commit diff, affected authoritative full-diff regions, and related callers; reused the earlier full review for unchanged code rather than repeating the entire 236-file review. Previous Pullfrog threads are already resolved. Hosted checks on this exact head had not completed when checked; this review does not establish CI readiness.

  ^{｜ View workflow run ｜ Using gpt-6.1-sol ｜ 𝕏}

[pullfrog]

✅ No new issues found.

Reviewed changes: Reviewed the bundled-model and merge-integration delta since the previous Pullfrog review at 3580dfdb.

• Added bundled hardware models for the iPhone 18 Pro/Pro Max, iPad Pro 13-inch M5, Magic Keyboard, and iPhone Duo, with provenance and the owner-accepted licensing decision documented.
• Added asynchronous model loading with cancellation, resource cleanup, validation, and procedural fallbacks; adjusted phone/iPad lighting and Duo scale-aware replacement.
• Added Magic Keyboard controls that attach the matching accessory and preserve its landscape resting orientation.
• Updated merge integration to reserve remote contract revision 26 across the shared fixture and client assertions, register model tests, and remove the stale Bot query reference in chat-pane.tsx.

Validation: All 77 focused 3D/model tests, 157 protocol/renderer integration tests, 61 Android contract/viewer tests, and 13 iOS shipping checks passed. TypeScript checking, the production renderer build, and the bundle budget passed. An isolated Electron file:// probe loaded all five real GLBs, decoded embedded textures, rendered the device bodies, and passed 12 off-center phone/iPad input checks across all four orientations.

Scope: Read the complete incremental delta, affected authoritative full-diff regions, and related consumers; reused prior full-review coverage for unchanged implementation rather than repeating the entire PR. iOS XCTest and macOS simulator acceptance were not run here. Hosted CI on this exact head was still incomplete, so this review does not establish merge readiness.

  ^{｜ View workflow run ｜ Using gpt-6.1-sol ｜ 𝕏}

[pullfrog]

✅ No new issues found.

Reviewed changes: Reviewed the floating-player E2E synchronization fix in 8dc3757, following the test-label-only acknowledgement at b1a61d24 and the previous substantive review at 3da48fbb.

• Added toast synchronization before dragging, using the existing helper to move the pointer away from hover-paused notifications and wait for their removal.
• Added handle actionability checking before measuring and dragging the player, preserving the existing snap and position-persistence assertions without increasing timeouts or retries.
• Corrected the Bot approval expectation in the previously acknowledged b1a61d24 to match the existing sentence-case tool label.

Validation: E2E TypeScript checking and lint for the changed spec passed; Playwright collected the updated stream test successfully. The actual Electron test requires macOS and was not run on this Linux runner, so elimination of the hosted flake remains unverified here.

Scope: Read the complete incremental and commit diffs, the affected authoritative full-diff regions, fixture helper, complete spec, and player pointer handlers. Reused prior full-review coverage for unchanged implementation rather than repeating the entire PR for this test-only change. All earlier Pullfrog threads remain resolved; repository sources are unchanged. This review does not establish hosted CI readiness.

  ^{｜ View workflow run ｜ Using gpt-6.1-sol ｜ 𝕏}

[pullfrog]

✅ No new issues found.

Reviewed changes: Reviewed the two test-behavior fixes since the previous Pullfrog review at 8dc3757.

• Changed the floating-player drag target from header offset 40 to 10, placing the initial press on the grip rather than a header button while preserving snap and position-persistence assertions.
• Targeted Android Back at the sheet dialog using an explicit Espresso dialog-root matcher, preserving real Back-key injection and the nested navigation, restoration, and dismissal assertions.

Validation: E2E TypeScript checking, lint for the changed spec, and focused Playwright collection passed; all 10 focused mini-player renderer/layout tests passed. Android instrumentation-test compilation completed successfully with cached tasks using --offline --no-daemon; two preceding synchronous attempts hit Tool 'shell' timed out after 120000ms. The actual Android instrumentation tests and macOS Electron flow were not run here, so runtime elimination of the flakes remains unverified locally.

Scope: Read the complete incremental and both commit diffs, affected authoritative full-diff regions, both complete test files, and the production drag/back handlers. Reused prior full-review coverage for unchanged implementation rather than repeating the entire PR for two isolated test fixes. Earlier Pullfrog threads are already resolved. CI on this exact head still had Android, iOS, Linux, and Electron E2E checks in progress when inspected; this review does not establish merge readiness.

  ^{｜ View workflow run ｜ Using gpt-6.1-sol ｜ 𝕏}