Skip to content

feat(app): opt into experimental-media-uri and keep draft media bytes local - #58743

Merged
kitlangton merged 4 commits into
media-backfillfrom
media-app
Oct 25, 2026
Merged

kitlangton merged 4 commits into
media-backfillfrom
media-app

Conversation

@kitlangton

@kitlangton kitlangton commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

What it does

Desktop and web ask for media:<hash> refs, so a long session with screenshots stops re-sending every image as base64 each time it loads. Images load on demand through a cache instead. It's experimental: nothing changes until a session's location turns on experimental.media_blob, which is off by default. Until then the server keeps sending inline data.

Stack, built on the merged #58656: write (#58741) → backfill (#58742) → wire (#58657) → app (this PR).

Why

Inline media is a third larger than its bytes and arrives again on every message read, for every image in the session. With refs, each image is fetched once per server and cached. Before → after bytes per load have not been measured yet.

How it works

  1. The app's API client sends X-OpenCode-Capabilities: experimental-media-uri.
  2. A per-server cache fetches bytes through experimental.media.get with the server's auth, turns them into object URLs and revokes the oldest beyond 128. Message attachments render from it.
  3. Drafts own their bytes. A media: image restored into the composer is copied into the local draft store on the first save, so server-side cleanup can't break a draft. Submitting always sends bytes.
  4. Undoing a queued prompt, reverting one that hasn't been delivered yet, and editing a queued prompt all end in an inbox cancel, and the cancel sweeps blobs nothing else references. Each one first copies the prompt's media: bytes through the cache into local draft blobs. If undo or revert can't load them, they show an error and leave the prompt queued. Edit swaps in the local copies as they load.

Risk

App UI and draft persistence. If a media load fails, the image shows as unavailable and the draft stays intact. Tool-result images are not shown from refs yet. Legacy /api/media/ URLs are not supported; only media:<hash> refs are. Not verified end-to-end in the desktop app against a live server with the flag on.

Proof

  • bun test --conditions=solid on drafts.test.ts, image.test.ts, queue.test.ts and prompt.test.ts (packages/app): 72 pass. They cover draft save and restore with media: refs, a submit after the server swept the blob, and undo pinning media before the cancel (and not cancelling when the bytes are gone).
  • bun test --conditions=browser on draft-history-cache.test.ts and prompt-persistence.test.ts (packages/app): 18 pass.
  • bun test src/components/message-file.test.ts (packages/session-ui): 3 pass.
  • bun run typecheck (app, session-ui): clean. Prettier: clean. oxlint on changed files: 0 errors.

The server sends media:<hash> refs, never route paths, and user files keep
their source, so drop the legacy route parsing and the origin fallback.
Reverting an undelivered prompt, undoing a queued prompt and editing one all end in an inbox cancel that sweeps blobs nothing else references. Each now copies its media: bytes through the media cache into local draft blobs first; revert and undo leave the prompt queued when the bytes cannot load.
Base automatically changed from media-wire to media-backfill October 25, 2026 23:38
@kitlangton
kitlangton merged commit 4259705 into media-backfill Oct 25, 2026
6 checks passed
@kitlangton
kitlangton deleted the media-app branch October 25, 2026 23:38
@kitlangton
kitlangton restored the media-app branch October 25, 2026 23:38
@kitlangton

Copy link
Copy Markdown
Contributor Author

Superseded by #59104 (GitHub auto-marked this merged into the stack base during a reorder; nothing reached dev).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant