Repository navigation
feat(app): opt into experimental-media-uri and keep draft media bytes local - #59104
Conversation
942f3e7 to
1f0a9e1
Compare
…he media loader, and submit bytes
The server sends media:<hash> refs, never route paths, and user files keep their source, so drop the legacy route parsing and the origin fallback.
Reverting an undelivered prompt, undoing a queued prompt and editing one all end in an inbox cancel that sweeps blobs nothing else references. Each now copies its media: bytes through the media cache into local draft blobs first; revert and undo leave the prompt queued when the bytes cannot load.
…ds, keep exports inline, and persist media refs without a URL
![opencodejr[bot]](https://avatars.githubusercontent.com/in/5497873?s=60&v=4){kind=small}
There was a problem hiding this comment.
The approach looks sound. Only the three ref-capable reads (message.list, session.message.get, session.inbox.list) get the capability header, export opts out, and the undo and undelivered-revert paths copy media: bytes locally before cancelling the inbox item, so a failed copy leaves the prompt queued. I couldn't test it end to end because there's no web/desktop app harness here with experimental.media_blob on. Unit tests (app 923, session-ui 318) and tsc -b for both packages pass.
Public API: unchanged. The server API, SDK, plugin API, config and CLI are untouched. The app sends the existing X-OpenCode-Capabilities: experimental-media-uri header on three reads, and session-ui's internal DataProvider gains optional readMedia/peekMedia props.
One concern is worth fixing, noted inline: the cache revokes object URLs that rendered attachments and drafts still use.
| const touch = (hash: string, url: string) => { | ||
| if (urls.has(hash)) urls.delete(hash) | ||
| urls.set(hash, url) | ||
| while (urls.size > maxEntries) { |
There was a problem hiding this comment.
Eviction calls URL.revokeObjectURL on URLs that other code may still be holding:
CurrentUserMessageDisplaykeeps the loaded URL inresolvedUrl, and clicking the image opensImagePreviewwith that same URL.resolveBlobUrlreturns it to the composer formedia:draft images.
The cache is shared per server, so once more than 128 distinct images have loaded (exactly the long screenshot sessions this targets), older images have dead srcs. You'd see a broken preview, a broken image after a re-mount, or a broken composer thumbnail, with no reload, because peek misses but the component never calls load again.
Two ways to fix it: only revoke entries nothing currently holds (a refcount or release from the component's onCleanup), or have consumers re-load when a URL might have been evicted.
There was a problem hiding this comment.
Fixed in 7709085. The cache no longer evicts. Every object URL it hands out stays valid until the server cache is disposed (server switch or teardown), so rendered attachments, ImagePreview and composer drafts can never hold a revoked URL. Refcounting would have needed a release from every consumer, and the composer and submit paths have no clear lifetime to release from. Images are still fetched only when they are viewed, so memory stays below the old inline data: payloads. Added a test that loads 200 images and checks the first URL still resolves until dispose.
…osed Clean the anti-slop lint findings in the files this branch touches.
There was a problem hiding this comment.
This commit fixes the earlier object-URL revocation point: the media cache now keeps object URLs until the server's cache is disposed, so images that are still on screen, open in a preview or in a draft stay valid. That holds a server's images in memory while it's connected, which is about what the inline base64 data cost before. The rest of the commit is formatting, lint suppressions and type tightening. I found nothing new worth blocking on. The description still says the cache "revokes the oldest beyond 128", which is no longer true.
I couldn't run it in the app against a live server with the flag on. Unit tests (app 924, session-ui 318) and tsc -b for both packages pass.
Public API: unchanged.
What it does
Desktop and web ask for
media:<hash>refs, so a long session with screenshots stops re-sending every image as base64 each time it loads. Images load on demand through a cache instead. It's experimental: nothing changes until a session's location turns onexperimental.media_blob, which is off by default. Until then the server keeps sending inline data.Replaces #58743, which GitHub marked merged into its stack base while the stack was being reordered. Nothing from it reached
dev.Stack, built on the merged #58656: write (#58741, merged) → wire (#59103) → app (this PR) → backfill (#58742, optional, can land later).
Why
Inline media is a third larger than its bytes and arrives again on every message read, for every image in the session. With refs, each image is fetched once per server and cached. Before → after bytes per load have not been measured yet.
How it works
message.list,session.message.get,session.inbox.list) sendX-OpenCode-Capabilities: experimental-media-uri, as a per-call header. Every other request stays exactly as before, so no new CORS preflights. Export passes{ mediaRefs: false }, so exported sessions always carry inline bytes.experimental.media.getwith the server's Basic auth, turns them into object URLs, and keeps each URL until the server switches or the app closes, so images on screen, in a preview or in a draft never break. Message attachments render from it.media:image restored into the composer is kept as a draft reference with an empty URL (the same shape as a draft-store id, so older builds still read the draft) and is copied into the local draft store on the first save, so server-side cleanup can't break a draft. Submitting always sends bytes.media:bytes through the cache into local draft blobs. If undo or revert can't load them, they show an error and leave the prompt queued. Edit swaps in the local copies as they load.Flag off (the default) and older servers
The server ignores the capability unless
experimental.media_blobis on, so these three reads return the same inline attachments as today. The only request difference is that header on those three GETs. Restoring attachments into the composer follows the old rule exactly (source.type === "inline" || datanon-empty), so URI-sourced attachments with inline bytes still come back on revert, fork and recall.Risk
App UI and draft persistence. If a media load fails, the image shows as unavailable and the draft stays intact. Tool-result images are not shown from refs yet. Legacy
/api/media/URLs are not supported; onlymedia:<hash>refs are. Not verified end-to-end in the desktop app against a live server with the flag on.Proof
bun run test:unit(packages/app): 923 pass. New cases: the capability header goes only on the three ref reads (and is dropped withmediaRefs: false), export requests inline media, a URI-sourced attachment with inline bytes restores, and a restoredmedia:image encodes and decodes through the draft schema.bun test src(packages/session-ui): 318 pass.bun typecheck(app, session-ui): clean.