Repository navigation
feat(core): store session media as media:<hash> refs in a content-addressed blob table - #58656
Conversation
54b9209 to
3449c4e
Compare
3449c4e to
6bcb56a
Compare
Rows reference blobs by a URL-neutral media:<hash> in the existing uri field. The hash (lowercase hex SHA-256 of the raw bytes) is defined once in SessionMediaBlob. Reads still accept the legacy /api/media/<hash> form.
…aceholder on read
6bcb56a to
7fe11c2
Compare
There was a problem hiding this comment.
The read-path hydration looks sound. I checked it with the new projector tests (19/19 pass), tsc -b on core (clean) and bun run migration --check (no drift). Nothing writes refs yet, so there was nothing to drive in the TUI. I went through the callers that still decode rows without hydrating (projector getAssistant/getShell/getCompaction, revert.ts, compaction recovery in llm.ts). They read and write back assistant, shell or compaction metadata, so keeping the raw refs there seems right.
Public API: unchanged. The only addition is two new internal SQLite tables (blob, blob_ref), created by a migration that rewrites no data.
One small point inline: the legacy /api/media/ ref handling looks unnecessary, since no released build could have written those refs.
The stored user-file shape (uri, bytes, origin) isn't part of Prompt.FileAttachment. So it is only described by the local StoredUserFile type and as any in the tests (AGENTS.md says to avoid any). It may be worth giving it a real stored schema when #58741 defines the write side.
| */ | ||
| export const REF_SCHEME = "media:" | ||
| // Written only by early builds of this feature; still read, never written. | ||
| const LEGACY_REF_PREFIX = "/api/media/" |
There was a problem hiding this comment.
The blob table first appears in this PR, so I don't think any released build can have written /api/media/<hash> refs; they only existed earlier in this stack. Could this legacy branch, and the legacy cases in the tests, be dropped? As it stands, any file uri that happens to be /api/media/<64 hex> would be rewritten from the blob table or turned into a placeholder.
There was a problem hiding this comment.
The legacy /api/media/ point is addressed: only media:<hash> is treated as a ref now, and the test checks that a URI shaped like /api/media/<hash> is left as is. I re-ran the projector tests (19/19 pass) and tsc -b on core (clean). Public API unchanged. Nothing new to raise. The PR description still mentions legacy /api/media/<hash> refs, which this commit removed.
… where bytes are unused
There was a problem hiding this comment.
The new commit looks sound. Delivery now keeps media: refs in the stored row, and the readers that never use bytes (title generation, compaction recent text, nextPromotable, the output-limit check) skip blob reads. I checked their callers: they use only text, type, delivery or id, so the empty data from elide is safe there. Projector, title, runner and related session tests pass (313/313), and tsc -b on core is clean. Public API unchanged. The earlier /api/media/ point was addressed in the previous commit. Nothing new to raise.
There was a problem hiding this comment.
The schema refactor looks sound. SessionStored addresses the earlier stored-shape point: stored user files and tool refs now have a real schema, and the tests no longer use as any. I traced the callers of the slimmer nextPromotable / firstUserMessage results (runner, execution, title); they only read fields that are still returned. Delivery round-trips refs through the stored schema, and the projector test confirms the raw row still holds media:<hash>. The projector, runner, title, history, store and compaction tests pass (368/368), and tsc -b on core is clean. Public API unchanged. Nothing new to raise.
There was a problem hiding this comment.
This commit is a tidy-up with no behaviour change: an explicit sequential forEach, checks for a missing value instead of ! assertions, and a Ref type for refHash. tsc -b on core is clean and the session projector, runner, title, history, store and compaction tests pass (304/304). Public API unchanged. Nothing new to raise.
Part 1 of 5 in the media blob stack (#58656 → #58741 → #58742 → #58657 → #58743). No API URL is ever persisted, and the endpoint is experimental.
What it does
Adds a general-purpose, content-addressed
blobtable so large inline attachments can live outsidesession_messagerows, and makes every read hand back the same inline base64 as before. Nothing writes refs yet; that's #58741.The tables are named
blob/blob_refrather than anything media-specific so large tool outputs (and later instruction values, today ininstruction_blob) can share the same store and cleanup.How it works
media:<hash>in its existingurifield.<hash>is the SHA-256 hex of the decoded bytes. Onlymedia:<64 hex>counts as a ref.session/stored.ts(SessionStored) is the schema for what a stored file or tool attachment looks like on disk: either inline data or a ref. Readers decode rows with it instead of casting, and thesql.tscolumns are typed with it. Noas anyleft in the read path.blob_refrecords which message uses which blob, so unreferenced blobs can be garbage-collected later.data+source, so clients and the model see the same shape as today.blob.tsis split into small single-purpose functions (find refs, load blobs, hydrate one file, hydrate one tool result).firstUserMessagereads only the text viajson_extract. The output-limit streak check selects onlytypeandfinish.session_message, so itsmedia:refs survive promotion.[media unavailable: name (mime)]text placeholder instead of a baremedia:ref (user files become atext/plainfile, tool results a text item).Risk
Core storage and read path only. Rows without refs are untouched, and no ref rows exist yet in production, so the small behaviour changes below can't affect existing data:
source;The migration (
20261025123711_blob) only creates two empty tables and two indexes; it rewrites no data.Proof
bun test(packages/core): 334 tests acrosssession-projector,session-runner,session-title,session-compaction, all pass. The projector tests cover "missing blob → placeholder" and check the rawsession_messagerow still holdsmedia:<hash>after an inbox prompt is delivered.bun run typecheck(core): cleanbun run migration --check: cleanbunx oxlinton changed files: 0 errorsAdded by the opencode bot (AI-generated).
Related: #58436, #36444