fix(rag): say that embedding generation runs, and test it through context.db (#1128, #1272)

.changeset/brisk-columns-refuse.md

@@ -25,8 +25,3 @@ await context.db.Article.update({
 Sudo is unchanged: `checkFieldAccess` returns `true` under sudo, so an elevated write never
 reaches the throw. A caller that relied on the silent drop to pass a denied field through
 an ordinary write must stop sending the key, or write under `sudo()`.
-
-`hookPipeline` — the transform+validate span of a write, exactly as `write-pipeline.ts`
-runs it — is exported from `@opensaas/stack-core/internal` so sibling packages whose field
-types are enforced inside it can test against the real pipeline rather than re-deriving
-its phase order.

.changeset/eager-vectors-arrive.md

@@ -0,0 +1,46 @@
+---
+'@opensaas/stack-rag': minor
+'@opensaas/stack-core': minor
+---
+
+Automatic embedding generation runs, and the package no longer says otherwise
+
+`@opensaas/stack-rag` was built and documented while the secured write surface could not
+execute on the Prisma 8 collection, so the plugin's escalated write threw on every
+invocation. That surface now executes, and generation with it:
+`context.db.Article.create({ data: { content } })` commits the row, and once that
+transaction settles the plugin embeds the **persisted** source text and writes the vector
+and its metadata past that field's own write denial. Writing the source text again
+regenerates it; a write that leaves the source text alone does not, because the
+`sourceHash` on the stored metadata short-circuits.
+
+Everything written for the inert surface is gone with it:
+
+- `generation-failure.ts` no longer classifies "the secured write surface has not been
+  ported" as a standing defect. The predicate matched
+  `findUnique is not a function` / `Unknown column "data"`, neither of which the write
+  pipeline can now raise, and the branch logged
+  `EMBEDDING GENERATION IS NOT RUNNING … No config change works around it` — a false
+  statement to a user. A provider `type` no factory answers to is still reported as
+  standing; everything else is still reported per occurrence as transient.
+- The write denial and the search helpers are tested through `context.db` rather than
+  through `hookPipeline`, and every vector under assertion is one the generation hook
+  produced from source text written through the same surface. `allowManualWrites` is
+  asserted by reading the columns back rather than by inspecting resolved data.
+
+`hookPipeline` is no longer exported from `@opensaas/stack-core/internal`. It was added
+there so `@opensaas/stack-rag` could prove its write denial one layer below
+`context.db`, which was the deepest seam that then existed; nothing depends on it now.
+That path carries no semver guarantee and the export was never released.
+
+`allowManualWrites` remains the deliberate opt-out for an application that maintains its
+own vectors — it is not a workaround for anything:
+
+```typescript
+manualVector: embedding({ dimensions: 1536, allowManualWrites: true })
+```
+
+Core's multi-column write-access gate is now also proven through `context.db`: a denied
+create or update throws naming the field and leaves the per-part columns untouched, a
+granted one writes both, `sudo()` bypasses the gate, and clearing the field with `null`
+clears both columns.

.changeset/humble-handles-narrow.md

@@ -27,6 +27,6 @@ const plugin = {
 }
 ```
 
-The Write Pipeline rebinds `ormHandle` wherever it rebinds `context.db`, exactly as it did before — this rename changes nothing about when a hook's database work is transactional. (On `prisma-8` no write currently opens a transaction at all: #1205.)
+The Write Pipeline rebinds `ormHandle` wherever it rebinds `context.db`, exactly as it did before — this rename changes nothing about when a hook's database work is transactional. Every write opens a transaction (ADR-0010), so a hook's own write through either handle rolls back with the write that failed.
 
 `getContext()`'s second positional parameter is renamed to match; it is positional, so no call site changes. `@opensaas/stack-auth`'s better-auth wiring and `@opensaas/stack-rag`'s vector search now read `context.ormHandle`.

.changeset/loud-comets-invent.md

@@ -110,29 +110,22 @@ provider and saying `ollamaEmbeddings({ dimensions })` is required — a generat
 refusal rather than a compile error. Use the `ollamaEmbeddings()` / `openaiEmbeddings()`
 helpers, whose parameters are the concrete config types, to get the error from `tsc`.
 
-**Embedding generation does not run in this release (#1124, #1127).** Everything above —
-the column, its dimension, the index declaration, the write denial, `nearest()` — is real
-and works. Generation itself does not: the plugin writes a generated embedding through the
-secured write surface under sudo, and that surface has not been ported onto the Prisma 8
-collection yet, so the write throws on **every** invocation.
-
-What an application sees today: `context.db.Article.create({ data: { content } })`
-succeeds and the row commits normally; the embedding column stays `null`; and the log says
-so, naming #1124 and #1127. Semantic search over that field returns nothing, because there
-is nothing in the column. There is no config change that works around it.
-
-The log says it once in full per field, and then one line per row after that, because it
-is a standing defect rather than a per-row event. Which of the two you get is decided by
-the **error**, not by where in the hook it was raised: a failure that matches the unported
-write surface, or a provider `type` no factory answers to, is reported as standing —
-naming what has to change and saying that retrying will not help. Anything else is
-reported per occurrence as transient, saying the row is committed and to retry by writing
-the source field again.
-
-There is also no regeneration path (#1271), so rows written before #1127 lands keep their
-null embeddings afterwards — plan to re-save the source field, or backfill, once it does.
-If you need vectors before then, use `embedding({ allowManualWrites: true })` and write
-them yourself.
+**Embedding generation runs end to end.** `context.db.Article.create({ data: { content } })`
+commits the row, and once that transaction settles the plugin embeds the **persisted**
+source text and writes the vector and its metadata to the column past that field's own
+write denial. Writing the source text again regenerates it; a write that leaves the source
+text alone does not,
+because the `sourceHash` on the stored metadata short-circuits.
+
+When a generation does fail, the log distinguishes a **standing** defect from a transient
+one by the **error**, not by where in the hook it was raised. A provider `type` no factory
+answers to is standing: it is said once in full per field and then one line per row, naming
+what has to change and saying that retrying will not help. Anything else is reported per
+occurrence as transient, saying the row is committed and to retry by writing the source
+field again.
+
+There is no regeneration command (#1271), so a row whose generation failed keeps its null
+embedding until its source field is written again.
 
 **Further known limits on generation (#1271).** Embeddings are generated in an
 `afterTransaction` hook, after the row commits, which bounds what it can do:
@@ -141,15 +134,16 @@ them yourself.
   it as a failure would invite a retry that duplicates the row. The row keeps a null
   embedding, and there is no regeneration path yet.
 - A **nested** record is never embedded — `afterTransaction` carries a persisted row for
-  the top-level record only, so `User.create({ data: { articles: { create: [...] } } })`
-  leaves those Articles with a null embedding, with a warning naming the list.
+  the top-level record only. On this release that row cannot be created in the first place:
+  a nested spelling under a relationship key is refused by `NestedRelationInputError`
+  (ADR-0050), so the hook's warning is a backstop rather than something a write reaches.
 
 Generation keys on the **persisted** source text, not the caller's input, so a source
 field a `resolveInput` hook derives is embedded like any other.
 
 The embedding and its metadata are write-denied to application code: an ordinary create or
-update naming them throws. The plugin writes them itself under sudo, after the write's
-transaction settles. Applications that maintain their own vectors opt out explicitly:
+update naming them throws. The plugin writes them itself, past that denial and running no
+hook of the list's (ADR-0066), after the write's transaction settles. Applications that maintain their own vectors opt out explicitly:
 
 ```typescript
 manualVector: embedding({ dimensions: 1536, allowManualWrites: true })

.changeset/nine-otters-describe.md

@@ -30,5 +30,6 @@ A `db.nativeType` value outside the Postgres types the contract carries is now a
 
 ```ts
 const article = await context.db.article.findFirst()
-article.body // import('@opensaas/stack-tiptap').JSONContent | null
+// `null` here is "no row, or the Access Filter denied it" — guard before reading.
+article?.body // import('@opensaas/stack-tiptap').JSONContent | null | undefined
 ```

.changeset/quiet-columns-settle.md

@@ -0,0 +1,78 @@
+---
+'@opensaas/stack-core': minor
+'@opensaas/stack-rag': minor
+---
+
+A plugin's write of a column it owns runs no hook, which stops it destroying derived fields
+
+An embedding is write-denied to application code, so the RAG plugin's generation hook
+wrote it through `sudo().db.<list>.update()`. That is an ordinary secured write, so it
+re-ran the list's **whole** hook pipeline carrying the embedding column and nothing else —
+and a list-level `resolveInput` that derives one field from other input, the pattern
+`CLAUDE.md` documents, then recomputed the derived field from values that were not there:
+
+```typescript
+Article: list({
+  fields: {
+    title: text(),
+    body: text(),
+    content: text(),
+    contentEmbedding: embedding({ sourceField: 'content', dimensions: 3 }),
+  },
+  hooks: {
+    // Ran a second time on the plugin's write, with `title` and `body` absent
+    resolveInput: ({ resolvedData }) => ({
+      ...resolvedData,
+      content: [resolvedData.title, resolvedData.body].join(' '),
+    }),
+  },
+})
+```
+
+`create({ data: { title: 'red', body: 'hot' } })` committed `content: 'red hot'` and then
+overwrote it with `' '` — the join of two `undefined`s — and embedded that. No error, no
+log: the row and its vector were both silently wrong. The write threw on every invocation
+before the Write Pipeline landed, so this was only reachable once generation began running.
+
+The plugin's write no longer goes through `context.db`. Core owns it as a single-field
+write, `writePluginOwnedField`, exported from `@opensaas/stack-core/extend` for any plugin
+that injects a field it computes:
+
+```typescript
+import { writePluginOwnedField } from '@opensaas/stack-core/extend'
+
+runtime: (context) => ({
+  [WRITE_VECTOR]: async (listName, id, fieldName, value) =>
+    await writePluginOwnedField({ context, listName, id, fieldName, value }),
+})
+```
+
+It splits the value through the field's own `splitColumns` exactly as the Write Pipeline
+does, issues one scoped `UPDATE`, and runs no hook.
+
+It reaches no field but the one named, and that is enforced rather than asked of the
+caller: the field is resolved against the config the context was built from, so the
+columns written are that field's own and the caller passes no layout. A list or a field
+the config does not declare — including one named for a key it inherits from
+`Object.prototype`, which a bare lookup answers for — is refused by name, as is a context
+carrying no config, as is an `undefined` value, which would otherwise wipe a multi-column
+field and no-op a single-column one, two outcomes for one input. This narrows the escalated
+`db` update it replaces, which could write any column on the row, but it is not a privilege
+boundary: a plugin holding `context.ormHandle` can already write anything, and this refuses
+the mistake rather than the intent.
+
+It takes the `AccessContext` `Plugin.runtime` receives as its first argument; the
+`StackContext` `getContext` returns carries no ORM handle and is refused by name. That
+second argument, `sudo`, is now declared as the `StackContext` it always was — a plugin
+reaching `sudo().db` is unaffected, one reaching `ormHandle` off it was already getting
+`undefined` and now fails to compile. See ADR-0066.
+
+A write refused by name is a wiring defect that fails identically on every row, so the RAG
+plugin's failure log now reports all three refusals — and `WriteCollectionMissingError`
+beside them — as the standing defect they are, rather than telling the reader to retry a
+write that can never succeed.
+
+What changes for an application: a list hook no longer fires a second time when the plugin
+writes a generated column, so one logical change now fires one side effect. Nothing about
+`context.db` changes — an application write runs the pipeline exactly as before, and
+`embedding({ allowManualWrites: true })` still writes through it.

.changeset/silver-moths-gather.md

@@ -30,9 +30,10 @@ pgvector column. Two sections were wrong in ways that produced code that throws:
   `prisma db migrate`.
 - Automatic generation is gated on `autoGenerate` alone; a field carrying it
   with no `sourceField` is a config error that `pnpm generate` throws on, not a
-  silent skip. And the dimension-change recipe no longer tells the reader to
-  re-save rows to regenerate without noting that the plugin's write is inert on
-  this branch (#1124, #1127), so nothing regenerates yet.
+  silent skip. Re-saving a row's source field is what regenerates its embedding
+  after a dimension change, and the recipe now says so: a null vector reads back
+  as no stored embedding at all, so the `sourceHash` gate has nothing to match
+  and does not short-circuit.
 
 `examples/rag-ollama-demo`'s README described `pnpm generate` as writing "the
 contract's own migrations". It writes the Contract module and `prisma.config.ts`,

CONTEXT.md

@@ -83,7 +83,7 @@ The deliberately unsecured client, reached as `context.unsafe` — a name that s
 _Avoid_: raw client, escape hatch, prisma passthrough, context.sql
 
 **ORM handle**:
-The engine's own ORM client, carried as `ormHandle` on the `AccessContext` and narrowed to one model by `ormModel()`. It is what the secured surface's terminals, the Write Pipeline and the access filter actually issue their queries through: the engine applies the Access Filter, Field Visibility and hooks _around_ it, so the handle itself enforces none of them. The Write Pipeline rebinds it alongside `context.db` to whatever client the write runs against; no write currently opens a transaction, so a hook's database work through it does not roll back with the write (#1205). It is not the Unsafe surface — that is `context.unsafe` on the request context, the application's documented bypass, and it is not a member of `AccessContext`. Plugin hooks and `runtime()` factories are handed an `AccessContext`, so the handle is what a plugin author reaches, and it bypasses as much as `unsafe` does.
+The engine's own ORM client, carried as `ormHandle` on the `AccessContext` and narrowed to one model by `ormModel()`. It is what the secured surface's terminals, the Write Pipeline and the access filter actually issue their queries through: the engine applies the Access Filter, Field Visibility and hooks _around_ it, so the handle itself enforces none of them. The Write Pipeline rebinds it alongside `context.db` to whatever client the write runs against, and every write opens a transaction (ADR-0010), so a hook's database work through it rolls back with the write. It is not the Unsafe surface — that is `context.unsafe` on the request context, the application's documented bypass, and it is not a member of `AccessContext`. Plugin hooks and `runtime()` factories are handed an `AccessContext`, so the handle is what a plugin author reaches, and it bypasses as much as `unsafe` does.
 _Avoid_: `context.prisma` (its name before #1207's rename landed), raw client, internal prisma
 
 **Extension pack**:
@@ -110,6 +110,10 @@ _Avoid_: nested connect, link, attach
 The single module that runs the canonical, secured write sequence (operation-level access → hooks → validation → writable-field filtering → relationship resolution → persistence → after-hooks → Field Visibility) for one create/update/delete. Operation-level access is resolved first, outside the transaction (#590) — a denied write short-circuits to `null` before any hook fires. Owns the phase order in one place; per-operation differences (target resolution, which input phases run, the database verb and returned row) are supplied by a per-operation strategy.
 _Avoid_: operation handler, mutation service
 
+**Plugin-owned field write**:
+The write a plugin makes to a field it computes and application code is denied — an embedding, say. It is not an application update: it carries that field's columns alone and completes a write the application already made, whose hooks have already run against the caller's real input, so it runs **no** hook. `writePluginOwnedField` on `@opensaas/stack-core/extend` is the whole of it: resolve the field against the config the context carries, split the value through that field's own `splitColumns`, issue one id-scoped `UPDATE` marked with the engine origin. It reaches no field but the one named — enforced, since the caller supplies no column layout — and a list, a field or a value (`undefined`) it cannot make sense of is refused by name. Driving it through `sudo().db` instead re-runs the list's pipeline over a payload naming one field, which recomputes a derived field from input that is not there and destroys it (ADR-0066).
+_Avoid_: sudo write, plugin update, escalated update
+
 **Hook Pipeline**:
 The module that runs the transform+validate span of a write — list `resolveInput` → field `resolveInput` → list `validate` → field `validate` → built-in field rules → split multi-column fields — owning that order and the `resolvedData` threading through it. It throws a validation error (never silent) when a validate hook reports via `addValidationError` or a built-in field rule fails, and returns the transformed `resolvedData` on success. A multi-column field (e.g. storage `image()`/`file()` in Keystone-parity mode) is validated under its logical field key BEFORE it is split into its per-part physical columns, so an unrecognised value throws instead of being silently split into null/undefined columns (#789). The Write Pipeline delegates this span to it; side-effect hooks (`beforeOperation`/`afterOperation`), access, writable-field filtering, persistence and Field Visibility stay in the Write Pipeline.
 _Avoid_: validation service, input resolver