fix(rag): say that embedding generation runs, and test it through context.db (#1128, #1272) by borisno2 · Pull Request #1332 · OpenSaasAU/stack

borisno2 · 2026-09-07T22:49:29Z

The RAG work in #1128 was built and documented while the secured write surface could not execute on the Prisma 8 collection. It executes now — #1281 landed the write pipeline — so the delivery's own statements about itself were inverted: for three rounds the docs oversold, and this round they undersold the one thing the spec exists to deliver.

Confirming the premise first

I did not take the write surface from a commit message. On the Test context (PGlite + pgvector, createTestDatabase, a pure-function fake provider) I drove the real surface and then deleted the probe:

Probe	Result
`context.db.Note.create/update/delete`	all execute; readback confirms `title: 'after'`, then the row is gone
`context.db.Article.create({ data: { content: 'red' } })`	commits, then reads back `{"vector":[1,0,0],"metadata":{…,"sourceHash":"2f0x"}}`
update `content` `'red'` → `'blue'`	regenerates: `vector:[0,1,0]`, new `sourceHash` — the re-entry gate does not wrongly short-circuit
`.nearest('contentEmbedding', [1,0,0])` over plugin-written vectors	`red@1.00 reddish@0.80 blue@0.00`

One probe failed first and was informative: Note with no declared operation access returned null from create, which is the documented silent failure, not a broken surface. With access declared it executes.

And the delivery's own the plugin's sudo write reaches the column — written to skip itself by name — now runs and passes. packages/rag reports 456 passed / 0 skipped.

1 — The four claims that said the feature was inert

.changeset/loud-comets-invent.md — the sharpest, because it is pending and publishes as the release note. "Embedding generation does not run in this release… the write throws on every invocation… Semantic search over that field returns nothing… There is no config change that works around it." Replaced with what generation actually does, including the sourceHash short-circuit.
.changeset/silver-moths-gather.md — the dimension-change recipe's "the plugin's write is inert on this branch, so nothing regenerates yet". Probe 2 regenerates on re-save.
packages/rag/CLAUDE.md:363-365 — the generation hook's Known limits.
packages/rag/CLAUDE.md:633-636 — "On the prisma-8 branch that re-save regenerates nothing".

allowManualWrites sweep. Nine sites across .changeset/, packages/rag/CLAUDE.md, docs/content/how-to/rag-advanced.md and docs/content/reference/rag.md. Exactly one offered it as a way around the broken write path — loud-comets-invent.md's "If you need vectors before then, use embedding({ allowManualWrites: true }) and write them yourself" — and it is gone. The other eight already frame it as the deliberate opt-out for an app that maintains its own vectors; unchanged.

Also corrected, same class: the Known limits entry "a nested record is never embedded". ADR-0050 made a nested spelling under a relationship key a NestedRelationInputError, verified against secured-write.test.ts's eight-spelling table — the row that limit describes cannot be created at all, so the hook's warning is a backstop rather than something a write reaches.

2 — The accommodations

isUnportedWriteSurface removed, with the reporter branch it gated. It matched findUnique is not a function / Unknown column "data"; findUnique is now a collection member (context/index.ts:1126), and the branch was shipping a console.error reading EMBEDDING GENERATION IS NOT RUNNING … No config change works around it; track #1127 to a user.
plugin.test.ts's reports a write that cannot execute as the standing defect it is removed — it manufactured new TypeError('model.findUnique is not a function') and was green coverage of an unreachable branch.
Two stale test file headers, the #1124/#1127 comment in plugin.ts, and two further #1127 comments in plugin.test.ts (one of which pinned an assertion "until Spec: MCP and the admin UI on the secured surface #1127 lands").
One stale assertion followed the removal: expect(said).not.toContain('#1127') no longer discriminates against anything.

Sweep, re-run and widened. grep -rn "1124\|#1127" over every .md/.ts/.tsx outside node_modules, dist and specs/ now returns zero hits. The two it previously left — docs/content/how-to/write-a-plugin.md and docs/content/reference/context-api.md, saying a hook's database work is not rolled back — were flagged as unsettleable on a mis-probe: _transactionOpener is set on the internal AccessContext and deliberately omitted from the returned StackContext, so its absence there proved nothing. They are settled and false, and corrected. Keying the sweep on #1205 as well found two more instances the first pass could not see: the pending .changeset/humble-handles-narrow.md and CONTEXT.md's ORM handle entry. A committed test now pins it: an afterOperation writing through ormHandle and then throwing leaves no row, with a control confirming it lands without the throw. No ctx.skip and no #1124/#1127 Known limits block survives in packages/rag or packages/core.

3 — Meeting the PRD's testing decision (#1272)

The PRD says a good test "seeds rows with real vectors through the secured surface". Every re-pointed suite now does.

embedding-write.test.ts — the write denial block drives context.db.Article.create() / .update() instead of hookPipeline. Rows are seeded by writeSource(content), which writes source text and lets the plugin produce the vector. allowManualWrites moved to the DB-backed describe, so it is asserted by reading the columns back rather than by inspecting resolved data. The sudo write's try/catch + ctx.skip is gone.
search.test.ts — seedPalette/seedAxis write source text through context.db; the off-surface seed() helper and its withOrigin('unsafe') import are gone.
core's multi-column-read-write.test.ts — a new DB-backed describe drives the write-access gate through context.db over a field with a real contract (two columns), covering denied update, denied create, granted write, sudo() bypass, an ungated field and a null clear.
plugin.test.ts's embeds a source value a list-level resolveInput produced re-pointed into embedding-write.test.ts as the embedded text is the persisted source, not the caller's input, against a real row.
hookPipeline un-exported from @opensaas/stack-core/internal — nothing depends on it now (core's own tests import it by relative path). The export is unreleased (c0d443fb post-dates the last Version Packages touching packages/core/CHANGELOG.md), so the paragraph announcing it was removed from .changeset/brisk-columns-refuse.md rather than left to publish a lie.

Two rows kept off the surface, deliberately and commented: metadata-present-with-no-vector, which the generation hook cannot produce (it writes both columns or neither).

Mutations — each re-pointed test broken on purpose, restored after

Mutation (production code)	Tests that failed
`splitMultiColumnFields`' denial → `if (!canWrite && false)`	rag `an ordinary create naming the embedding throws`, `an ordinary update naming the embedding throws`, `a create the operation gate admits throws`; core `THROWS when update access is denied`, `THROWS when create access is denied`
the above plus `filterWritableFields`' split-column-owner throw disabled	additionally rag `the write the throw refused never reaches the columns`
generation hook returns before its sudo write	rag `the two columns read back as one stored embedding`, `nearest ranks by the column the field declares`, `the embedded text is the persisted source, not the caller's input`, `the write the plugin owns still lands`; 11 of 13 `search.test.ts` tests — 15 failures, not 14
`allowManualWrites` ignored (field always denies)	rag `allowManualWrites lets the same payload reach the columns`
`splitColumns`' output not merged into the write	core `THROWS when update access is denied`, `writes both per-part columns`, `sudo bypasses the gate`, `a field WITHOUT field-level access writes`
a `null` value dropped instead of split	core `clearing the field with null clears both columns`

The third row is the one that matters for the PRD's decision: under the old withOrigin('unsafe') seeding, breaking generation moved nothing. It now takes down almost the whole search suite. The two survivors there assert the absence of an embedding and a missing id, correctly.

Two mutation results worth recording rather than hiding:

The single-gate mutation leaves the write the throw refused never reaches the columns green, because filterWritableFields is a second, defence-in-depth gate that throws a differently-worded message. The exact-message assertions catch it; that test's substring does not. Disabling both gates kills it.
Writing the Derived fixture surfaced a live data-corruption bug, since fixed rather than guarded — see section 5.

4 — The null dereference (`packages/rag/CLAUDE.md:541`)

const article = await …first(); const queryVector = article.contentEmbedding.vector — TS18047, and a TypeError at runtime whenever the Access Filter denies the row, which is the silent-failure contract the root CLAUDE.md makes a named Critical Pattern.

Harness, falsified before trusting it. A strict tsc project rooted inside packages/rag/ so @opensaas/stack-* resolve through the workspace to the built declarations. The original block reproduces TS18047: 'article' is possibly 'null' exactly; the corrected block is clean; injecting the dereference back into the corrected block reproduces TS18049. Checked against the package's real exported StoredEmbedding rather than a local copy of it — the guard narrows stored.vector to number[], which is what proves the declarations resolve rather than degrading to any.

Sweep. .first() / findUnique( / findFirst( across the whole RAG documentation surface: packages/rag/CLAUDE.md:540 is the only one, so within scope the class has one member and it is fixed.

Beyond scope, the same sweep over every markdown file in the repo found one more in a pending changeset — .changeset/nine-otters-describe.md:32, const article = await …findFirst(); article.body — fixed here and compile-checked in the same harness, because it publishes.

It also found the class is widespread on the docs site, in files this delivery does not own: docs/content/concepts/field-types.md:632, docs/content/reference/fields-api.md:1663 and :1718 all dereference a findUnique result directly, and several more bind without guarding. SECURITY.md:177, docs/content/concepts/access-control.md:167 and docs/content/reference/context-api.md:859 get it right and are the model. Not fixed here — it is a docs-site sweep of its own, and worth a ticket.

5 — The silent data corruption, fixed (ADR-0066)

The Derived fixture was defending itself against a live bug, and the defence hid it. Removed, and fixed at the source.

The corruption, run. The plugin's escalated write was sudo().db.<list>.update() — an ordinary secured write, so it re-ran the list's whole hook pipeline carrying the embedding column and nothing else. A list-level resolveInput deriving one field from other input, the pattern the root CLAUDE.md documents, then recomputed that field from values that were not there.

	`content` after `create({ data: { title: 'red', body: 'hot' } })`	list `afterOperation` calls
Before	`' '` — the join of two `undefined`s, and the embedding is of `' '`	`['after', 'after']`
After	`'red hot'`, vector `[0,0,1]`, `sourceHash: 'hvoym6'`	`['after']`

Silent in both places: no error, and the stored vector is the vector of the destroyed text. Unreachable before #1281 made the write execute.

The mechanism, and why this one. Core now owns the write as a single field: writePluginOwnedField on @opensaas/stack-core/extend splits the value through the field's own splitColumns exactly as the Write Pipeline does, issues one id-scoped UPDATE marked with the engine origin, and runs no hook. It reaches no other field — a narrower capability than the escalated db update it replaces, which could write any column on the row.

The write is not an application update. It carries one column and completes a write the application already made, whose hooks have already run against the caller's real input; running them again over a payload naming one field lies to every one of them. Not only resolveInput: validate sees a mostly-absent record, the side-effect hooks double-fire, and the generation hook's own afterTransaction re-entered itself — which only the stored sourceHash was stopping. Re-entry is now structural rather than guarded.

Carrying the persisted row was considered and rejected, with a concrete reason: a field-level resolveInput that transforms rather than derives would re-transform its own stored output — password()'s hook hashes resolvedData[fieldKey], so handing back the stored hash double-hashes it. That trades one silent corruption for another. The other options and why they lost are in ADR-0066.

One wiring correction fell out. sudo() returns the public StackContext, which omits ormHandle — it reached Plugin.runtime through an as unknown as AccessContext cast, and a write over it found nothing. The write takes the AccessContext runtime receives as its first argument; a context with no handle is refused by name. The RAG plugin's runtime no longer takes sudo at all.

The tests, and the mutation

The fixture's guard is gone; Derived now carries the derivation an application would write. Three rag tests and six core tests pin the contract.

Mutation	Tests that failed
the plugin's write restored to `sudo().db.<list>.update()`	rag `the embedded text is the persisted source, not the caller's input` (`expected ' ' to be 'red hot'`), `the generation write leaves the derived source it did not name intact` (same), `the generation write fires no second afterOperation` (`['after','after']` vs `['after']`)
`writePluginOwnedField` ignores `splitColumns`	core `writes the field's own columns past its write denial`, `leaves a derived field the list's resolveInput owns untouched`, `fires no list hook of its own`, `clearing with null clears every column the field owns`

Both restored after.

Gates

Gate	Result
`pnpm lint`	pass — 0 errors, 2 warnings, both pre-existing (`examples/blog/test-singleton.ts`, `packages/cli/src/migration/generators/migration-generator.ts`)
`pnpm build` (incl. the docs build)	pass — 11/11
`pnpm manypkg fix` / `pnpm format`	clean
`packages/rag`	pass — 460 passed, 0 skipped, 20 files
`packages/core`	pass — 1624 passed / 1 skipped, 83 files
`packages/cli`	pass — 407 passed, 42 files

Core's suite gained 13 tests and rag's skip count went 2 → 0.

Out of scope, untouched

#1310 (storage/tiptap test coverage), #1311 and #1304 (the ollama example's seed script, its lowercase context.db key, the hand-rolled cosine scan, the suite running twice from src/ and dist/), and #1265, #1271, #1278, #1282, #1283, #1288, #1294, #1297, #1298, #1299, #1301, #1303, #1306, #1307, #1308, #1309, #1313, #1316, #1317, #1319, #1320, #1321, #1322. PR #1324 is untouched.

Refs #1128, #1272.

🤖 Generated with Claude Code

…text.db The secured write surface executes on the Prisma 8 collection since #1281, so automatic embedding generation runs end to end. Four places still told users and agents it was inert, one of them a pending changeset that would have published "the write throws on every invocation" as the release note for the feature. - Correct `.changeset/loud-comets-invent.md`, `.changeset/silver-moths-gather.md` and `packages/rag/CLAUDE.md`. `allowManualWrites` is presented only as the deliberate opt-out for an app that maintains its own vectors, never as a way around a broken write path. - Remove `isUnportedWriteSurface` and the reporter branch it gated, which was dead code logging a false "EMBEDDING GENERATION IS NOT RUNNING ... No config change works around it", plus the test that certified the unreachable branch. - Re-point the write-denial and search tests at `context.db` (#1272): rows are seeded by writing source text through the secured surface, so every vector under assertion is the plugin's own output. Same for core's multi-column write-access gate, and for the derived-source case that drove `hookPipeline`. - `hookPipeline` leaves `@opensaas/stack-core/internal`; nothing depends on it. - Guard the possibly-null `.first()` in the "Find Similar Items" sample. Refs #1128, #1272 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

changeset-bot · 2026-09-07T22:49:33Z

🦋 Changeset detected

Latest commit: 7a05f35

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 9 packages

Name	Type
@opensaas/stack-core	Minor
@opensaas/stack-rag	Minor
@opensaas/stack-auth	Minor
@opensaas/stack-cli	Minor
@opensaas/stack-storage	Minor
@opensaas/stack-tiptap	Minor
@opensaas/stack-ui	Minor
@opensaas/stack-storage-s3	Minor
@opensaas/stack-storage-vercel	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

chatgpt-codex-connector · 2026-09-07T22:49:34Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

vercel · 2026-09-07T22:49:35Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
stack-docs	Ready	Preview	Sep 8, 2026 12:43am UTC

github-actions · 2026-09-07T22:59:49Z