fix(rag): reconcile Ollama's declared dimensions against the real vector by borisno2 · Pull Request #1541 · OpenSaasAU/stack

borisno2 · 2026-09-13T11:14:54Z

Summary

OllamaEmbeddingProvider.embed() now compares the model's actual returned vector length against the declared dimensions and throws immediately if they differ, naming the model, the declared width, and the real one.
Previously the declared width was never reconciled against reality: a wrong OLLAMA_EMBEDDING_DIMENSIONS (or field dimensions) stamped a false column width at generation time, the index was written with that false width and "looked fine," and the mismatch only surfaced later — an opaque 500 at search time, or in embedBatch, silent zero-padding at the wrong width producing a ragged batch instead of a named failure.
This closes the related loose end noted in the issue comment about DEFAULT_OLLAMA_DIMENSIONS ignoring the model: once a mismatch is caught at embed() time, that gap now produces a named failure instead of ragged batch widths.
Documented (in docs/README.md) that the docs site's committed search index (.embeddings/docs.json) is tied to whichever embedding provider generated it (currently OpenAI, 1536 dimensions), and has to be regenerated with EMBEDDING_PROVIDER=ollama pnpm generate:embeddings before local Ollama-backed search works against it — this is the third item from the issue ("document the requirement where someone setting that variable will see it").

Test plan

Added tests in packages/rag/src/providers/providers.test.ts covering: a dimension mismatch failing with a message naming the model/declared/real widths, a matching width passing through unchanged, and embedBatch failing the whole batch with a named error rather than padding a ragged width.
pnpm build (core, ui, rag)
pnpm test in packages/rag — 255 passed
pnpm lint at repo root
pnpm format / pnpm manypkg fix

Closes #1288

🤖 Generated with Claude Code

https://claude.ai/code/session_01E5rczwjjc8CUDVQ4Z66ZQn

Generated by Claude Code

OllamaEmbeddingProvider.embed() now compares the model's actual returned vector length against the declared `dimensions` and fails immediately, naming the model, the declared width and the real one. Previously the declaration was never checked against reality: a wrong width stamped a false column size at generation time, and the mismatch only surfaced later as an opaque 500 at search time (or, in embedBatch, as a ragged batch of vectors padded to the wrong width). Also documents (docs/README.md) that the docs site's committed search index is tied to whichever provider generated it, and has to be regenerated with EMBEDDING_PROVIDER=ollama before local Ollama search works against it. Closes #1288 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E5rczwjjc8CUDVQ4Z66ZQn

changeset-bot · 2026-09-13T11:14:58Z

🦋 Changeset detected

Latest commit: f4ce487

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 9 packages

Name	Type
@opensaas/stack-rag	Patch
@opensaas/stack-cli	Patch
@opensaas/stack-auth	Patch
@opensaas/stack-core	Patch
@opensaas/stack-storage-s3	Patch
@opensaas/stack-storage-vercel	Patch
@opensaas/stack-storage	Patch
@opensaas/stack-tiptap	Patch
@opensaas/stack-ui	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

vercel · 2026-09-13T11:14:59Z

Deployment failed for project stack-docs with the following error:

Resource is limited - try again in 24 hours (more than 100, code: "api-deployments-free-per-day").

Learn More: https://vercel.com/open-saas?upgradeToPro=build-rate-limit

chatgpt-codex-connector · 2026-09-13T11:14:59Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

borisno2 · 2026-09-13T11:15:52Z

The Vercel status check is failing with Deployment rate limited — retry in 24 hours (more than 100, code: "api-deployments-free-per-day"). That's the docs-site preview deployment hitting Vercel's free-tier daily deployment quota — an account-wide limit unrelated to this PR's diff (which doesn't touch docs/app build output beyond a README edit), and there's no fix to push for it; it clears on its own once the 24-hour window resets. The GitHub Actions checks (test, e2e (postgres), e2e (dev-database), example-gates) are the ones that actually exercise this change and are still running.

Generated by Claude Code

borisno2

Code review (effort: medium-high)

Reviewed the full diff plus the surrounding code the change interacts with (packages/rag/src/config/plugin.ts's autoGenerate hook, packages/rag/src/config/generation-failure.ts's error classification, and OpenAIEmbeddingProvider for comparison). The core fix — comparing response.embedding.length against this.dimensions and throwing a named error before embed() returns — is correct, matches issue #1288's description, and the error message correctly names the model, the declared width, and the actual width. The changeset and docs/README.md update are accurate and consistent with the code change; OpenAIEmbeddingProvider doesn't need an analogous check since it derives dimensions from a static per-model map rather than a user-declared value.

Three findings, in descending order of importance:

1. The new error isn't recognized by the generation-failure reporter (not inline — file untouched by this PR)

packages/rag/src/config/generation-failure.ts — createGenerationFailureReporter

In the one real runtime path that triggers this check (the autoGenerate hook, packages/rag/src/config/plugin.ts ~lines 296-319), the new dimension-mismatch error falls into the generic non-standing-defect bucket, which tells operators to "retry by writing the source field again." A dimension mismatch is a pure config defect — every row will fail identically until dimensions / OLLAMA_EMBEDDING_DIMENSIONS is corrected — so it deserves the same clean, deduplicated, config-focused diagnosis this reporter already gives for other permanent misconfigurations (unregistered provider type, refused write, missing writer), rather than a console.error-per-row wall of duplicate stack traces suggesting a fix that can never work.

2. Dimension check dereferences `response.embedding.length` outside the try/catch (inline comment on `packages/rag/src/providers/ollama.ts:84`)

A malformed 200 response missing embedding now throws a raw, unwrapped TypeError instead of the friendly Ollama embedding generation failed: ... message every other failure path in this method produces — regressing the previous (also wrong, but non-crashing) behavior of silently returning undefined. Suggest bringing the check inside the try, or wrapping it separately, so this failure mode gets the same clear diagnosis.

3. Manual `vi.unstubAllGlobals()` instead of `afterEach` (inline comment on `packages/rag/src/providers/providers.test.ts:281`)

The embedBatch dimension-mismatch test cleans up the fetch stub manually after its assertion, unlike the sibling describe block immediately above it which uses afterEach correctly. If the assertion ever fails, the stub leaks into later tests in the file, turning one clear failure into confusing cascading ones.

Other angles checked

embedBatch error-wrapping: confirmed it still surfaces the inner error's message usefully — embedBatch wraps failures from embed() in a way that preserves the named dimension-mismatch text, matching the PR's stated intent and the added test's assertion.
Test coverage of the new path: the stubbed-fetch tests do exercise the real embed()/embedBatch() code path (they don't stub makeRequest or the check itself), and cover both the mismatch and matching-width cases. Sound apart from finding #3 above.
Comment density (package CLAUDE.md rule): the new 7-line comment above the if check in ollama.ts carries real, non-obvious rationale (why the check lives here rather than elsewhere in the pipeline, and what silently breaks without it) rather than restating the code — consistent with the repo's comment guidance.
No simplification/efficiency issues found beyond the above; the happy-path logic is minimal and correct.

🤖 Generated with Claude Code

https://claude.ai/code/session_01E5rczwjjc8CUDVQ4Z66ZQn

Generated by Claude Code

- Move the dimension check inside embed()'s try block so a malformed response missing `embedding` gets the same wrapped "Ollama embedding generation failed: ..." diagnosis as every other failure path, instead of a raw unwrapped TypeError. - Classify a dimension-mismatch error in generation-failure.ts's reporter as a standing configuration defect (like an unregistered provider type or a refused write), so the autoGenerate hook stops telling operators to "retry" a failure that can never clear on its own, and logs it once per field instead of once per row. - Fix a test-hygiene issue: the new embedBatch test now unstubs `fetch` in `afterEach` instead of manually after its assertion, so a failing assertion can't leak the stub into later tests. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E5rczwjjc8CUDVQ4Z66ZQn

borisno2 · 2026-09-13T11:26:19Z

Pushed f4ce487 addressing all three review findings:

generation-failure.ts classification — added isDimensionMismatch() and a reporter branch so the autoGenerate hook now reports a declared-dimensions mismatch as a standing configuration defect (like an unregistered provider type or a refused write), rather than the generic bucket that tells operators to "retry" a failure that can never clear. Added a test in plugin.test.ts covering it (a misdeclared fake provider that throws the same message shape OllamaEmbeddingProvider does).
Dimension check outside the try/catch — moved inside embed()'s try block, so a malformed response gets the same wrapped Ollama embedding generation failed: ... diagnosis as every other failure path here, instead of a raw TypeError.
Manual vi.unstubAllGlobals() — moved into an afterEach on the test's own nested describe, matching the sibling block above it.

All 256 tests pass (pnpm test in packages/rag), plus pnpm build, pnpm lint, and pnpm format at the repo root.

Generated by Claude Code

github-actions · 2026-09-13T11:44:56Z