ci: typecheck the documented TypeScript blocks, with the shadowing check repaired by borisno2 · Pull Request #1324 · OpenSaasAU/stack

borisno2 · 2026-09-07T21:37:30Z

Split out of #1318 at review request. That PR's documentation fixes were individually sound; this script is repo tooling with no relationship to @opensaas/stack-rag's published surface, it needs its own CI wiring, and its central check needed reviewing as a program. #1318 is now documentation and examples only.

References #1301, which asked for exactly this tool. It does not close it — see "What this does not cover" below.

The script itself ships to nobody; the one edit under packages/* (packages/rag/CLAUDE.md's hybrid-search sample) carries a minor changeset for @opensaas/stack-rag.

What it does

pnpm build && pnpm check:doc-ts-blocks extracts every fenced typescript/ts/tsx block from the six paths named in scripts/doc-blocks/files.txt (five markdown documents plus a changeset) and runs three checks per block.

Compile. The block is written into a scratch project under packages/rag (so vitest and @types/node resolve as they do anywhere in the repo), alongside the two preludes, and type-checked under strict. @opensaas/* resolves through a scratch node_modules/@opensaas/* of symlinks to packages/*, so each package's own exports map decides what a subpath means: one it exports resolves to its built .d.ts, one it does not export fails TS2307 for the checker exactly as it fails for a reader (the Docs samples import from subpaths the packages do not export, and nothing checks them #1301 class).
Imports. Prelude bindings are typeof import(...) of the real export, so a nested excerpt still gets the shipped signature. Each block is then compiled a second time without them, and a block that carries imports of its own yet still needs a prelude name is failed for not importing it.
Shadowing. A block that exports — or declares at module scope, under any spelling the checker sees as an export — a type whose name a package also exports is compared against the package's own. The comparison is a member-by-member diff computed from the checker (below).

The prelude's context is parameterised. scripts/doc-blocks/prelude.d.ts declares a db for the three lists the listed prose invents — Article, Document, DocumentChunk — with rows carrying every field a listed block reads or writes, over a query surface mirroring the generated ListOps (composable reads and their terminals, select narrowing the row, nearest accepting only the row's vector columns, the three writes). A wrong-cased list, an unknown list, a misspelt vector column, a misspelt row field and an unchecked null are all compile errors now. Every helper type lives inside a declare namespace, so no bare name a block might write resolves to the prelude by accident — only context and getContext are reachable.

A block that cannot stand alone carries an entry in scripts/doc-blocks/fragments.json, and the entry names what it excuses: { "excuses": ["provider", "TS2307 'cohere-ai'"], "reason": … } covers exactly those diagnostics — an unresolved bare name the prose supplies, or a TSnnnn 'token' code-and-token pair — and every other diagnostic in the block still fails it. A block that is not a statement list at all (an object-literal body, a ... elision) instead carries the distinct, greppable { "whole": "reason" }, which excuses every compile diagnostic; the summary counts those separately. The entry is a claim about the block, so it is checked as one: an excuse matching no diagnostic is STALE, so is a whole entry on a block that now compiles or one this check never compiles at all, an entry matching no block is ORPHAN, and no entry ever excuses a shadowing failure or a missing import.

Current result on the merged tree:

97 blocks: 71 compile, 0 not compiled; 26 carry a fragment entry (10 whole-block, 0 stale). Redeclared exported names: 3 compared against the package, 0 not compared.

The comparison

Neither built-in relation answers the question on its own. Assignability in both directions cannot see an optional member appear or disappear — ChunkingConfig has no required members, so any rearrangement of its surface passed. The identity relation sees that, but compares type flags before members, so an intersection is never identical to its flattened spelling — and the core field types ship as intersections, so the ordinary way of documenting one would have failed.

The decision is therefore a diff: member presence, optionality, readonly — declared, or introduced by a mapped type such as Readonly<>, read off the symbol's check flags — any against something narrower, index signatures, call-signature arity, and each member's type, recursing into object-typed members whether required or optional (T | undefined and T | null are entered after their nullability is compared on its own) and into non-generic signatures, and falling back to assignability in both directions at the leaves, for unions of more than one object type, and for generic signatures. Identity is kept only as a fast path — when the checker holds the two to be the same type there is nothing to diff. The failure names the difference:

FAIL   scripts/doc-blocks/self-test/fixture.md:317 (redeclares ChunkingOptions)
         shadows ChunkingOptions from @opensaas/stack-rag/runtime — not the type the package
         declares — the block declares `minTokens`, the package does not

A name two specifiers export as different types (ChunkingStrategy from @opensaas/stack-rag and from @opensaas/stack-rag/runtime) is held as two candidates. A block is compared against the candidates it imports from, or against every candidate when it imports from none, and fails only when it matches none.

Type parameters are compared rather than worked around. A parameter the block declares past the package's arity is a difference, and so is a default on one side and none on the other — checked by a second instantiation at the shared required arity. An unused parameter is filled with never whatever its position and whatever its constraint, so no no-op edit can retire the comparison; a used one takes a fresh opaque type, and a used one carrying a real constraint is the one case that bails.

A redeclaration the parser cannot fully read — a ... elision inside its body — is compared one way, over the members the parser recovered: an invented or mistyped member still fails, a member the block elided does not. A whole fragment entry therefore cannot buy an invented sibling a free pass.

The self-test

scripts/doc-blocks/self-test/fixture.md holds one known-bad and one known-good block for each shape the reviews found. Each marker says exactly what the checker must report:

 — reported FAIL.
 — clean, and redeclares no shipped name.
 — clean, and every shipped name it redeclares was compared against the package; a bail is a mismatch, not a pass.
 — clean, with at least one comparison reported NOT COMPARED.
 — every compile diagnostic is excused by the entry on the marker, and every redeclared name was compared.

A marker carries its fragment entry as excuses="a, b" or whole="reason". node scripts/check-doc-typescript-blocks.mjs --self-test runs the checker over the fixture and exits non-zero on any mismatch, so every known-good block is shown to have been compared rather than assumed. CI runs it before the real documents.

self-test: 55 blocks — expected 29 fail, 9 pass, 12 pass compared,
1 pass not-compared, 4 excused; Redeclared exported names: 32 compared
against the package, 1 not compared. 0 mismatch(es).

The shapes: a non-compiling block; an unexported subpath (the good twin imports @opensaas/stack-auth/server, a real subpath the old hand list could not resolve); a wrong-cased context.db key; an unknown list; an unchecked null dereference; a misspelt vector column; a self-contained block leaning on the prelude for an import; and the four the last review reproduced against the previous claims —

(a) a phantom type parameter carrying a constraint (<X extends string = string>) — previously tripped the constraint bail and passed; a phantom parameter is now filled with never whatever its constraint, so the invented member is reported;
(b) declare module '@opensaas/stack-rag/runtime' { interface ChunkingOptions { minTokens?: number } } — previously only NOT COMPARED; any declare module '@opensaas/…' in a block is a FAIL;
(c) import type { SearchResult as X } from '@opensaas/stack-rag'; export type { X as ChunkingConfig } — previously no output at all; exports are enumerated through checker.getExportsOfModule + getAliasedSymbol, so every spelling is seen;
(d) a fragment entry plus strategy?: NotReal inside the redeclaration — previously the whole comparison bailed; an unresolved member is skipped and reported by name and its siblings are still compared, so the invented minTokens? beside it is a FAIL. The comparison bails only when the type itself does not resolve (a heritage clause, an intersection operand);

— plus export { type Wrong as SearchResult }, an unexported module-scope redeclaration, a readonly member and a Readonly<> mapped type, an any member, an invented member hidden inside an intersection (with its faithful intersection twin passing), and the two-specifier ChunkingStrategy case in four spellings.

The second review added: an invented member under an optional object member, in two spellings — a flat 40-member EmbeddingField copy with bogus?: number under its optional db, and TextChunk with metadata?: Record<string, unknown> & { bogus?: number } — each beside the exact copy that must still pass; a constrained parameter the block does not use within the package's arity; a type-parameter default that differs from the package's, and one dropped entirely; a ... elision inside a redeclared shipped name, with and without an invented sibling; and a fragment entry proving it excuses only the diagnostics it names.

Hygiene

The scratch directory is mkdtempSync(packages/rag/.doc-blocks-check-XXXXXX), so two overlapping runs no longer crash each other. The pattern is in .gitignore and in eslint.config.js ignores (ESLint 9 flat config does not read .gitignore).
--json exits non-zero on any failure and its payload carries ok, the summary, every block's verdict, orphans and missingFiles.
A fence info string may carry attributes (```ts title="x").
A file-less diagnostic — a TS2688/TS6046-class option error — is a tooling failure (exit 2), not something dropped with the diagnostics of other files.
The guard checks every exports[*].types of every packages/*/package.json exists, so the shadow targets and the resolvable modules are the same list, derived rather than hand-kept.

CI

The two sibling checks (check:adr-duplicates, check:prisma-error-codes) already run in test.yml. This one runs in the same job after a full package build (turbo-cached; the test steps depend on the same task), as two steps: --self-test, then the real run.

It is skipped for PRs into main. The fixtures describe this branch's documents: files.txt names them and fragments.json keys entries by file:line. main's RAG guidance is a different text at different lines, so every entry would report as orphaned. The gate and the one instruction for flipping it live together in test.yml; the script's Known limits and the warning annotation point there.

What this does not cover

Stated plainly, because the point of the tool is to stop certifying more than it measures. The full list is the Known limits block in the script; the load-bearing ones:

Coverage is the file list, not the tree. Six paths. A new doc, or context-api.md with the four bad imports Docs samples import from subpaths the packages do not export, and nothing checks them #1301 actually reports, is never compiled, and nothing here detects that a file was added. That is why this references Docs samples import from subpaths the packages do not export, and nothing checks them #1301 rather than closing it.
Fragment keys are file:line. The orphan check catches a key that has drifted off every block; a key that drifts onto a different block's first line still excuses that block instead.
context.db is hand-written, not generated. Three lists, with rows carrying the fields the listed prose uses. where and orderBy take the package's untyped vocabulary rather than the list's own columns, so a misspelt key in either is not a compile error; include, distinct, distinctOn and cursor are not modelled, nor are select/include on a write. create takes a fully partial data, because CreateInput requires a member exactly where the contract shows a non-nullable column with no default and no listed page declares one — validation: { isRequired: true } is an application-layer check and leaves the column nullable. A documented create omitting a field a reader's own stricter list requires is therefore not a compile error here. The generated SecuredList cannot be used here because it is instantiated from the emitted Prisma contract, which nothing but the generator can write.
The comparison sees types, not meaning. It does not see a @default that no longer matches the code, a member whose name is right and whose meaning has changed, or an option documented as accepting a range the package narrows only at runtime. A generic member signature is compared by arity and the outer assignability only. Two declarations can compare equal and still document the package wrongly.
A generic is sometimes out of reach. A comparison bails when it would have to supply a genuinely constrained parameter the block uses, or when the package requires a type argument the block does not declare. Every bail is NOT COMPARED on stderr, excluded from the compared tally, and never a pass — a shape that cannot be compared soundly is reported, never failed.
A type that does not resolve cannot be compared. A member declared in terms of an unresolved name is skipped by name and its siblings compared; a type whose heritage or operand does not resolve is NOT COMPARED.
The package's side is trusted to resolve. Every program here sets skipLibCheck, so a packages/*/dist that is present but internally broken degrades an export to an error type with no diagnostic, and a wrong block then compares clean. The guard checks only that the entry declaration files exist. Tracked as The doc-block check silently weakens against a partial build instead of failing #1350.
Blocks are compiled, not run. Nothing here says a documented call does what the prose claims.
tsx fences are not compiled — no jsx option, no React in the scratch project. Reported UNCHECKED, never a failure.
Extraction is textual. The closing fence must match the opening indent exactly; four-backtick fences and fences with trailing whitespace are handled wrongly.
A run killed outright leaves a scratch directory. SIGINT and SIGTERM remove it and re-raise; SIGKILL cannot, which is what the .gitignore entry is for.

Follow-ups

Raised in review and deliberately not in this PR:

Extend files.txt to the rest of the docs tree — six paths is the coverage limit, and closing Docs samples import from subpaths the packages do not export, and nothing checks them #1301 needs the tree, not a list. Out of scope here: the fragment keys and the fixture describe these documents, so widening the list is a re-keying exercise with its own review surface.
Compile tsx fences — needs a jsx option and React resolvable from the scratch project, i.e. a second scratch layout. Reported UNCHECKED today, never a false pass.
Reuse the TypeScript program across blocks — the run takes ~33 s because every block builds a fresh program. Caching changes the isolation each block currently gets, so it wants its own change.
Key fragment entries by content rather than file:line — a key that drifts onto a different block's first line still excuses that block. A content hash would close it, at the cost of an entry that must be regenerated on any edit.
skipLibCheck hides a broken packages/*/dist — tracked separately as The doc-block check silently weakens against a partial build instead of failing #1350.

Checks

pnpm lint 0 errors (2 pre-existing warnings elsewhere in the tree) · pnpm format clean · pnpm build (all packages) green · --self-test exit 0, 0 mismatches · pnpm check:doc-ts-blocks exit 0 on the tree merged with current prisma-8 (post-#1370), and from a foreign cwd · kill -INT mid-run exits 130 leaving nothing under packages/rag/.

🤖 Generated with Claude Code

… they redeclare References #1301, which asked for this tool. It does not close it: the check runs over the nine files named in scripts/doc-blocks/files.txt, not all of docs/content/, and the four `@opensaas/stack-core/context` imports that issue reports sit in context-api.md, which is not on the list. scripts/check-doc-typescript-blocks.mjs extracts every fenced TypeScript block from the listed files and compiles each one under `strict` against this branch's own built declarations, with @opensaas/* mapped to packages/*/dist. A block that cannot stand alone carries a reason in scripts/doc-blocks/fragments.json, and the reason is treated as a claim: an entry whose block compiles is STALE, an entry matching no block is ORPHAN. The check landed inside #1318 as supporting evidence for a docs sweep. It is repo tooling with no relationship to any published package, it needs CI wiring, and its central check was defeated, so it belongs here on its own. The shadowing check saw 15 of 236 exported type names — 4 of them on a tree where packages/auth/dist had not been built, which is the run the review reported. `collectExportedTypeNames` filtered `getExportsOfModule` by Interface|TypeAlias|Class, and every barrel in this repo re-exports with `export type { X } from './y.js'`, whose symbols carry only SymbolFlags.Alias. `SearchResult`, `ChunkingConfig`, `NearestMatch` and `StackContext` were among the ~220 missed, so a block could document any of them as the opposite of what ships and be certified as compiling. Exports are now resolved through the alias before the flag test. Two further ways the check reported a number that was not about the docs: - The shadow probe filtered diagnostics to probe.ts, so a redeclaration nested in a function never reached module scope, the probe's two assignability checks compared against an error type, and the block was counted as agreeing with a type it was never compared to. Diagnostics on the appended re-export now fail the probe, and a block that exports the name itself is not given a duplicate one. - fragments.json was checked for staleness one way only. A key orphaned by any edit above its block was ignored, so the file could only grow. The summary line also counted every non-clean block as a classified fragment, which reported 28 classifications against 27 entries whenever one block failed. It now counts entries. A missing packages/*/dist aborts with exit 2 instead of blaming a doc block for an unresolved import — that is the state the review's own run was in. Wired into the test workflow beside check:adr-duplicates and check:prisma-error-codes, with the package build it needs. It is skipped for PRs into main, where the fixtures do not describe the documents. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

vercel · 2026-09-07T21:37:35Z

Deployment failed for project stack-docs with the following error:

Resource is limited - try again in 24 hours (more than 100, code: "api-deployments-free-per-day").

Learn More: https://vercel.com/open-saas?upgradeToPro=build-rate-limit

chatgpt-codex-connector · 2026-09-07T21:37:36Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

changeset-bot · 2026-09-07T21:37:36Z

🦋 Changeset detected

Latest commit: 8858878

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 9 packages

Name	Type
@opensaas/stack-rag	Minor
@opensaas/stack-cli	Minor
@opensaas/stack-auth	Minor
@opensaas/stack-core	Minor
@opensaas/stack-storage-s3	Minor
@opensaas/stack-storage-vercel	Minor
@opensaas/stack-storage	Minor
@opensaas/stack-tiptap	Minor
@opensaas/stack-ui	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

The check's fixtures are keyed `file:line`, and #1318 (de80685) and #1332 (3138401) rewrote the documents it reads. Merging origin/prisma-8 shifted every line in rag-advanced.md and packages/rag/CLAUDE.md, and brought in .changeset/swift-pandas-listen.md, which the check had been skipping because the file did not yet exist on this branch. The orphan arm did its job: 14 of the 27 keys matched no block and were reported, so no shifted key silently excused a different block. The 13 that still matched were re-read against their current diagnostics rather than assumed, and each still describes the block it lands on. Every entry here is re-derived from the block's actual diagnostic. Two classifications crossed over in rag-advanced.md and would have been wrong had the keys been moved mechanically: the block now at 523 is one entry of a `lists:` object, not the `lists:` property its old key described, and the block now at 880 is the `lists:` property, not the `fields:` one. Three more reasons named things that are no longer true — the plugin hook at 66 closes over `writePluginOwnedField` and `embeddingWriter`, not `writeUnderSudo`; the `stored.vector` failure at CLAUDE.md:546 is a `{}` narrowed from `unknown`, not `unknown` itself; and the `NearestMatch` mismatch at 910 is stated as the diagnostic reports it. The block redeclaring `EmbeddingProvider` with an optional `embedBatch` no longer fails: #1318 made it required, so it now agrees with the shipped type and needs no entry. 97 blocks: 70 compile, 27 classified fragments, 2 redeclare an exported type and are checked against it. No failures, none stale, none orphaned. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

borisno2 · 2026-09-08T02:42:32Z

Re-baselined against the merged documents

Updated by merging origin/prisma-8 (8dc25a0b) rather than rebasing — the branch is checked out in another worktree and the PR has CI history against ca61a6ff, so a merge keeps both intact. It merged cleanly; no conflicts.

de80685a (#1318) and 31384011 (#1332) rewrote the documents this check reads, and the merge also brought in .changeset/swift-pandas-listen.md, which the check had been skipping entirely because the file did not yet exist on this branch. Blocks went 95 → 97 (the changeset contributes 2).

The orphan check fired — on 14 of 27 keys

This is the result that mattered. Every key in rag-advanced.md, packages/rag/CLAUDE.md and rag-ollama-demo/README.md had drifted off its block, and all 14 were reported:

ORPHAN docs/content/how-to/rag-advanced.md:64,206,268,289,495,659,797,852,882
ORPHAN examples/rag-ollama-demo/README.md:441
ORPHAN packages/rag/CLAUDE.md:510,525,540,554

Not one shifted key silently landed on a different block. The 13 keys that did still match were re-read against their current diagnostics rather than assumed correct — each still describes the block it lands on.

Counts

The honest comparison is the post-merge run before re-baselining against the run after it — I did not re-run the check against the pre-merge documents, so no pre-merge compile count is quoted here.

	after merge, old fixtures	after re-baseline
blocks extracted	97	97
compile clean	70	70
fixture entries matching a block	13 of 27	27 of 27
orphaned keys	14	0
blocks failing with no classification	14	0
stale	0	0
redeclare an exported type (checked against it)	2	2
exit code	1	0

Entry count is unchanged at 27, but every entry was re-derived from the block's actual diagnostic — none carried forward on the strength of still sounding plausible.

Block count rose 95 → 97 because .changeset/swift-pandas-listen.md arrived with the merge; per-file counts are otherwise identical (rag-advanced 32, reference/rag 22, ollama README 9, rag CLAUDE 17, rag README 15).

Classifications that changed

Two crossed over. Moving the keys mechanically would have put a true-sounding but wrong reason on each:

key	old reason	actual
`rag-advanced.md:523` (was 495)	"a bare `lists:` object-literal property"	it is one entry of a `lists:` object — a bare `DocumentChunk:` property
`rag-advanced.md:880` (was 852)	"a bare `fields:` object-literal property"	it is the bare `lists:` property

Three named things that are no longer true:

rag-advanced.md:66 (was 64) — the old reason listed writeUnderSudo as a closure binding. fix(rag): make the RAG docs' own examples compile, and load the chatbot config #1318 replaced that mechanism: the block now closes over WRITE_EMBEDDING, writePluginOwnedField and embeddingWriter. Reason rewritten, and it now also records the actual parse failure (runtime: and afterTransaction: are bare object-literal properties).
packages/rag/CLAUDE.md:546 (was 540) — old reason said article.contentEmbedding is unknown. The diagnostic is TS2339: Property 'vector' does not exist on type '{}' — unknown narrowed to {} by the stored ? guard. Stated as the compiler reports it.
rag-advanced.md:910 (was 882) — restated as the diagnostic reads: nearest() on the un-parameterised StackContext yields NearestMatch<OrmRow>, which does not fit the block's own NearestMatch<{ id: string }>[].

One entry retired itself. The EmbeddingProvider block (now rag-advanced.md:218) that redeclared embedBatch as optional against a required one — the failure this PR's description called out — no longer fails. #1318 made it required in the doc, so it now agrees with the shipped type and needs no entry at all. It is one of the 2 blocks the shadowing check compares.

The remaining 21 entries re-derived to the same reason they already carried, at shifted keys.

No documentation defects this round

All 27 non-clean blocks are genuine fragments — third-party SDKs this repo does not install (cohere-ai, @huggingface/inference), bare object-literal properties, alternative snippets sharing a fence and rebinding the same name, deliberate // ... lists elisions, and names the prose supplies. The defects that were real were already fixed by #1318 and #1332.

Harness falsified, not trusted

Shadowing defeat reproduced. Appended export type SearchResult = { totallyWrong: boolean } to packages/rag/README.md → FAIL ... (redeclares SearchResult), caught in both directions (TS2739 missing item, score; TS2741 missing totallyWrong), exit 1. Removed → clean, exit 0.
A fragment entry cannot excuse shadowing. Added a bogus excuse at that same key — still failed, exit 1.
STALE arm — classified a block that compiles → reported STALE, exit 1.
ORPHAN arm — besides the 14 real ones, a synthetic key at a line with no block → reported ORPHAN, exit 1.

CI wiring

Unchanged and still correct. .github/workflows/test.yml has not moved on prisma-8 since the merge base, so the two steps land exactly where they were reviewed — the turbo run build --filter='./packages/*' step before pnpm check:doc-ts-blocks, both gated if: github.base_ref != 'main', matching the surrounding convention for integration-branch steps.

Verification

pnpm lint — pass (0 errors; 2 pre-existing warnings unrelated to this change)
pnpm format:check — pass
pnpm build (packages + docs) — pass
pnpm check:doc-ts-blocks — pass, exit 0

97 blocks: 70 compile, 27 classified fragments, 2 redeclare an exported type (checked against it).

No changeset: nothing under packages/*/src was touched.

borisno2

Review — REQUEST CHANGES

Covers the two most recent commits on claude/doc-block-typecheck:

8dc25a0b — Merge remote-tracking branch 'origin/prisma-8' into HEAD
549a59ca — ci: re-baseline the doc-block fixtures against the merged documents

Posted as a Comment, not a formal REQUEST_CHANGES event: the reviewing identity is borisno2, the PR's own author, and GitHub refuses a review event on your own PR. Treat the verdict line above as the verdict.

Everything was re-run against 549a59ca with pnpm install && turbo run build --filter='./packages/*'. Baseline reproduces: 97 blocks: 70 compile, 27 classified fragments, 2 redeclare an exported type, exit 0.

What checks out

I audited all six verification asks and they hold.

All 27 fixture entries are honest. I dumped --json and read every entry's reason against the block's actual diagnostics. Every one of the 27 blocks genuinely fails to compile standalone, and every reason names the real first cause. Nothing in the "contains an elision" family survives as a false claim: reference/rag.md:396 really does write openaiEmbeddings({/* ... */}) literally, and README.md:136 / rag-ollama-demo/README.md:151 really do write // ... lists. No block that should compile is being excused, and unexcused correctly means no entry is masking an unimported name (none of the 27 has one).

Both crossovers confirmed, no third. rag-advanced.md:523 opens DocumentChunk: list({ — one entry of a lists: object, so the old 495 reason ("a bare lists: property") would have been wrong on it. rag-advanced.md:880 opens // opensaas.config.ts / lists: { — so the old 852 reason ("a bare fields: property") would have been wrong there too. Mapping the nine old rag-advanced keys to the nine new ones in order and checking each block textually, those are the only two that moved onto a differently-shaped block.

The block that stopped needing an entry genuinely compiles. rag-advanced.md:218 declares EmbeddingProvider with a required embedBatch(texts: string[]): Promise<number[][]>; the run reports errors: 0, shadowErrors: 0, no fragment, and it is one of the two entries in the shadow tally. It is still extracted and still compared — not dropped.

The coverage gap is real and the file list is otherwise complete. .changeset/swift-pandas-listen.md does not exist at ca61a6ff and does at 8dc25a0b. Re-running extraction against the ca61a6ff tree gives exactly 95 blocks with that file skipped; the merged tree gives 97. All six listed paths exist today and all six yield blocks.

Both fixture arms fire. Restoring 8dc25a0b's fragments.json onto the merged tree produces exactly 14 ORPHAN lines and exit 1. Adding an invented entry for the compiling block at rag-advanced.md:218 produces STALE … "invented excuse", exit 1.

The merge is clean. git diff --stat 6143ea29 8dc25a0b is 8 files, 531 insertions, 0 deletions — every one of them this PR's own additions (scripts/check-doc-typescript-blocks.mjs, scripts/doc-blocks/*, the workflow steps, .gitignore, package.json). The merge result is byte-identical to prisma-8 everywhere else, so nothing from de80685a or 31384011 was reverted or half-applied.

No any anywhere in the script or the preludes (two textual hits, both in comments).

Blockers

B1 — The merge reopened the defeat this PR exists to close. `type X<T = default>` escapes the shadowing check entirely.

DECLARATION (scripts/check-doc-typescript-blocks.mjs:148) ends its type-alias branch with (?:<[^=]*>)?\s*=. A type-parameter list containing a default has an = inside the angle brackets, so [^=]* cannot cross it and the whole alternative fails to match.

#1318 — brought in by 8dc25a0b — fixed reference/rag.md's SearchResult sketch by changing SearchResult<T> to SearchResult<T = unknown>. That fix removed the block from the shadowing check.

type SearchResult<T = unknown> = {   →  NO MATCH
type SearchResult<T> = {             →  SearchResult
interface SearchResult<T = unknown> {→  SearchResult   (the interface branch is fine)

The two blocks in the "2 redeclare an exported type" tally are EmbeddingProvider and StoredEmbedding. SearchResult is not among them, which makes the PR body's account ("the shadow count on the merged tree is 2 rather than 1" because the check caught the RateLimiter and SearchResult defects) misleading about the current tree — the count is 2 for unrelated reasons and SearchResult is now uncovered.

Reproduced the exact defeat from the PR description, on the merged tree, line-count preserving so nothing else shifts. Editing docs/content/reference/rag.md:582 to:

type SearchResult<T = unknown> = {
  totallyWrong: boolean
  alsoWrong: T
}

gives:

97 blocks: 70 compile, 27 classified fragments, 2 redeclare an exported type (checked against it).
EXIT=0

Counted as compiling, shadow tally unmoved, clean exit — the same certification failure, on the same type name, in the same file, that this PR was split out to fix. The interface branch matches generics fine; only the type alias branch has the hole. Fix and re-run before merge; the shadow tally should move to 3.

B2 — A doc block that names any generic exported type is an unfixable CI failure, and is reported as "checked against it" when nothing was compared.

compileShadowProbe (:239) emits declare const documented: Documented / declare const shipped: Shipped with no type arguments. 41 of the 289 type names in the map are generics with at least one required parameter — including Row, ListConfig, QueryResult, ColumnFilter, ListWhere, CreateInput, UpdateInput, SecuredList.

Appending this entirely innocuous block to packages/rag/README.md:

type Row = { id: string; title: string }
const r: Row = { id: 'a', title: 'b' }
void r

produces:

FAIL   packages/rag/README.md:502 (redeclares Row)
         shadows Row from @opensaas/stack-core — TS2314: Generic type 'Row' requires 3 type argument(s).
         shadows Row from @opensaas/stack-core — TS2314: Generic type 'Row' requires 3 type argument(s).
98 blocks: 70 compile, 27 classified fragments, 3 redeclare an exported type (checked against it).

Two things go wrong. The block is a hard failure whose only remedy is renaming a type inside prose (unexcused at :333 correctly refuses to let fragments.json excuse a shadow error, so there is no escape hatch) — and Row is a very plausible name for a doc block in search guidance to introduce. And the summary counts it in "3 redeclare an exported type (checked against it)" when no structural comparison happened at all. That is the same over-claim the F3 repair was about, reached through TS2314 instead of TS2661.

At minimum the probe needs to skip (or parameterise) generics and the summary must stop claiming a comparison that the probe errored out of. Note the merge widened this: prisma-8 grew the exported-type map from the 236 the PR body cites to 289 on this tree.

B3 — `existsSync` silently drops a listed file and its fragment keys. This PR is the proof.

:73 filters out any listed path that does not exist, and listedFiles (:75) is built from the already-filtered list, so :351's orphan check also skips that file's keys. A missing file therefore produces: fewer blocks, no orphans, exit 0, no signal of any kind.

This is not hypothetical — it is exactly what happened here. The changeset was listed in files.txt from ca61a6ff onward and was skipped for the entire life of the branch. The check said 95 blocks, exit 0, and certified nothing about it. It was only discovered because a merge happened to bring the file in. The same failure with docs/content/reference/rag.md renamed would silently retire 22 blocks and 4 fragment entries with a green run.

Scope the exemption to .changeset/* (which is the only path with a legitimate disappearance) and make every other missing listed path as loud as an orphaned key.

Follow-ups

.github/workflows/test.yml:111,115 — the check permanently self-disables the moment prisma-8 reaches main. Both steps are gated if: github.base_ref != 'main', and the workflow only triggers on pull_request into [main, prisma-8]. Unlike the sibling test steps at :118/:128 and :134/:140, which pair each != 'main' step with a == 'main' counterpart, these two have none. After the integration branch lands, every subsequent PR has base_ref == 'main' and the check stops running silently, forever, with nothing failing to prompt the flip. The PR body says "the check reaches main when prisma-8 does" but the condition as written guarantees the opposite. Please file the flip as a tracked follow-up so it isn't carried only in a PR description.
:108 — tsx fences are extracted but cannot compile. The fence regex accepts tsx; compileBlock writes block.ts and compilerOptions sets no jsx. Nothing in the six listed files today, but packages/ui docs are the obvious growth path for files.txt, and the only repair available would be a fragments.json entry that misdescribes the cause.
:191 — a doc block can trip the prelude guard and abort the whole run with no attribution. The guard assumes any diagnostic in a prelude file means a broken build. A block containing declare global { const context: number } puts TS2451: Cannot redeclare block-scoped variable 'context' in prelude.ts, so the script prints prelude prelude.ts: … and exit(2) — no file:line, remaining blocks unrun.
:351 — the orphan check has a second hole. listedFiles.has(key.slice(0, key.lastIndexOf(':'))) silently ignores a key naming a file not in files.txt — a typo, a path left after a rename, or a key with no colon at all (lastIndexOf returns -1, slice(0, -1) truncates the path). That is F4's "the file only ever grows" reached by a different route.
The Known limits block is now materially incomplete for a tool whose stated purpose is to stop certifying more than it measures. It says nothing about B1 (generic-default aliases are not shadow-checked), B2 (generic names cannot be probed), or B3 (a missing listed file is skipped silently along with its keys). The B3 case in particular is currently described as a feature in files.txt's own header comment.
Two PR-body numbers are off. "the nine markdown files named in scripts/doc-blocks/files.txt" — files.txt lists six paths, five markdown documents plus a changeset. ("Nine" is right for packageEntries, and the script's limits block uses it correctly.) And "15 of 236" names — the merged tree has 289.
Two fixture reasons are terse to the point of being partial. reference/rag.md:266 and packages/rag/CLAUDE.md:143 are both two-part sketches — a ragPlugin({...}) call followed by a // In fields / fields: {...} property — and both entries describe only the tail. The reason names the real first diagnostic so neither is wrong, but "a bare fields: object-literal property" reads as though the whole block is one, which is what the crossover cases show is easy to get away with.

Verdict

The fixture re-baseline itself is good work and I could not fault it: 27 entries re-derived from real diagnostics, two genuine crossovers caught that a mechanical key move would have got wrong, no third missed, the retired entry genuinely compiling, and both fixture arms demonstrated firing. The merge is clean.

But B1 puts the tree back in the state the PR was split out to fix — SearchResult can be documented as the opposite of what ships and the check exits 0 — and B3 is a silent-skip that already ate a whole file for the life of this branch. Neither is a large change. REQUEST CHANGES; happy to re-review on a push.

The fixtures were sound; the checker was not. Each of these let it report a clean run over something it had not checked. B1 — a type-parameter default escaped the shadowing check. The alias branch ended `(?:<[^=]*>)?\s*=`, and `[^=]*` cannot cross the `=` in `type SearchResult<T = unknown>`. #1318 made exactly that edit, which silently removed the block from the check. The parameter list now tolerates a default and one level of nesting. The tally moves 2 -> 3; no fixture was re-keyed. B2 — a generic export was an unfixable failure reported as a comparison. The probe emitted a bare `declare const documented: Documented`, so a block declaring `type Row = { id: string }` against the exported `Row<C, R, K>` got TS2314 twice, compared nothing, could not be excused, and was still counted as "checked against it". The probe now picks a type-argument arity both declarations accept and fills it with fresh opaque parameters; where no such arity exists, or the parameters carry constraints it cannot fill, the name is reported as NOT COMPARED and left out of the tally. A comparable generic is still compared in both directions. A redeclaration that never reaches module scope still fails, and is no longer counted as compared either. B3 — a listed file that did not exist was dropped along with its fixture keys, for exit 0 and no signal. This PR was the proof: a changeset was listed and missing for the branch's whole life. Only `.changeset/*` is now exempt, and that exemption covers its keys too; any other unreadable listed path fails by name with its keys reported as orphans. The orphan arm also stops ignoring keys that name an unlisted file, or carry no `file:line` shape at all. Also: `tsx` fences are reported as unchecked rather than given a misleading `TS1161`; the prelude guard names the block it aborted on; and the `main` gate gains the counterpart the sibling test steps have — it cannot run there, so it now says so in the job log on every PR into main instead of skipping in silence. Flipping it is tracked on #1301, named at the gate. `Known limits` covers all of the above. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

vercel · 2026-09-08T03:56:51Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
stack-docs	Ready	Preview	Sep 9, 2026 3:47am UTC

borisno2 · 2026-09-08T03:57:56Z

Blockers addressed — `6db546dd`

Every defect was reproduced on 549a59ca first, then fixed, then shown caught. The fixtures were not touched: fragments.json is byte-identical and files.txt changed only its header comment. No entry was re-keyed and no classification moved.

Baseline before and after, side by side:

before  97 blocks: 70 compile, 27 classified fragments, 2 redeclare an exported type (checked against it).
after   97 blocks: 70 compile, 27 classified fragments, 3 redeclare an exported type (checked against it), 0 redeclared name(s) not compared.

Same 97 blocks, same 70 compiling, same 27 entries. The tally moves 2 → 3 because SearchResult is covered again — exactly the number you predicted.

B1 — `type X<T = default>` escaped the shadowing check

Reproduced. Your defeat, verbatim, line-count preserving, at docs/content/reference/rag.md:582:

type SearchResult<T = unknown> = {
  totallyWrong: boolean
  alsoWrong: T
}

97 blocks: 70 compile, 27 classified fragments, 2 redeclare an exported type (checked against it).
EXIT=0

Fixed. The alias branch's parameter list was (?:<[^=]*>)?, which cannot cross a =. It is now (?:<[^<>]*(?:<[^<>]*>[^<>]*)*>)? — it tolerates a default, tolerates one level of nesting (<T extends Record<string, unknown>>), still spans newlines as the old character class did, and cannot run away past an unbalanced bracket. The interface branch is untouched.

Caught. Same edit, after:

FAIL   docs/content/reference/rag.md:582 (redeclares SearchResult)
         shadows SearchResult from @opensaas/stack-rag — TS2739: Type 'SearchResult<P0>' is missing the following properties from type 'SearchResult<P0>': item, score
         shadows SearchResult from @opensaas/stack-rag — TS2739: Type 'SearchResult<P0>' is missing the following properties from type 'SearchResult<P0>': totallyWrong, alsoWrong
97 blocks: 69 compile, 27 classified fragments, 3 redeclare an exported type (checked against it), 0 redeclared name(s) not compared.
EXIT=1

Both directions, and note P0 — see B2 for why the comparison is now at arity 1 rather than against the defaults.

B2 — a generic export was an unfixable failure, reported as a comparison

Reproduced. Your Row block, appended to packages/rag/README.md:

FAIL   packages/rag/README.md:504 (redeclares Row)
         shadows Row from @opensaas/stack-core — TS2314: Generic type 'Row' requires 3 type argument(s).
         shadows Row from @opensaas/stack-core — TS2314: Generic type 'Row' requires 3 type argument(s).
98 blocks: 70 compile, 27 classified fragments, 3 redeclare an exported type (checked against it).

What comparing a generic against a non-generic redeclaration should mean. The probe needs a type-argument arity both declarations accept, and parameters it can actually fill. So it now computes one:

arity = min(documented.length, shipped.length) — the most arguments both sides take, so the comparison is between the two type constructors, not between one instantiation of each;
if that is below either side's required count, no arity works and the two names are simply unrelated;
if any parameter it would fill carries a constraint (R extends RemainderBase, K extends keyof R & string), it cannot supply an inhabitant and does not pretend to.

Where an arity exists the probe emits export function probe<P0, …>(documented: Documented<P0, …>, shipped: Shipped<P0, …>) and does both assignability directions with fresh opaque parameters. Where none exists the name is reported NOT COMPARED and excluded from the tally.

A block whose name merely collides with an unrelated generic is not a defect, so it is not failed:

NOT COMPARED packages/rag/README.md:504 — Row from @opensaas/stack-core — the package declares 3 type parameter(s), 3 required, and the block declares 0
98 blocks: 71 compile, 27 classified fragments, 3 redeclare an exported type (checked against it), 1 redeclared name(s) not compared.
EXIT=0

The block compiles (71, up from 70), the tally stays at 3, and the run says what it did not do.

Falsified, so this is not a new blind spot. A comparable generic must still be compared and still fail. ColumnFilter<V> — 1 required parameter, unconstrained — declared wrongly:

FAIL   packages/rag/README.md:504 (redeclares ColumnFilter)
         shadows ColumnFilter from @opensaas/stack-core — TS2322: Type '…/shadowed.ColumnFilter<P0>' is not assignable to type '…/secured-list.ColumnFilter<P0>'.
         shadows ColumnFilter from @opensaas/stack-core — TS2322: … Type 'P0' is not assignable to type 'ColumnFilter<P0>'.
98 blocks: 70 compile, 27 classified fragments, 4 redeclare an exported type (checked against it), 0 redeclared name(s) not compared.

And the constrained arm, SearchableList<TRow extends Row = Row> redeclared as SearchableList<T>:

NOT COMPARED packages/rag/README.md:504 — SearchableList from @opensaas/stack-rag/runtime — its type parameters carry constraints the probe cannot fill

One extra over-claim of the same shape, found while fixing this. Your F3 case — a redeclaration nested inside a function — failed correctly but was still counted in "checked against it" while its own message said it was never compared. compileShadowProbe now reports whether the comparison ran, and the tally excludes it. The (redeclares …) tag still lists every redeclared name, since that is factual either way:

FAIL   packages/rag/README.md:504 (redeclares SearchResult)
         shadows SearchResult from @opensaas/stack-rag — TS2661: Cannot export 'SearchResult'. … — not declared at module scope, so it was never compared
98 blocks: 70 compile, 27 classified fragments, 3 redeclare an exported type (checked against it), 0 redeclared name(s) not compared.

Tally stays at 3, not 4.

B3 — a listed file that does not exist was silently dropped with its keys

Reproduced. docs/content/reference/rag.md moved aside, exactly your rename scenario:

75 blocks: 52 compile, 23 classified fragments, 1 redeclare an exported type (checked against it).
EXIT=0

22 blocks and 4 fixture entries retired in silence, green.

Fixed. Files are read once into a map; a path that throws is recorded. .changeset/* is the only exemption, and it now covers the keys as well as the file. Everything else is a MISSING failure, and its keys fall through to the orphan arm.

Caught.

MISSING docs/content/reference/rag.md — listed in files.txt but could not be read
ORPHAN docs/content/reference/rag.md:169 — fragments.json classifies no block at that line:
ORPHAN docs/content/reference/rag.md:266 — …
ORPHAN docs/content/reference/rag.md:396 — …
ORPHAN docs/content/reference/rag.md:669 — …
0 failing, 0 stale and 4 orphaned classification(s), 1 listed file(s) missing.
EXIT=1

The release path still works. .changeset/swift-pandas-listen.md moved aside: 95 blocks, its one key exempted, EXIT=0. That exemption is now described in files.txt as the single exception it is, rather than as the general rule.

Follow-ups

The main gate. I checked whether the skip is still justified rather than assuming it: all five markdown documents exist on main but differ from these by 919 insertions / 848 deletions, and main does not carry the secured-surface exports the shadowing check compares against. So the fixtures genuinely cannot apply there yet and the gate stays — but it is no longer silent. The steps now have the == 'main' counterpart their siblings have, which does not run the check and instead emits a workflow annotation on every PR into main:

Doc-block typecheck did not run — scripts/doc-blocks fixtures are keyed to the prisma-8 documents … When prisma-8 lands on main, re-key scripts/doc-blocks/fragments.json and remove both if: gates (#1301).

The reason and the exact flip are written at the gate and in Known limits, both naming #1301. I have not opened a separate tracking issue — say the word if you want the follow-up filed as its own issue rather than carried on #1301.

tsx fences. Before, a JSX block produced TS1161: Unterminated regular expression literal and TS2304: Cannot find name 'div' — the misdescribing-fragments.json-entry trap you named. React is not resolvable from the scratch project, so it now reports the real cause and is unexcusable:

FAIL   packages/rag/README.md:504
         a tsx fence carries JSX, which this check sets no `jsx` option for

Adding a bogus fragments.json excuse for it still fails, exit 1.

Prelude guard attribution. declare global { const context: number } in a block, before and after:

before  prelude prelude.ts: TS2451: Cannot redeclare block-scoped variable 'context'.
        EXIT=2

after   prelude prelude.ts: TS2451: Cannot redeclare block-scoped variable 'context'.
        Raised while checking packages/rag/README.md:504 — either the build is broken or
        that block's own code reaches into a prelude. Blocks after it did not run.
        EXIT=2

Still exit 2 — a diagnostic in a prelude means nothing downstream is trustworthy — but now attributable, and recorded in Known limits including the fact that later blocks do not run.

The orphan arm's second hole. A key naming an unlisted file, and a key with no colon (lastIndexOf returning -1 and truncating the path), were both silently ignored. Before: EXIT=0, nothing said. After:

ORPHAN docs/content/how-to/not-in-files-list.md:12 — fragments.json classifies no block at that line:
ORPHAN no-colon-at-all — fragments.json classifies no block at that line:
0 failing, 0 stale and 2 orphaned classification(s), 0 listed file(s) missing.
EXIT=1

Known limits. Rewritten to cover all three blockers — the arity rule and what it declines to compare, the tsx gap, the missing-file rule and its one exemption, the prelude abort, and the CI gate. files.txt's header no longer describes B3's behaviour as a feature.

The two numbers. Corrected in the body: files.txt lists six paths (five documents plus a changeset), not nine, and the map holds 242 unique names — the nine entry points export 289 type names in total, which is where your 289 comes from. 41 of the 242 are generics with at least one required parameter, matching your count.

Regression controls

Every previously-demonstrated arm re-run against the new script: STALE fires, ORPHAN fires, an injected const broken: number = 'str' fires, the nested-redeclaration F3 arm fires, the original module-scope export type SearchResult defeat fires in both directions, and the clean tree exits 0.

Checks

pnpm lint — 0 errors, the same 2 pre-existing warnings · pnpm format + format:check — clean · pnpm build including the docs build — pass · pnpm check:doc-ts-blocks — exit 0.

No changeset: nothing under packages/*/src was touched.

github-actions · 2026-09-08T04:09:34Z