fix(db): getDb() applies pending migrations on open (#202)#210
Merged
Conversation
getDb() opened the shared DB and applied PRAGMAs but never ran applyMigrations — only initDb did. A live DB therefore stayed pinned at whatever user_version it held when init last ran, and every CLI/MCP/hook opening via getDb() silently used a stale schema (observed live: v8 vs source v15 → memory_add INSERTing a missing provenance column → silent write failures). - Extract initDb's idempotent schema bootstrap into ensureSchema() and call it from both initDb and getDb. Preserves the load-bearing CREATE_TABLES → applyMigrations → CREATE_INDEXES/FTS/VECTOR ordering by construction (DRY). - getDb runs ensureSchema best-effort: a read-only FS or lock-contended open swallows the error and proceeds, so no new throw for the 68 read callers. - applyMigrations re-reads user_version under BEGIN IMMEDIATE before each migration, so a concurrent migrator can't double-apply. - Tests: v(N-1) → current via getDb; code_* base-table self-heal; concurrent convergence. Updated the repair legacy-DB test for the new self-heal contract.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #202.
Problem
getDb()(src/db/connection.ts, 68 callers — the shared hot open path) opened the DB and applied PRAGMAs but never calledapplyMigrations(). OnlyinitDb()(run byrecall init/ the installer) migrated. So once new migrations landed in source, a live DB stayed pinned at whateveruser_versionit held the last timeinitran, and every CLI/MCP/hook opening viagetDb()silently used a stale schema.Observed live (2026-06-25): a real DB sat at
user_version=8while source was 15 → theprovenancecolumn (ADR-0001/#42) was missing →memory_addwrites failed silently.Fix (Option 1 from the issue — runtime self-heal)
initDb's idempotent schema bootstrap intoensureSchema()and call it from bothinitDbandgetDb.getDbnow brings a drifted DB current on open, exactly asinitDbdoes.CREATE_TABLES → applyMigrations → CREATE_INDEXES/FTS/VECTOR → guarded vec table) is preserved by construction — both paths run the same helper (DRY), soinitDb's semantics are unchanged.Must-investigate findings
1. Ordering safety. Migrations are mostly self-contained (7→8 creates its own indexes; 14→15 / 15→16 create their own FTS objects inline). The exceptions are migrations that depend on base tables created by
CREATE_TABLES, not by the migration itself: 3→4 (extraction_*), 9→10 (dedup_lineage), and 15→16 (code_*). For a live v15 DB thecode_*base tables don't exist, and 15→16'stry/catchswallows their absence and only bumps the version — so callingapplyMigrationsalone fromgetDbwould advance to v16 while leavingcode_*missing (the version gate would then lie). Running the fullensureSchemasequence (i.e.CREATE_TABLESbeforeapplyMigrations, asinitDbdoes) is what makes the self-heal complete. Regression-tested.2. Concurrency.
applyMigrationsreaduser_versiononce before its loop. Now that every open can migrate, two processes can both observe version i and race. Added an under-lock re-read: afterBEGIN IMMEDIATEtakes the write lock (the loser waits outbusy_timeout=5000), re-readuser_versionand skip the iteration if a concurrent migrator already advanced past i — never double-apply. Each migration still runs in its own transaction.3. Read-only / locked DB.
getDbrunsensureSchemabest-effort: on a read-only FS, or when the write lock can't be won withinbusy_timeout, the throw is swallowed and the DB is used as-is — no new crash for the 68 callers (incl. pure read paths) that work today. The DB stays at its current version; the next writable open retries.initDbstill lets migration errors surface loudly (it's the explicit setup path).Tests
getDbmigrates a DB pinned atuser_version = N-1toMIGRATIONS.lengthon open (not justinitDb).getDbself-heals a base table a pending migration depends on — dropscode_*, pins to N-1, asserts they're recreated (proves the full ordering runs in the open path; regresses the live v15 gap).repairlegacy-DB test for the new contract:getDbnow self-healsdedup_lineageon open, sorepairsees a current schema (no crash;pending = 0).Gate
bun run lint(tsc) ✅ ·bun run build✅bun test: 1195 pass / 10 fail. The 10 failures are pre-existing onorigin/main(verified with this branch's changes stashed) and are entirely in the unrelated opencode/pi MCP-configure script (tests/opencode-integration.test.ts/tests/pi-*), not the DB layer. Not touched here — flagging separately.getDb()incl.ensureSchemaon an already-current DB ≈ 0.93 ms (idempotent DDL, once per process — same costinitDbalready paid).Out of scope (deliberate)
The
recall doctor --fixcheck (issue option 3) and any Phase 2 /code_*/ query-layer work.