Repository navigation
feat(db): add the identity_aliases table (schema only) - #2080
Open
amarkanday wants to merge 2 commits into
Open
amarkanday wants to merge 2 commits into
amarkanday wants to merge 2 commits into
Conversation
Part of #449. Schema only: one new core table, `identity_aliases`, and its model. Nothing reads or writes it in this change; no service, route, request, response or OpenAPI change. The table: `anonymous_id varchar(255)` primary key, `user_id varchar(255) NOT NULL`, `created_at timestamptz NOT NULL DEFAULT now()`, the named `check_identity_aliases_ids_differ` (`anonymous_id <> user_id`) and `ix_identity_aliases_user_id (user_id)`, schema-qualified, with the same names on the model (`backend/app/models/identity_alias.py`, added to CORE_MODEL_MODULES). Revision `t449_identity_aliases` extends 0.47.0's core head `t1812_user_totp`; the heads are `t449_identity_aliases` and `modules_0005_redshift`. Expand only: no foreign key, no ALTER, no index on an existing table, no backfill, and each step runs only when its object is absent. It opens with a `SET LOCAL` that bounds any wait for another session's hold on an object at 250 ms (one module constant, never 0) and ends by resetting that bound to the default; a wait that reaches the bound (SQLSTATE 55P03) refuses with a message that names the table and the bound, says nothing was applied and says to run the upgrade again. Offline (`--sql`) is refused in both directions. The downgrade drops the table and every link; assignments and events are untouched. Tests: - a wait-budget test (test_upgrade_lock_budget.py) with one literal map per release, each with its start heads: 0.48.0's is {t449_identity_aliases: 0.25}, walked from 0.47.0's heads in this profile (ancestors of the heads minus ancestors of the start heads), cap 4 s; and a static test that 0.47.0's three revisions set no such bound and that 4955571d232c alters experiments and variants; - the revision read without a database: id, literal down_revision, heads, model/migration agreement, the bound, the refusal, offline; - the migration on PostgreSQL: upgrade from 0.47.0 in both profiles, catalog equal to create_all, an online statement capture (exactly the SET, CREATE TABLE with no REFERENCES, CREATE INDEX on the new table and the reset; every other read is a catalog read), commits beside sessions holding the six SDK tables, a concurrent CREATE TABLE ends it with the refusal and nothing applied, the downgrade, the heads; - the downgrade's announcement by name in the migrations page, the rollback runbook and db-migrate.yml's help text. Moves with it: every `CORE_HEAD` pin (20 files), FULL_PLAN 45 -> 46, and the core head in the graph and target maps of test_alembic_commands, test_alembic_plan, test_unresolvable_revisions_message and test_db_migrate_target_graph. Docs: the self-hosting migrations page gains a `t449_identity_aliases` section with the upgrade note "for an upgrade from 0.47.0", and every documented downgrade target below the revision says what its downgrade drops; registries updated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011kqtSDC7sd5ue3m6LuwdXV Signed-off-by: amarkanday <a.markanday@gmail.com>
Part of #449. Twelve existing database tests downgrade from the core head to a revision below `t449_identity_aliases` and compare the tables left with the tables before, minus what later revisions' downgrades drop. With `t449_identity_aliases` on top, each of those downgrades also drops `identity_aliases`, so 24 test cases (core and full) failed with exactly one difference: "Extra items in the right set: 'identity_aliases'". Each file's `LATER_DOWNGRADE_TABLES` gains `identity_aliases`; test_user_totp_migration.py, which had no such set, gains one and subtracts it in its downgrade assertion. Nothing else changes: the 24 cases pass, and no other assertion moves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011kqtSDC7sd5ue3m6LuwdXV Signed-off-by: amarkanday <a.markanday@gmail.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #449.
Schema only. This adds one core table,
identity_aliases, its model, and the revision that creates it. Nothing reads or writes the table in this change: no service, route, request, response or OpenAPI change (make openapileaves the tree unchanged).What it adds
identity_aliases, schema-qualified, with the same names on the model (backend/app/models/identity_alias.py, added toCORE_MODEL_MODULES):anonymous_idvarchar(255), primary key: an anonymous id is linked at most onceuser_idvarchar(255) NOT NULL: the known id it is linked tocreated_attimestamptz NOT NULL DEFAULT now()check_identity_aliases_ids_differCHECK (anonymous_id <> user_id)ix_identity_aliases_user_id(user_id)Revision
t449_identity_aliasesextends 0.47.0's core head,t1812_user_totp(literaldown_revision; the id is 21 characters).alembic headsprints exactlyt449_identity_aliases (head)andmodules_0005_redshift (modules) (head).pg_catalog), so a schema built bycreate_allis left alone.SET LOCALthat bounds its wait for another transaction's hold on an object at'250ms'(one module constant,_LOCK_TIMEOUT, never 0) and ends by setting it backTO DEFAULT. A wait that reaches the bound (SQLSTATE 55P03) refuses the upgrade with a message that names the table and the 250 ms, says nothing was applied and says to run the upgrade again.--sql) is refused in both directions.db-migrate.yml's help text say so wherever they name a target below it.Wait budget, for an upgrade from 0.47.0
{"t449_identity_aliases": 0.25}inbackend/tests/unit/db/test_upgrade_lock_budget.py.RELEASESholds one map per release, oldest first, each with its own start heads; a release's walk is the ancestors of where it ends (the next release's start heads, or the current heads) minus the ancestors of its start heads present in the profile; cap 4 s. A later release appends its own entry and changes none. (RELEASESis a tuple of named records rather than a dict keyed"0.48.0":test_engine_versions_frozen.pytreats any dict keyedN.N.Nunderbackend/tests/unitas a table of stats-engine pins.)t1812_user_totpandmodules_0005_redshiftare v0.47.0's revisions and outside that sum:git diff --name-only v0.47.0 origin/main -- backend/app/db/migrations modules/backend/app/db/migrationsprints nothing, andgit tag --contains 119f37f5(feat(auth): store two-step sign-in secrets for local accounts #1948,t1812_user_totp) andgit tag --contains f333d513(feat(warehouse): Amazon Redshift connector type (ships disabled, no client) #1943,modules_0005_redshift) both namev0.47.0first.upgrade headsruns every pending revision in one transaction (env.pysets notransaction_per_migration). The 55P03 test shows it from 0.47.0 and from the revision before (4955571d232c), where the refused run also takes backt1812_user_totp's table.log_lock_waits=onwith a 10 ms reporting threshold): from a v0.47.0 bootstrap with SDK-style transactions running back to back (experiment and variant reads, assignment and event inserts),alembic upgrade headstook 0.57 s; 2,386 of those transactions ran during it, and the slowest of the whole run was not among them; 345 polls found no ungranted request, and the server log had no wait line. Behind a rival's uncommittedCREATE TABLE <schema>.identity_aliases, the log showsstill waiting for ShareLock on transaction ... after 11.985 msat 56.355 and the statement cancelled at the bound (55P03) at 56.594, the upgrade exits 1 with the refusal, andalembic_versionis unchanged.4955571d232c,t1812_user_totpandmodules_0005_redshift, none of which bounds its waits. Measured on PostgreSQL 14 while this change was planned: behind one open transaction that had readexperiments,4955571d232cwaited until the probe's statement timeout ended it, and SDK reads ofexperimentsqueued behind it for that time. That is a property of v0.47.0's revisions, not oft449_identity_aliases; a new static test pins the premise (none of the three sets such a bound, and4955571d232caltersexperimentsandvariants), and the migrations page says it in its upgrade note.What moves with it
CORE_HEAD = "t1812_user_totp"pin (20 files) now namest449_identity_aliases; 0 left at the old text.FULL_PLANand its length test: 45 -> 46.test_alembic_commands.py(3 tests),test_alembic_plan.py(3),test_unresolvable_revisions_message.py(5),test_db_migrate_target_graph.py(11; each named downgrade's list gainst449_identity_aliases, the branch-point plan 21 -> 22, and a newt1812_user_totpentry for the page's "undoes this revision alone" command).LATER_DOWNGRADE_TABLESgainsidentity_aliases(test_user_totp_migration.pygains the set). Without it 24 cases fail with exactlyExtra items in the right set: 'identity_aliases'.t449_identity_aliasessection on the self-hosting migrations page (what it adds; the previous release, 0.47.0, runs against it unchanged; the upgrade note for an upgrade from 0.47.0; the downgrade), the downgrade sentence beside every documented target below it, the history andcurrentexamples, the runbook and the workflow help; both registries updated (doc_claims.py check: problems 0); the page's skip count inscripts/doc_examples.toml15 -> 16.Removed assertion lines (
git diff -U0 origin/main...HEAD -- <the modified test files> | grep -E '^-.*\bassert\b'):Files (
git diff --name-only origin/main...HEAD):Verification
Base: main 373744b. The evidence below was measured on an earlier build of this branch, whose tree differs from this head by one docstring line in a test file; at this head the migration's integration test and its two unit test files were run again: 30 passed. Database tests on a scratch PostgreSQL 14.17 (TCP only,
LC_COLLATE=C,LC_CTYPE=en_US.UTF-8,TimeZone=UTC), no Docker.backend/tests/integration/databasebackend/tests/unit -k "alembic or migration or plan or revision"backend/tests/unit, the whole suite, full treebackend/tests/smoketest_version_sources.pycases need setuptools>=77, asrelease.ymlinstalls; with it, 25 passed)ruff check/ruff format --checkoverbackend/ modules/ scripts/lint-imports(root andbackend/lambda)reuse lintpython scripts/doc_claims.py checkpython scripts/doc_examples.py --checkmake openapi, thengit status --porcelainscripts/check_dco.py origin/main HEAD;gitleaks gitoverorigin/main..HEADactionlint .github/workflows/db-migrate.yml(the lint job'sSHELLCHECK_OPTS)scripts/core_build.sh(core profile) end to end, in its own venv with the lint extrasAWS_DEFAULT_REGIONin the shell: with it set,backend/lambda/feature_flag_evaluationfails two tests on main too (the network guard stops a client from connecting)Transitions (
alembic_versionafter each step):alembic_versiont449_identity_aliasesalembic upgrade heads(core database met by the full image)modules_0005_redshift, t449_identity_aliasesmodules_0005_redshift, t449_identity_aliasesalembic upgrade headsand bootstrap (full database met by the core image)modules_0005_redshift, t449_identity_aliases(nothing to apply)modules_0005_redshift, t1812_user_totpalembic upgrade heads(Running upgrade t1812_user_totp -> t449_identity_aliases)modules_0005_redshift, t449_identity_aliasesalembic upgrade heads, and bootstrap, meet italembic downgrade t1812_user_totpmodules_0005_redshift, t1812_user_totpmodules_0005_redshift, t1812_user_totpalembic_version present in schema ...: running upgrade heads, thenRunning upgrade t1812_user_totp -> t449_identity_aliases)modules_0005_redshift, t449_identity_aliasesalembic upgrade headst1812_user_totp, thent449_identity_aliasesEvery new test, broken and restored (each plant in a scratch copy of the tree; restored runs all green):
core_build.sh's tree (nomodules/, no.git)test_every_revision_each_release_runs_is_declared_and_the_sum_fits: for0.48.0, the declared sum 4.5 s exceeds 4 s; also the exact-entry and range tests0.48.0,t449_identity_aliasesreported as a revision with no declared boundt1812_user_totpalonemodules_0001_rbactomodules_0005_redshiftreported as revisions with no declared bound (the core tree has no modules branch, so the same walk is correct there)RELEASESnames 0.48.0 twicetest_the_releases_are_named_once_each:assert (['0.48.0', '0.48.0'] and 2 == 1)SET LOCALbound added to4955571d232c, both treestest_0_47_0_revisions_set_no_lock_timeout_and_4955571d232c_alters_experiments_and_variants: it names4955571d232cas setting the bound4955571d232c'svariantsconstant renamedassert {'experiments', 'variants'} <= {'experiments', 'variant_rows'}assert 3 == 4(core and full); the 55P03 test:canceling statement due to statement timeout(57014), the refusal text missing (both start points); the unit bound testeventsstatement touches events: CREATE INDEX ix_plant_events_user ON <schema>.events (user_id)experimentsREFERENCES in the alias table's DDL: CREATE TABLE <schema>.identity_aliases (...)SELECT count(*)fromeventsin the revisiona read set aside reads a table: {'SELECT count(*) FROM <schema>.events': ['<schema>.events']}refusal text missing from --sql output: ('upgrade', 't1812_user_totp:t449_identity_aliases', '--sql'); both unit offline casesassignmentstest_the_upgrade_commits_beside_sessions_holding_the_sdk_tables:upgrade blocked by a reader on an SDK tableuser_idat 128 (model unchanged)user_id,assert 128 == 255assert ([], [...]) == ([(...)], [...])(core and full)down_revisionleft on4955571d232cassert '4955571d232c' == 't1812_user_totp'; the heads tests (three heads)tree_profiles.CORE_HEADleft behindassert 't1812_user_totp' == 't449_identity_aliases'_LOCK_TIMEOUT = "0"assert '0' == '250ms'; the one-constant testSET LOCALstatements end without the resetTO DEFAULTtest_any_other_error_is_raised_as_it_wastest_identity_aliases_migration.py has mark.skipdb-migrate.yml's help texttest_every_target_that_crosses_this_revision_says_it_drops_the_links:{'db-migrate.yml': ['7b6b4b4fe246']}t1812_user_totpsection{'migrations.md': ['4955571d232c']}db-migrate.ymlexamples moved to the modules branchtest_each_document_names_a_target_that_crosses_this_revision:{'migrations.md', 'rollback-runbook.md'}test_a_target_whose_section_leaves_the_revision_out_is_reported:KeyError: 'migrations.md'NOT VERIFIED
core-buildwith its Redis services (locally the Redis-backed integration tests skip: no Redis here), and thelintjob's eslint, tsc and hadolint (not run: this change touches no frontend file or Dockerfile).CREATE TABLEopen.🤖 Generated with Claude Code
https://claude.ai/code/session_01NpxPyav9uBxNuiXsDvD8qn