Skip to content

feat(db): add the identity_aliases table (schema only) - #2080

Open
amarkanday wants to merge 2 commits into
mainfrom
feat/449-i1-identity-aliases-table
Open

amarkanday wants to merge 2 commits into
mainfrom
feat/449-i1-identity-aliases-table

Conversation

@amarkanday

Copy link
Copy Markdown
Contributor

Part of #449.

Schema only. This adds one core table, identity_aliases, its model, and the revision that creates it. Nothing reads or writes the table in this change: no service, route, request, response or OpenAPI change (make openapi leaves the tree unchanged).

What it adds

identity_aliases, schema-qualified, with the same names on the model (backend/app/models/identity_alias.py, added to CORE_MODEL_MODULES):

Column / object Definition
anonymous_id varchar(255), primary key: an anonymous id is linked at most once
user_id varchar(255) NOT NULL: the known id it is linked to
created_at timestamptz NOT NULL DEFAULT now()
check_identity_aliases_ids_differ CHECK (anonymous_id <> user_id)
ix_identity_aliases_user_id index on (user_id)

Revision t449_identity_aliases extends 0.47.0's core head, t1812_user_totp (literal down_revision; the id is 21 characters). alembic heads prints exactly t449_identity_aliases (head) and modules_0005_redshift (modules) (head).

  • Expand only. No foreign key, no ALTER, no index on an existing table, no backfill. Each step runs only when its object is absent (the existence checks read pg_catalog), so a schema built by create_all is left alone.
  • The one wait, bounded. The revision opens with a SET LOCAL that bounds its wait for another transaction's hold on an object at '250ms' (one module constant, _LOCK_TIMEOUT, never 0) and ends by setting it back TO DEFAULT. A wait that reaches the bound (SQLSTATE 55P03) refuses the upgrade with a message that names the table and the 250 ms, says nothing was applied and says to run the upgrade again.
  • Offline mode (--sql) is refused in both directions.
  • Downgrade drops the table and every link; assignments and events are untouched; an upgrade afterwards creates the table empty, and the links must be sent again. The self-hosting migrations page, the rollback runbook and db-migrate.yml's help text say so wherever they name a target below it.

Wait budget, for an upgrade from 0.47.0

  • The budget map's sum at this head is 0.25 s, for an upgrade from 0.47.0: {"t449_identity_aliases": 0.25} in backend/tests/unit/db/test_upgrade_lock_budget.py. RELEASES holds one map per release, oldest first, each with its own start heads; a release's walk is the ancestors of where it ends (the next release's start heads, or the current heads) minus the ancestors of its start heads present in the profile; cap 4 s. A later release appends its own entry and changes none. (RELEASES is a tuple of named records rather than a dict keyed "0.48.0": test_engine_versions_frozen.py treats any dict keyed N.N.N under backend/tests/unit as a table of stats-engine pins.)
  • t1812_user_totp and modules_0005_redshift are v0.47.0's revisions and outside that sum: git diff --name-only v0.47.0 origin/main -- backend/app/db/migrations modules/backend/app/db/migrations prints nothing, and git tag --contains 119f37f5 (feat(auth): store two-step sign-in secrets for local accounts #1948, t1812_user_totp) and git tag --contains f333d513 (feat(warehouse): Amazon Redshift connector type (ships disabled, no client) #1943, modules_0005_redshift) both name v0.47.0 first.
  • A 55P03 refusal in any revision applies none of that run's revisions: upgrade heads runs every pending revision in one transaction (env.py sets no transaction_per_migration). The 55P03 test shows it from 0.47.0 and from the revision before (4955571d232c), where the refused run also takes back t1812_user_totp's table.
  • From 0.47.0: the upgrade takes nothing on an existing table, SDK transactions do not wait on it, and its own wait is bounded at 0.25 s. Measured again on this branch (PostgreSQL 14.17, log_lock_waits=on with a 10 ms reporting threshold): from a v0.47.0 bootstrap with SDK-style transactions running back to back (experiment and variant reads, assignment and event inserts), alembic upgrade heads took 0.57 s; 2,386 of those transactions ran during it, and the slowest of the whole run was not among them; 345 polls found no ungranted request, and the server log had no wait line. Behind a rival's uncommitted CREATE TABLE <schema>.identity_aliases, the log shows still waiting for ShareLock on transaction ... after 11.985 ms at 56.355 and the statement cancelled at the bound (55P03) at 56.594, the upgrade exits 1 with the refusal, and alembic_version is unchanged.
  • From 0.46.0 or earlier, the upgrade also runs 4955571d232c, t1812_user_totp and modules_0005_redshift, none of which bounds its waits. Measured on PostgreSQL 14 while this change was planned: behind one open transaction that had read experiments, 4955571d232c waited until the probe's statement timeout ended it, and SDK reads of experiments queued behind it for that time. That is a property of v0.47.0's revisions, not of t449_identity_aliases; a new static test pins the premise (none of the three sets such a bound, and 4955571d232c alters experiments and variants), and the migrations page says it in its upgrade note.

What moves with it

  • Every CORE_HEAD = "t1812_user_totp" pin (20 files) now names t449_identity_aliases; 0 left at the old text.
  • FULL_PLAN and its length test: 45 -> 46.
  • The core head in the graph and target maps of four unit test files, recomputed at 373744b: test_alembic_commands.py (3 tests), test_alembic_plan.py (3), test_unresolvable_revisions_message.py (5), test_db_migrate_target_graph.py (11; each named downgrade's list gains t449_identity_aliases, the branch-point plan 21 -> 22, and a new t1812_user_totp entry for the page's "undoes this revision alone" command).
  • Twelve database test files whose downgrade tests subtract what later revisions drop: each LATER_DOWNGRADE_TABLES gains identity_aliases (test_user_totp_migration.py gains the set). Without it 24 cases fail with exactly Extra items in the right set: 'identity_aliases'.
  • Docs: a t449_identity_aliases section on the self-hosting migrations page (what it adds; the previous release, 0.47.0, runs against it unchanged; the upgrade note for an upgrade from 0.47.0; the downgrade), the downgrade sentence beside every documented target below it, the history and current examples, the runbook and the workflow help; both registries updated (doc_claims.py check: problems 0); the page's skip count in scripts/doc_examples.toml 15 -> 16.

Removed assertion lines (git diff -U0 origin/main...HEAD -- <the modified test files> | grep -E '^-.*\bassert\b'):

-    assert _tables(test_db, scratch_schema) == tables - {TABLE}
-    assert len(_downgrade_plan(script, "modules@base")) == len(FULL_PLAN) == 45
-    assert len(plan) == 21, plan

Files (git diff --name-only origin/main...HEAD):

.github/workflows/db-migrate.yml
backend/app/db/migrations/versions/t449_identity_aliases_anonymous_ids_linked_to_known_ids.py
backend/app/models/__init__.py
backend/app/models/identity_alias.py
backend/tests/integration/database/test_balance_within_migration.py
backend/tests/integration/database/test_event_dedup_keys_migration.py
backend/tests/integration/database/test_events_created_at_utc_migration.py
backend/tests/integration/database/test_experiment_correction_settings_migration.py
backend/tests/integration/database/test_experiment_resume_at_migration.py
backend/tests/integration/database/test_feature_flag_default_value_migration.py
backend/tests/integration/database/test_feature_flag_value_columns_migration.py
backend/tests/integration/database/test_holdout_population_migration.py
backend/tests/integration/database/test_identity_aliases_migration.py
backend/tests/integration/database/test_profile_transitions.py
backend/tests/integration/database/test_rollback_record_config_link_migration.py
backend/tests/integration/database/test_rollout_stage_started_at_migration.py
backend/tests/integration/database/test_sdk_evaluation_counts_migration.py
backend/tests/integration/database/test_segment_members_migration.py
backend/tests/integration/database/test_user_totp_migration.py
backend/tests/integration/database/test_users_email_lower_migration.py
backend/tests/integration/database/test_visual_experiments_migration.py
backend/tests/integration/database/tree_profiles.py
backend/tests/unit/db/test_alembic_commands.py
backend/tests/unit/db/test_alembic_plan.py
backend/tests/unit/db/test_identity_aliases_revision.py
backend/tests/unit/db/test_unresolvable_revisions_message.py
backend/tests/unit/db/test_upgrade_lock_budget.py
backend/tests/unit/docs/test_identity_aliases_downgrade_docs.py
backend/tests/unit/infrastructure/test_db_migrate_target_graph.py
docs/_claims/docs/deployment/rollback-runbook.md.toml
docs/_claims/docs/self-hosting/migrations.md.toml
docs/deployment/rollback-runbook.md
docs/self-hosting/migrations.md
scripts/doc_examples.toml

Verification

Base: main 373744b. The evidence below was measured on an earlier build of this branch, whose tree differs from this head by one docstring line in a test file; at this head the migration's integration test and its two unit test files were run again: 30 passed. Database tests on a scratch PostgreSQL 14.17 (TCP only, LC_COLLATE=C, LC_CTYPE=en_US.UTF-8, TimeZone=UTC), no Docker.

Gate Result
backend/tests/integration/database 443 passed, 1 skipped (the editable-install check, as on main)
backend/tests/unit -k "alembic or migration or plan or revision" 2217 passed, 8 skipped
backend/tests/unit, the whole suite, full tree 25382 passed, 171 skipped (three docs-walk tests also see an untracked notes file in the builder's clone; with it moved aside they pass)
backend/tests/smoke all pass (the 15 test_version_sources.py cases need setuptools>=77, as release.yml installs; with it, 25 passed)
ruff check / ruff format --check over backend/ modules/ scripts/ clean; 1981 files already formatted
lint-imports (root and backend/lambda) 3 kept, 0 broken; 1 kept, 0 broken
reuse lint compliant
python scripts/doc_claims.py check problems: 0
python scripts/doc_examples.py --check 60 document(s): contract holds
make openapi, then git status --porcelain no change
scripts/check_dco.py origin/main HEAD; gitleaks git over origin/main..HEAD ok; no leaks
actionlint .github/workflows/db-migrate.yml (the lint job's SHELLCHECK_OPTS) clean
scripts/core_build.sh (core profile) end to end, in its own venv with the lint extras every step ok: reuse, secrets, import, bootstrap, unit (25273 passed, 273 skipped), integration (6861 passed, 377 skipped), smoke, lambda, frontend, vectors, licences. The Lambda step was re-run on the same copy without AWS_DEFAULT_REGION in the shell: with it set, backend/lambda/feature_flag_evaluation fails two tests on main too (the network guard stops a client from connecting)

Transitions (alembic_version after each step):

Step Tree Exit alembic_version
1. bootstrap, fresh this branch, core 0 t449_identity_aliases
1. alembic upgrade heads (core database met by the full image) this branch, full 0 modules_0005_redshift, t449_identity_aliases
2. bootstrap, fresh this branch, full 0 modules_0005_redshift, t449_identity_aliases
2. alembic upgrade heads and bootstrap (full database met by the core image) this branch, core 0 modules_0005_redshift, t449_identity_aliases (nothing to apply)
3. bootstrap, fresh v0.47.0, full 0 modules_0005_redshift, t1812_user_totp
3. alembic upgrade heads (Running upgrade t1812_user_totp -> t449_identity_aliases) this branch, full 0 modules_0005_redshift, t449_identity_aliases
3. alembic upgrade heads, and bootstrap, meet it v0.47.0, full 1 (refuses: "records migration(s) this build has no file for (t449_identity_aliases)") unchanged
3. alembic downgrade t1812_user_totp this branch, full 0 modules_0005_redshift, t1812_user_totp
3. bootstrap after the downgrade v0.47.0, full 0 modules_0005_redshift, t1812_user_totp
3b. bootstrap over a v0.47.0 bootstrap (what Deploy runs: alembic_version present in schema ...: running upgrade heads, then Running upgrade t1812_user_totp -> t449_identity_aliases) this branch, full 0 modules_0005_redshift, t449_identity_aliases
3c. bootstrap, fresh, then alembic upgrade heads v0.47.0 core, then this branch core 0, 0 t1812_user_totp, then t449_identity_aliases

Every new test, broken and restored (each plant in a scratch copy of the tree; restored runs all green):

Plant Failing test and line
budget map: t449 at 4.5 s, in the full tree and in core_build.sh's tree (no modules/, no .git) test_every_revision_each_release_runs_is_declared_and_the_sum_fits: for 0.48.0, the declared sum 4.5 s exceeds 4 s; also the exact-entry and range tests
budget map: t449 at 2.5 s (the earlier plant (a)) the sum test passes, as the cap allows
budget map: the entry left out, both trees for 0.48.0, t449_identity_aliases reported as a revision with no declared bound
budget map: walked from t1812_user_totp alone full tree: modules_0001_rbac to modules_0005_redshift reported as revisions with no declared bound (the core tree has no modules branch, so the same walk is correct there)
RELEASES names 0.48.0 twice test_the_releases_are_named_once_each: assert (['0.48.0', '0.48.0'] and 2 == 1)
the same SET LOCAL bound added to 4955571d232c, both trees test_0_47_0_revisions_set_no_lock_timeout_and_4955571d232c_alters_experiments_and_variants: it names 4955571d232c as setting the bound
4955571d232c's variants constant renamed the same test: assert {'experiments', 'variants'} <= {'experiments', 'variant_rows'}
the SET dropped the capture: assert 3 == 4 (core and full); the 55P03 test: canceling statement due to statement timeout (57014), the refusal text missing (both start points); the unit bound test
an index on events the capture: statement touches events: CREATE INDEX ix_plant_events_user ON <schema>.events (user_id)
a foreign key to experiments the capture: REFERENCES in the alias table's DDL: CREATE TABLE <schema>.identity_aliases (...)
a SELECT count(*) from events in the revision the capture's separate assertion: a read set aside reads a table: {'SELECT count(*) FROM <schema>.events': ['<schema>.events']}
the offline guard removed refusal text missing from --sql output: ('upgrade', 't1812_user_totp:t449_identity_aliases', '--sql'); both unit offline cases
a foreign key to assignments test_the_upgrade_commits_beside_sessions_holding_the_sdk_tables: upgrade blocked by a reader on an SDK table
the migration's user_id at 128 (model unchanged) the upgrade test and the catalog comparison (core and full); unit: user_id, assert 128 == 255
the downgrade also deletes assignments assert ([], [...]) == ([(...)], [...]) (core and full)
down_revision left on 4955571d232c assert '4955571d232c' == 't1812_user_totp'; the heads tests (three heads)
tree_profiles.CORE_HEAD left behind assert 't1812_user_totp' == 't449_identity_aliases'
_LOCK_TIMEOUT = "0" assert '0' == '250ms'; the one-constant test
the reset removed the capture: the captured SET LOCAL statements end without the reset TO DEFAULT
55P03 not mapped / every error mapped the refusal test / test_any_other_error_is_raised_as_it_was
an integration test skipped test_identity_aliases_migration.py has mark.skip
t449's clause left out of db-migrate.yml's help text test_every_target_that_crosses_this_revision_says_it_drops_the_links: {'db-migrate.yml': ['7b6b4b4fe246']}
t449's sentence left out of the page's t1812_user_totp section the same test: {'migrations.md': ['4955571d232c']}
db-migrate.yml examples moved to the modules branch test_each_document_names_a_target_that_crosses_this_revision: {'migrations.md', 'rollback-runbook.md'}
the checker made blind test_a_target_whose_section_leaves_the_revision_out_is_reported: KeyError: 'migrations.md'

NOT VERIFIED

  • Aurora, and PostgreSQL 15 and 16: everything above ran on PostgreSQL 14.17. The first 0.47.0-to-t449 upgrade on Aurora is the release candidate's Deploy on staging.
  • CI itself: the full unit run, core-build with its Redis services (locally the Redis-backed integration tests skip: no Redis here), and the lint job's eslint, tsc and hadolint (not run: this change touches no frontend file or Dockerfile).
  • The skip-a-release measurement was made before this change was written, not re-run here; the static test pins only its premise.
  • Nothing reads the table yet, so the 250 ms bound has been exercised only by a test that holds a rival CREATE TABLE open.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NpxPyav9uBxNuiXsDvD8qn

amarkanday and others added 2 commits November 19, 2026 07:20
Part of #449. Schema only: one new core table, `identity_aliases`, and
its model. Nothing reads or writes it in this change; no service, route,
request, response or OpenAPI change.

The table: `anonymous_id varchar(255)` primary key, `user_id
varchar(255) NOT NULL`, `created_at timestamptz NOT NULL DEFAULT now()`,
the named `check_identity_aliases_ids_differ` (`anonymous_id <>
user_id`) and `ix_identity_aliases_user_id (user_id)`, schema-qualified,
with the same names on the model (`backend/app/models/identity_alias.py`,
added to CORE_MODEL_MODULES).

Revision `t449_identity_aliases` extends 0.47.0's core head
`t1812_user_totp`; the heads are `t449_identity_aliases` and
`modules_0005_redshift`. Expand only: no foreign key, no ALTER, no index
on an existing table, no backfill, and each step runs only when its
object is absent. It opens with a `SET LOCAL` that bounds any wait for
another session's hold on an object at 250 ms (one module constant,
never 0) and ends by resetting that bound to the default; a wait that
reaches the bound (SQLSTATE 55P03) refuses with a message that names the
table and the bound, says nothing was applied and says to run the
upgrade again. Offline (`--sql`) is refused in both directions. The
downgrade drops the table and every link; assignments and events are
untouched.

Tests:
- a wait-budget test (test_upgrade_lock_budget.py) with one literal map
  per release, each with its start heads: 0.48.0's is
  {t449_identity_aliases: 0.25}, walked from 0.47.0's heads in this
  profile (ancestors of the heads minus ancestors of the start heads),
  cap 4 s; and a static test that 0.47.0's three revisions set no such
  bound and that 4955571d232c alters experiments and variants;
- the revision read without a database: id, literal down_revision,
  heads, model/migration agreement, the bound, the refusal, offline;
- the migration on PostgreSQL: upgrade from 0.47.0 in both profiles,
  catalog equal to create_all, an online statement capture (exactly the
  SET, CREATE TABLE with no REFERENCES, CREATE INDEX on the new table and
  the reset; every other read is a catalog read), commits beside sessions
  holding the six SDK tables, a concurrent CREATE TABLE ends it with the
  refusal and nothing applied, the downgrade, the heads;
- the downgrade's announcement by name in the migrations page, the
  rollback runbook and db-migrate.yml's help text.

Moves with it: every `CORE_HEAD` pin (20 files), FULL_PLAN 45 -> 46, and
the core head in the graph and target maps of test_alembic_commands,
test_alembic_plan, test_unresolvable_revisions_message and
test_db_migrate_target_graph. Docs: the self-hosting migrations page
gains a `t449_identity_aliases` section with the upgrade note "for an
upgrade from 0.47.0", and every documented downgrade target below the
revision says what its downgrade drops; registries updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011kqtSDC7sd5ue3m6LuwdXV
Signed-off-by: amarkanday <a.markanday@gmail.com>
Part of #449. Twelve existing database tests downgrade from the core
head to a revision below `t449_identity_aliases` and compare the tables
left with the tables before, minus what later revisions' downgrades
drop. With `t449_identity_aliases` on top, each of those downgrades also
drops `identity_aliases`, so 24 test cases (core and full) failed with
exactly one difference: "Extra items in the right set:
'identity_aliases'".

Each file's `LATER_DOWNGRADE_TABLES` gains `identity_aliases`;
test_user_totp_migration.py, which had no such set, gains one and
subtracts it in its downgrade assertion. Nothing else changes: the 24
cases pass, and no other assertion moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011kqtSDC7sd5ue3m6LuwdXV
Signed-off-by: amarkanday <a.markanday@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant