initial commit for multidb - #7862
Conversation
|
/retest |
getattr(value, 'pulp_domain_id', None) silently returns None for models that reach their domain transitively rather than through their own field (RepositoryVersion.repository, RepositoryContent.repository, ContentArtifact.content, PublishedArtifact.publication). RepositoryVersion is the single most common CreatedResource target in pulpcore (every sync/publish creates one), so this meant most real-world post-move CreatedResource rows for satellite-hosted domains silently resolved content_object on the wrong (control-plane) alias instead of raising or logging -- the exact KI-18 landmine, reached via an uncovered path. Found via live validation against a real two-Postgres environment: a genuine post-move CreatedResource pointing at a satellite RepositoryVersion resolved to a stale pre-cleanup copy on 'default' instead of the live row on the satellite alias. Fixes by walking the target's concrete FK/O2O fields (bounded depth, cycle-guarded) to find any related object exposing pulp_domain_id, rather than a hardcoded model list -- covers all four known cases generically and any future/plugin model with the same shape. Co-authored-by: Cursor <cursoragent@cursor.com>
…ntext test_content_object_domain_id_set_for_repository_version created the CreatedResource outside with_task_context(task), so CreatedResource.task (a required NOT NULL FK) never got auto-populated -- IntegrityError on insert. The aborted transaction from that failure can poison the connection state for whatever DB-touching test runs next in the same session, which is why this surfaced downstream (test_reconciliation.py) in a full-suite run rather than pointing at the actual buggy test. Verified clean: full pulpcore.tests.unit suite (363 passed, 2 skipped, 0 failed) against two real local Postgres instances mirroring the CI 'multi_db' matrix leg, plus a separate single-DB (no data_1) run confirming the skip path still works. Co-authored-by: Cursor <cursoragent@cursor.com>
|
@gerrod3 What do you think? Can I get a review? |
gerrod3
left a comment
gerrod3
left a comment
There was a problem hiding this comment.
Round 1 of reviews. This is honestly quite unreviewable in its current state. The AI comments are a nightmare and make references to docs and comments I don't have access to. The commits are not logically structured either. I would probably have ordered them something like:
- The initial adding of the database-alias and db-router
- Fixing management commands and other models (GenericReleation)
- Adding the database domain migration command
- CI work and tests
I would like to set expectations now that this will require major changes and many iterations before we are close to a state that might be mergeable.
| if domain is not None: | ||
| return getattr(domain, "database_alias", "default") |
There was a problem hiding this comment.
Does this getattr not trigger something crazy too?
There was a problem hiding this comment.
It won't be crazy, but it might add extra queries, so makes sense to change it.
| CROSS_PLANE_RECONCILIATION_GRACE_MINUTES = 60 | ||
|
|
||
| # KI-11: how long, in days, a confirmed-orphaned cross-plane row is kept (logged/alerted on every | ||
| # sweep) before the reconciliation sweep deletes it outright. 0 disables purging entirely -- | ||
| # orphans are only ever logged, never deleted, which is the safe default. | ||
| CROSS_PLANE_RECONCILIATION_PURGE_AFTER_DAYS = 0 |
There was a problem hiding this comment.
Why do we need these two different settings?
There was a problem hiding this comment.
The first one makes sure we don't "flag" objects in "flight" as orphans. For example in the case of a migration of an active domain to a new DB, we need some grace period before we start flagging as orphans
And than how regularly we purge is a different setting, the 2x can be very different values. Multiple days vs multiple hours. Makes sense to be 2 phased in a way.
| if len(settings.DATABASES) > 1: | ||
| DATABASE_ROUTERS = ["pulpcore.app.db_router.PulpDomainRouter"] | ||
| settings.set("DATABASE_ROUTERS", DATABASE_ROUTERS) |
There was a problem hiding this comment.
Remove this. Routers must be explicitly set by the user.
| if len(settings.DATABASES) <= 1: | ||
| return super().filter(*args, **kwargs) |
There was a problem hiding this comment.
This check should be if the router is the special PulpMultiDBRouter, not if there are more than one 1 database.
There was a problem hiding this comment.
I agree, it's a bit extra-safe. Will change it.
| _DOMAIN_WALK_MAX_DEPTH = 2 | ||
|
|
||
|
|
||
| def _resolve_domain_id(value, _depth=0, _seen=None): |
There was a problem hiding this comment.
Do we really need this? Yeah it's probably the safest way to get the domain of the object, but I would expect that get_domain would always return the correct domain that the object is in. Maybe it doesn't matter since creating GenericRelationships typically never happen in a hot path
There was a problem hiding this comment.
I see it more as a protection for the future, just playing safe.
| read-only copy of every `Domain` row must also exist on every other configured `DATABASES` alias | ||
| so that per-process code (the router, `for_each_domain()`, `Domain.get_storage()`, etc.) can |
There was a problem hiding this comment.
Why should this be true? It doesn't seem like it should be. The only thing that I would expect that needs to exist on the satellite dbs is the default domain and the satellite domain.
There was a problem hiding this comment.
This was more related to how the sync would work. Changed that, so now it's not true/needed.
As i was keeping the domain metadata synced between dbs.
Addresses gerrod3's review comment on _resolve_db's Domain-hint branches
("Does this getattr not trigger something crazy too?"): getattr(domain,
"database_alias", "default") goes through the same DeferredAttribute.__get__
as pulp_domain_id does one branch up, so a Domain instance reaching here
with database_alias deferred would call refresh_from_db() for it.
It can't actually recurse the way the pulp_domain_id case (KI-27) does --
Domain is control-plane, so the re-entrant db_for_read(Domain, ...) call
hits the _is_control_plane short-circuit and returns "default" before ever
reaching the instance-hint branch again. So the real failure mode here is
a silent extra query, not a RecursionError. No call site defers the field
today (DomainMiddleware's plain .get(), and select_related("pulp_domain")
with no field restriction on every task-fetch path), so this isn't an
active bug, but it's the same class of "assume it's always loaded"
fragility KI-27 already burned us on once. Added _database_alias() to
close it out unconditionally, plus a regression test that asserts zero
queries when the field is deferred.
Co-authored-by: Cursor <cursoragent@cursor.com>
…cit, and drop external doc references Domain metadata now only replicates to `default` (everywhere) and to a domain's own current `database_alias`, instead of every domain onto every satellite; move-domain explicitly seeds the destination alias before copying data, and cleanup-moved-domain/sync-domains prune the stale replica left behind on the source alias after a move. Also stops auto-activating DATABASE_ROUTERS from DATABASES (must be set explicitly now, via the new is_multi_db_routing_active() check), and removes lingering comments/messages that referenced an external design doc not available to reviewers. Fixes two bugs found via live end-to-end validation against a local multi-database setup: cleanup-moved-domain deleting the authoritative Domain row when a domain's original alias was "default", and a DATABASE_ROUTERS unit test relying on override_settings, which silently never reached the real router registry under this project's dynaconf settings integration. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Rebase this please |
3 participants
Initial PR for the multidb implementation. To test github actions. etc