refactor(graph): migrate Dependencies & Diagram to nx.MultiDiGraph (#1492)

[dimitri-yatsenko]

Resolves #1492.

Migrates Dependencies and Diagram from nx.DiGraph to nx.MultiDiGraph, retiring the integer "alias node" workaround for renamed / multiple foreign keys between the same pair of tables. A renamed FK is now a first-class parallel edge — no synthetic intermediate node.

Why

A plain DiGraph allows one edge per ordered node pair, so a renamed FK was split into parent → "N" → child through an integer alias node, and ~14 .isdigit() special-cases across diagram.py (plus 5 in table.py) existed only to see through those nodes. MultiDiGraph represents parallel edges natively, so the whole workaround is removed — it was a historical design choice, not a NetworkX limitation.

Edge identity

Each edge is keyed by the FK's child-side attribute tuple (in declaration order), e.g. ("spouse1",) vs ("spouse2",). This key is:

• unique among parallel edges between a pair — every FK to a given parent references the parent's primary key, so parallel FKs differ only in their referencing columns;
• schema-derived — no DB-internal constraint names;
• stable & meaningful — so graph merges are idempotent by construction: Diagram + Diagram re-adds a shared FK under the same key (overwrite, not duplicate). No defensive dedup is needed at the call sites.

What changed

• Dependencies/Diagram subclass nx.MultiDiGraph; load() adds one keyed edge per FK; topo_sort drops its alias-collapse pass; cascade()/trace() rebuild via MultiDiGraph.
• Both propagation walkers merged their alias/direct branches into one loop over parallel edges; _find_real_edge_props → _edge_props (direct lookup).
• dependencies.parents()/children() return a parallel-safe list[(node, props)] (was a name-keyed dict that collapsed multiple FKs from the same parent); Table.parents/children/descendants/ancestors/parts drop the .isdigit() handling.
• Rendering: a renamed FK draws as an #FF8800 edge with the column renames in a hover tooltip (dot/SVG) and a linkStyle recolor (mermaid), instead of the orange alias-node dot. The orange color is layered on top of the existing line semantics — solid=primary / dashed=secondary, thick=1:1 / thin=multi are all preserved. _AliasNode tier removed.
• _apply_collapse keeps parallel edges between two expanded tables (by key) while merging edges into a collapsed schema node to a single arrow.

Verification

Full tests/ suite under the pixi test env (MySQL 8.0 + PostgreSQL 15 via testcontainers):

• Green baseline (95 passed) on the graph/cascade/trace/FK suite before any change, reproduced after the migration — behavior-identical.
• Renamed-FK behavior green across all three style cases (solid-thick, solid-thin, dashed-thin — all orange): test_aliased_fk, test_trace_renamed_fk, test_cascade_delete_with_renamed_attrs, test_cascade_part_of_part_renamed_fk, test_upward_u3_nonprimary_master_fk; render smoke tests (test_repr_svg, test_make_image).
• Full suite: 949 passed, 10 skipped, 0 failures (both backends).

Notes

• Milestone v2.3.2. Paired docs update: docs(diagram): renamed FKs as orange edges [merge with v2.3.2 release] datajoint-docs#215 (renamed-FK notation), gated on this shipping.

[dimitri-yatsenko]

Paired docs PR: datajoint/datajoint-docs#215 — updates the diagram-notation docs (renamed FKs render as orange edges with a rename tooltip instead of alias-node dots) and adds a renamed-primary example showing the line-style semantics are retained.

The docs PR re-executed its notebooks against this branch, so it is gated on this shipping in v2.3.2 and must not merge until the release.

[MilagrosMarin]

Thanks — clean, substantive refactor. All 12 CI checks green (lint + Py 3.10/3.14 × MySQL 8.0/8.4 test matrix + unit), and the PR body's "949 passed, 10 skipped, 0 failures on both backends" checks out with the CI signal here.

Verified:

• Edge key tuple(fk["attr_map"]) is deterministic — information_schema.KEY_COLUMN_USAGE returns FK columns ordered by ORDINAL_POSITION on both MySQL and Postgres, so the tuple order is stable. Idempotency claim (Diagram + Diagram overwrites shared FKs) is correct given keyed-edge semantics.
• _AliasNode fully retired — deletion in user_tables.py symmetric with removed import in diagram.py; _get_tier now returns None for the integer-node case (which no longer arises).
• .isdigit() special-cases cleanly removed across table.py (parents/children/descendants/ancestors/parts) and dependencies.py:topo_sort.
• Rendering — #FF8800 orange on renamed-FK edges (SVG via set_color, Mermaid via linkStyle N stroke:#FF8800), tooltips with column renames. Layered on top of existing line semantics (solid/dashed for primary/secondary, thick/thin for 1:1/multi) as the PR body claims.

Three non-blocking observations:

1. Breaking API change worth calling out in v2.3.2 release notes

Dependencies.parents() / Dependencies.children() return type changed from dict[str, dict] to list[tuple[str, dict]]. The change is correct (a name-keyed dict couldn't represent parallel edges), and in-tree call sites in table.py are updated cleanly. But downstream consumers walking Dependencies directly will break silently — deps.parents(name)[parent_name] → TypeError: list indices must be integers, deps.parents(name).items() → AttributeError. Elements packages, plugins, or user framework-level code are the exposure. Worth an explicit line in the release notes.

2. Consider adding a test for the parallel-edges capability itself

Existing tests listed in the PR body (test_aliased_fk, test_trace_renamed_fk, test_cascade_delete_with_renamed_attrs, test_cascade_part_of_part_renamed_fk, test_upward_u3_nonprimary_master_fk) all exercise the single-renamed-FK cases and pass — that's a solid sanity signal that behavior is unchanged. But two parallel edges between the same pair of tables (the novel capability MultiDiGraph enables and the alias-node workaround emulated) — e.g. Relationship with -> Person.proj(person1='person_id') and -> Person.proj(person2='person_id') — has no direct test. A targeted case that inserts both FKs and asserts (a) parents(Relationship) returns two entries for Person, (b) cascade/trace handle both edges, (c) rendering produces two distinct edges — would give the new parallel-edges path direct coverage rather than relying on implicit exercise.

3. Needs rebase before merge

Branch predates #1506 (merged 3 days later); the raw git diff origin/master shows my #1506 changes as "removals" purely because of that offset. Auto-mergeable (no file overlap with the gc.py changes on master), but worth rebasing so the diff view and history stay clean.

Approving on the substance — the refactor itself is well-executed.

[dimitri-yatsenko]

Thanks @MilagrosMarin — responses to the three observations:

1. Breaking API change — scoped to internal plumbing, not a user-facing break. The return-type change is on Dependencies.parents()/children() (the graph object), whose only callers are Table.parents()/Table.children() (table.py:379,411). The public, documented Table.parents()/children() return shape is unchanged — the diff only drops the alias-node .isdigit() branch; it still yields the same list[(name, props)]. Elements and plugins use Table.parents() / dj.Diagram, not conn.dependencies.parents() directly (confirmed: no such usage in Elements, and the graph accessor isn't documented API). Note too that the old dict-keyed return couldn't represent parallel edges — it silently dropped one — so the new list is strictly more correct. So I'm not tagging this breaking or adding a release-notes line; at most it's an internal-accessor note for anyone hacking on conn.dependencies.

2. Parallel-edges test — agreed, good gap to close. The single-renamed-FK cases pass, but the two-parallel-edges capability (-> Person.proj(person1=…) + -> Person.proj(person2=…)) has no direct test. I'll add a targeted case (assert parents() returns both Person entries, cascade/trace handle both, two distinct rendered edges) as a fast follow-up rather than block this merge.

3. Rebase — merging via rebase so history stays linear/clean.

Merging now.

[dimitri-yatsenko]

Follow-up for observation 2 (parallel-edges coverage) opened: #1514 — adds test_parallel_edges_to_same_parent using LocalSynapse (two renamed FKs to Cell), asserting two distinct parent edges across Table.parents()/children(), the dependency MultiDiGraph, and the rendered Diagram. Passes against MySQL 8.