feat(cluster): cluster graphs — link multiple repos into one connected graph (+ multigraph mode)

[StuartMVG]

Summary

This PR adds two related graph capabilities:

• Cluster graphs: compose multiple repositories into one directed graph with explicit cross-repository edges.
• Multigraph mode: preserve parallel relations between the same pair of nodes throughout extraction, analysis, queries, and exports.

It also makes member repositories cluster-aware, so existing query, path, explain, and affected commands can discover and use their cluster graph.

Unlike merge-graphs and the global graph, which union repositories side by side, a cluster graph explicitly models connections such as API calls, shared resources, mirrored files, dependencies, and references.

Quick start

Each member repository must have an existing graph:

cd ~/work/frontend
graphify extract .

cd ~/work/backend
graphify extract .

Create and build the cluster:

graphify cluster init ~/clusters/my-stack --name my-stack
cd ~/clusters/my-stack

graphify cluster add ~/work/frontend --as frontend
graphify cluster add ~/work/backend --as backend

Declare links in cluster.json:

{
  "links": [
    {
      "type": "api_call",
      "name": "events-api",
      "from": {
        "repo": "frontend",
        "file": "src/lib/events-client.ts"
      },
      "to": {
        "repo": "backend",
        "file": "src/routes/events.ts"
      }
    }
  ]
}

Then validate and build:

graphify cluster check
graphify cluster build

The resulting graphify-out/graph.json works with existing commands:

graphify query "how are events submitted?"
graphify affected "src/routes/events.ts"
graphify path "events-client" "events-route"

From inside a member repository:

graphify query "how are events submitted?" --cluster
graphify query "..." --cluster my-stack

Cluster graphs

A committable cluster.json defines members and cross-repository links.

Git URLs are used as portable member identities when available. Checkouts resolve per machine through:

1. A gitignored cluster.local.json override
2. A path hint in the spec
3. Configured search roots
4. Origin-matched sibling auto-discovery

URL-less repositories remain supported through path hints and local overrides.

Members are composed under tag:: namespaces into a directed graph. Link selectors use {repo, file|label|id}:

• file suffix-matches source_file
• label matches exactly, then case-insensitively
• id accepts a member-local node ID

Users do not need to write the namespace-prefixed IDs found in the composed graph.

Supported links include:

• api_call
• shared_resource
• mirrored_file
• depends_on
• references

direction: "both" materializes a real reverse edge rather than storing direction as metadata only.

auto_links.packages connects manifest dependencies to their unique provider in another member repository. Ambiguous providers are reported and skipped, and declared links take precedence.

cluster check performs the same spec, member, and selector validation as a build without writing output. It exits nonzero on errors, making it suitable for CI.

Member awareness

cluster build writes a portable graphify-out/cluster-ref.json marker into each member repository. This can be disabled with --no-refs, and cluster remove removes only its own membership entry.

The marker contains no absolute paths and can travel with a clone. It enables:

• query, path, explain, and affected with --cluster [NAME]
• Membership hints when a local lookup has no result
• Cluster awareness in MCP responses, installed skills, and the search-nudge hook
• Multiple cluster memberships per repository

Missing or corrupt markers never break ordinary queries or hooks.

Markers are treated as untrusted input. Automatically injected hook context uses static guidance plus a bounds-checked member count; user-invoked diagnostics sanitize and bound marker-controlled text.

Writing markers into member worktrees is intentionally opt-out. cluster init separately adds the cluster directory's generated graphify-out/ to .gitignore.

Multigraph mode

graphify extract . --multigraph

Multigraph builds use keyed MultiDiGraph edges so parallel relations between the same nodes survive through:

• Initial and incremental builds
• Watch rebuilds
• Cluster and global composition
• Community detection
• affected
• Graph diff
• CLI and MCP queries
• GraphML, Cypher, and Canvas exports

Edge keys are stable and content-derived. Cypher exports include graphify_key in MERGE operations to remain idempotent.

The graph format is sticky across rebuilds, including --force. Converting back requires an explicit command:

graphify extract . --no-multigraph

That conversion warns that parallel relations will collapse. Conversion in either direction carries the existing graph forward.

Simple graphs retain their existing edge order. A content tiebreak is applied only where multiple simple edges have the same (source, target, relation) identity, preventing run-to-run changes in which duplicate survives.

Community detection aggregates parallel edge weights before running algorithms that require a simple graph.

Impact analysis

affected now traverses the cluster-specific calls_api and mirrors relations by default.

depends_on remains opt-in to avoid changing the established default traversal behavior for existing single-repository graphs.

Parallel matching relations are all surfaced when running against a multigraph.

Failure behavior

Validation occurs before writing cluster output. Actionable errors are produced for:

• Missing or corrupt member graphs
• Empty clusters
• A cluster attempting to include itself
• Conflicting cluster names
• Invalid selectors or link directions
• Simple-mode links that would overwrite another relation on the same node pair
• Missing CLI flag values and unknown options

External-library selectors and community IDs are normalized across members so results do not depend on member order or accidentally merge unrelated communities.

Directed composition is load-bearing: converting member graphs through an undirected intermediate can reverse caller/callee endpoints based on node insertion order. Regression coverage pins this behavior.

Review map

Area	Primary files	Review focus
Shared graph composition	graphify/build.py, graphify/global_graph.py	Graph loading and namespace-prefixed merging
Cluster specification and CLI	graphify/cluster_graph.py, graphify/cluster_cli.py	Validation, member resolution, and commands
Cluster composition	graphify/cluster_graph.py	Directed composition, selectors, links, and auto-links
Community detection	graphify/cluster.py (unrelated to cluster specs, despite the name)	Parallel-edge weight aggregation only
Member awareness	graphify/cluster_ref.py, graphify/cli.py, graphify/serve.py	Marker lifecycle, --cluster, hints, and sanitization
Multigraph semantics	graphify/build.py, graphify/analyze.py, graphify/affected.py	Stable keys, conversion, and parallel-edge handling
Export behavior	graphify/export.py	GraphML, Cypher, and Canvas round-trips
Package auto-linking	graphify/manifest_ingest.py	Normalized package and dependency keys
Assistant integration	graphify/skill*.md, tools/skillgen/	Generated skill text and drift validation
Regression coverage	tests/test_cluster*.py, tests/test_multigraph_build.py	End-to-end and failure-path behavior

The first commit centralizes graph JSON loading and namespace-prefixed merging for merge-graphs, the global graph, and clusters. It is intended to preserve successful-path behavior while normalizing malformed-input handling into clean CLI errors.

Additional fix

watch rebuilds now pass root=project_root to build_from_json, aligning watch-generated source_file paths with those produced by graphify build (extends the #932 path normalization, which previously covered only the build/extract path, to watch rebuilds).

This fix is independent of clusters and can be split into a separate PR if maintainers prefer stricter scope.

Related issues and discussions

• Addresses how to integrate with multi repo project #1177: that issue asks how to query a project composed of separate backend, frontend, documentation, and module repositories from one workspace. A cluster provides the unified graph and query surface for that layout. Automatic repository discovery and the issue's separate context-mode integration question remain out of scope, so this PR does not close it.
• Partially addresses Cross-service edges: bridge client ↔ backend across the network boundary (API-contract matching) #1595: declared api_call links connect client call sites to backend handlers, and affected traverses those links across repository boundaries. The issue's proposed automatic framework-specific route extraction, API-contract matching, and dangling-contract detection remain future work.
• Implements the linked-project portion of Hierarchical context layer on top of Graphify — multi-project navigation with conversation indexing #425: clusters preserve per-project graphs while adding a composed graph and explicit cross-project connections. Conversation ingestion, remote source support, compact reports, and automatic workspace indexing from that broader proposal remain out of scope.
• Implements the core linked-workspace concept from Discussion #117: independently built corpora can be registered as members, queried together, and connected through declared links. The discussion's proposed automatic semantic connect operation is not included.
• Builds on Discussion #528: that release introduced merge-graphs as a namespaced cross-repository union. Cluster graphs retain namespacing while adding explicit edges between members.

Direction-preservation coverage follows the regression history in #760, #1061, and #1174. Those issues are already closed; they are cited as design and regression context rather than as issues resolved by this PR.

Testing

At 4ade974:

• 3,755 passed, 3 skipped
• 135 new test functions, plus parametrized cases
• Skill-generation drift checks pass:
  • --check
  • --audit-coverage
  • --schema-singleton
  • --monolith-roundtrip
  • --always-on-roundtrip

Coverage includes spec validation, directed composition, selector resolution, member-order independence, bidirectional links, marker lifecycle and sanitization, all four --cluster query surfaces, MCP and hook hints, stable multigraph keys, format conversion in both directions, mixed-mode composition, affected traversal, and exporter round-trips.

Related PRs

Supersedes the earlier stacked drafts #2120, #2121, and #2122, consolidating the same work into a single review.

[kitsupanic]

Additional real-world validation for the multigraph mode from an Expo / React Native TypeScript corpus on graphifyy 0.9.25.

A fresh code-only extraction reports exactly two same-endpoint relation collisions:

1. app_add_scan -> src_components_scanscreen

   • imports_from at app/add/scan.tsx:L1
   • re_exports at app/add/scan.tsx:L3

2. src_components_notificationlifecycle_notificationlifecycle -> src_components_notificationlifecycle_opennotificationdestination

   • calls at src/components/NotificationLifecycle.tsx:L45
   • indirect_call at src/components/NotificationLifecycle.tsx:L63

Both relations are semantically useful; a simple graph retains only one edge per pair. This corpus is a compact additional regression case for --multigraph, query/explain output, and stable round-tripping. I can test the PR branch against the full corpus if useful.

[StuartMVG]

Additional real-world validation for the multigraph mode from an Expo / React Native TypeScript corpus on graphifyy 0.9.25.

A fresh code-only extraction reports exactly two same-endpoint relation collisions:

1. app_add_scan -> src_components_scanscreen

   • imports_from at app/add/scan.tsx:L1
   • re_exports at app/add/scan.tsx:L3

2. src_components_notificationlifecycle_notificationlifecycle -> src_components_notificationlifecycle_opennotificationdestination

   • calls at src/components/NotificationLifecycle.tsx:L45
   • indirect_call at src/components/NotificationLifecycle.tsx:L63

Both relations are semantically useful; a simple graph retains only one edge per pair. This corpus is a compact additional regression case for --multigraph, query/explain output, and stable round-tripping. I can test the PR branch against the full corpus if useful.

Thank you. These are excellent real-world examples of the exact information-loss case --multigraph is intended to solve. Both relation pairs map to distinct keyed edges on this branch, and the query/explain paths display parallel relations independently.

I’ve added a focused explain regression test based on your calls + indirect_call example. It verifies that both edges are rendered with their respective relation sites (L45 and L63).

If you’re willing to test the PR branch against the full corpus, that would be very useful. Please test at 36ecb25 so the results are comparable. The most valuable checks would be:

• the resulting graph has "multigraph": true, and both relation pairs appear as separately keyed parallel edges;
• query and explain display both relations with their respective source sites;
• an unchanged rebuild and --force preserve the multigraph format, edge keys, and edge count;
• after a harmless edit to one affected file and re-extraction, neither relation is duplicated nor collapsed.

No need to share the corpus. The output snippets and before/after edge counts would be enough. Thanks again for the concrete case.