Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ Each stage is a single function in its own module. They communicate through plai
| `security.py` | validation helpers | URL / path / label → validated or raises |
| `validate.py` | `validate_extraction(data)` | extraction dict → raises on schema errors |
| `serve.py` | `start_server(graph_path)` | graph file path → MCP stdio server |
| `cluster_graph.py` | `build_cluster(cluster_dir)` | cluster.json + member graph.json files → one linked cross-repo graph (not community detection — that's `cluster.py`) |
| `cluster_cli.py` | `cmd_cluster(argv)` | `graphify cluster <sub>` CLI for cluster_graph.py |
| `watch.py` | `watch(root, flag_path)` | directory → writes flag file on change |
| `benchmark.py` | `run_benchmark(graph_path)` | graph file → corpus vs subgraph token comparison |

Expand Down
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@

Full release notes with details on each version: [GitHub Releases](https://github.com/safishamsi/graphify/releases)

## Unreleased

- New: `graphify cluster` — cluster graphs link multiple repos into one connected graph. A committable `cluster.json` names member repos by git URL (paths resolve per machine via a local override, the spec's path hint, or origin-remote auto-discovery) and declares the cross-repo contracts auto-detection can't see (`api_call`, `shared_resource`, `mirrored_file`, `depends_on`, `references`, each optionally `direction: "both"`); `cluster build` composes member graphs under `tag::` namespaces into a directed graph (reusing the global-graph external dedup) and resolves declared links into `EXTRACTED` edges, writing a standard `graphify-out/graph.json` so query/path/explain/affected/export work unchanged; `cluster check` dry-runs resolution for CI.
- New: cluster member back-references. `cluster build` writes a portable `cluster-ref.json` into each member's `graphify-out/` recording every membership (cluster name + git URL + roster, no absolute paths; `--no-refs` to skip, `cluster remove` drops only its own entry). Inside a member repo, `query`/`path`/`explain`/`affected` accept `--cluster` (or `--cluster NAME`), no-match failures note the membership, and the search-nudge hook + installed skills surface it to assistants; marker fields are sanitized before reaching any assistant-facing output. Without the cluster locally, `--cluster` explains exactly what to clone and build.
- New: `auto_links.packages` connects a member's direct package dependencies to their unique provider in another member (ambiguous matches are warned and skipped; declared links take precedence). Package-manifest nodes now carry normalized `package_key`/`dependency_keys` identities.
- New: `affected` traverses `calls_api`/`mirrors` relations by default, so impact analysis crosses repo boundaries; when several parallel relations match a hop, all are listed.
- New: multigraph mode — `graphify extract --multigraph` (and `graph_mode: "multi"` clusters) builds keyed MultiDiGraphs that preserve parallel relations end to end: build/merge, incremental updates and watch, community detection (parallel weights aggregate), graph diff, MCP/query output, and GraphML/Cypher/Canvas exports. The format persists across rebuilds (including `--force`); `--no-multigraph` converts back with a warning.
- Fix: `graphify watch` now passes the project root through to the graph builder, aligning watch rebuilds with `graphify build`'s root-relative `source_file` paths (#932).

## 0.9.25 (2026-07-22)

- License: the project is now licensed under the Apache License, Version 2.0 (previously MIT). Apache 2.0 adds an explicit patent grant and patent-retaliation clause and explicit contribution terms. Contributions made before the relicensing were submitted under MIT and remain available under those terms; the original MIT license text is retained in `LICENSE-MIT` and referenced from `NOTICE`.
Expand Down
93 changes: 93 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -383,6 +383,7 @@ graphify export callflow-html # Mermaid architecture/call-flow HTML (auto-r

graphify hook install # auto-rebuild on git commit
graphify merge-graphs a.json b.json # combine two graphs
graphify cluster build # cluster graph: link multiple repos with real cross-repo edges

graphify prs # PR dashboard: CI state, review status, worktree mapping
graphify prs 42 # deep dive on PR #42 with graph impact
Expand Down Expand Up @@ -436,6 +437,84 @@ graphify-out/cost.json # local only

---

## Cluster graphs (multi-repo)

A **cluster graph** links several repos' graphs into one connected graph with real cross-repo edges — for ecosystems where the coupling lives in contracts a single-repo scan can't see: service A calls service B's HTTP API via an env-var URL, two repos share a database table, a wire-format type file is copy-mirrored between a client and a worker. (`merge-graphs` and `global` union graphs side by side; a cluster also *connects* them. For community detection on a single graph, see `cluster-only`.)

A cluster is a directory with a `cluster.json` spec:

```json
{
"schema_version": 1,
"name": "my-stack",
"graph_mode": "multi",
"members": [
{"tag": "web", "url": "https://github.com/org/web", "path": "../web"},
{"tag": "worker", "url": "https://github.com/org/worker"}
],
"links": [
{
"type": "api_call",
"name": "ingest-api",
"from": {"repo": "web", "file": "src/lib/api-client.ts"},
"to": {"repo": "worker", "file": "src/index.ts"}
},
{
"type": "shared_resource",
"kind": "db_table",
"name": "events.pings",
"referents": [
{"repo": "web", "label": "pingSync"},
{"repo": "worker", "file": "src/sync.ts"}
]
}
],
"defaults": {"on_missing": "warn"},
"auto_links": {"externals": true, "packages": true}
}
```

`graph_mode` is `simple` by default. Set it to `multi` and extract members with
`graphify extract . --multigraph` to retain several relations between the same
nodes. The multigraph format persists across rebuilds (including `--force`);
`graphify extract . --no-multigraph` converts back, with a warning that
parallel relations collapse. YAML specs remain available when PyYAML is
installed, but initialization and documentation use JSON consistently.

Node selectors are `{repo, file|label|id}` — `file` suffix-matches `source_file` (preferring the file node), `label` matches exactly then case-insensitively, `id` matches the member-local node id. You never write raw graph ids. Externals (library nodes with no `source_file`) are deduplicated cluster-wide, so a `label` selector for one resolves under any member's tag regardless of spec order.

Direction: `from` is the dependent side (`from` depends on / calls / copies `to`), matching how `imports` and `calls` edges point. So `graphify affected <changed-node>` seeded on a link's `to` side reports the `from` side — "I changed the worker's payload type; the web client is affected." `direction: "both"` on a link (e.g. a mirrored file kept in sync by hand in both directions) materializes the reverse edge too, so `affected` works from either endpoint; the declared link still owns the node pair in simple mode.

Because members are identified by `url`, the spec commits cleanly and works on any machine: paths resolve via a gitignored `cluster.local.json` override (`graphify cluster locate <tag> <path>`), then the spec's `path` hint, then auto-discovery — scanning sibling directories for a checkout whose `origin` remote matches. A resolved checkout whose origin *doesn't* match the declared url gets a warning, so a same-named directory of the wrong repo can't sneak in.

```bash
graphify cluster init ~/clusters/my-stack --name my-stack
graphify cluster add ../web && graphify cluster add ../worker
# ...declare links in cluster.json, then:
graphify cluster build
cd ~/clusters/my-stack
graphify query "how does a ping reach the database?" # all existing commands work
graphify affected "payload.ts" # impact traverses calls_api/mirrors across repos
graphify path "api-client" "index.ts"
```

The build composes each member's `graphify-out/graph.json` under a `tag::` namespace, dedups external-library nodes by label across members (same behavior as the global graph), resolves the declared links into `EXTRACTED`-confidence edges, and writes a standard `graphify-out/graph.json` plus a `CLUSTER_REPORT.md` documenting every resolved/skipped link. Rebuilds are incremental-aware: unchanged members and spec skip the rebuild entirely. `graphify cluster check` dry-runs the whole thing (exit 1 on errors) — useful in CI to catch selector drift when a member repo refactors.

In simple mode, `cluster check` and `cluster build` reject a declared link that would overwrite another relation on the same pair. Multi mode preserves every keyed relation. `auto_links.packages` connects direct package dependencies to a unique provider in another member repo; external, same-repo, and ambiguous dependencies are skipped, and declared links take precedence.

Each member needs its own graph first (`graphify extract .` in that repo); `build` names exactly which members are missing one.

**Member back-references.** `cluster build` also writes a portable `cluster-ref.json` into each member's `graphify-out/` (skip with `--no-refs`; `cluster remove` cleans it up). Since `graphify-out/` is committed, the marker travels with each member repo: it records the cluster's name and git URL, this member's tag, and the full member roster — no absolute paths. Inside a member repo:

- `graphify query/path/explain/affected --cluster` selects the only membership; `--cluster NAME` selects one explicitly when the repo belongs to several clusters. Both forms are mutually exclusive with `--graph`.
- When a lookup on the local graph comes up empty, the failure message notes the repo is a cluster member and suggests `--cluster` — so an assistant hitting "No node matching 'verifyJwt'" learns the answer may live one repo over.
- If the cluster isn't available on a machine, `--cluster` fails safely with instructions: clone the marker's `cluster_url` and run `graphify cluster build` there (or how to create the cluster when no remote is recorded).
- The search-nudge hook and the installed skill mention cluster membership too, so LLM assistants are aware without running anything.

Note the asymmetry: member markers are committed and travel, while the **cluster directory's own `graphify-out/` stays gitignored** (each machine builds its own composed graph). The marker stores all cluster memberships and each build updates only its own entry.

---

## Using the graph directly

```bash
Expand Down Expand Up @@ -746,11 +825,25 @@ graphify export callflow-html --max-sections 8 # cap generated architecture
graphify export callflow-html --output docs/arch.html
graphify export callflow-html ./some-repo/graphify-out

graphify extract . --multigraph # opt in to keyed parallel relations (sticky across rebuilds)
graphify extract . --no-multigraph # convert back to a simple graph (collapses parallels)

graphify global add graphify-out/graph.json --as myrepo # register a project graph into ~/.graphify/global-graph.json
graphify global remove myrepo # remove a project from the global graph
graphify global list # show all registered repos + node/edge counts
graphify global path # print path to the global graph file

graphify cluster init ~/clusters/my-stack --name my-stack # start a cluster (multi-repo linked graph)
graphify cluster add ../frontend # add a member repo (url derived from its origin remote)
graphify cluster add https://github.com/org/backend --as api # or add by URL; path resolved per machine
graphify cluster locate api ~/work/backend # machine-local checkout override (cluster.local.json)
graphify cluster build # compose member graphs + resolve declared links
graphify cluster check # validate the spec + dry-run link resolution (CI-friendly)
graphify cluster status # member resolution + staleness vs last build
graphify query "..." --cluster # from inside a member repo: query the cluster graph
graphify query "..." --cluster my-stack # select by name when the member belongs to several
graphify path "A" "B" --cluster # (also explain/affected; uses graphify-out/cluster-ref.json)

graphify prs # PR dashboard: CI, review, worktree, graph impact
graphify prs 42 # deep dive on PR #42
graphify prs --triage # AI triage ranking (auto-detects backend from env)
Expand Down
17 changes: 14 additions & 3 deletions graphify/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -512,8 +512,10 @@ def _run_cli() -> None:
print(" --purge also delete graphify-out/ directory")
print(" path \"A\" \"B\" shortest path between two nodes in graph.json")
print(" --graph <path> path to graph.json (default graphify-out/graph.json)")
print(" --cluster [NAME] query this repo's cluster graph instead (from a member repo)")
print(" explain \"X\" plain-language explanation of a node and its neighbors")
print(" --graph <path> path to graph.json (default graphify-out/graph.json)")
print(" --cluster [NAME] query this repo's cluster graph instead (from a member repo)")
print(" diagnose multigraph report same-endpoint edge collapse risk in graph.json")
print(" --graph <path> path to graph/extraction JSON")
print(" (default graphify-out/graph.json)")
Expand Down Expand Up @@ -558,10 +560,12 @@ def _run_cli() -> None:
print(" --context C explicit edge-context filter (repeatable)")
print(" --budget N cap output at N tokens (default 2000)")
print(" --graph <path> path to graph.json (default graphify-out/graph.json)")
print(" --cluster [NAME] query this repo's cluster graph instead (from a member repo)")
print(" affected \"X\" reverse traversal to find nodes impacted by X")
print(" --relation R edge relation to traverse in reverse (repeatable)")
print(" --depth N reverse traversal depth (default 2)")
print(" --graph <path> path to graph.json (default graphify-out/graph.json)")
print(" --cluster [NAME] query this repo's cluster graph instead (from a member repo)")
print(" god-nodes list the most connected nodes (architectural hubs)")
print(" --top N how many to show (default 10)")
print(" --graph <path> path to graph.json (default graphify-out/graph.json)")
Expand Down Expand Up @@ -611,6 +615,9 @@ def _run_cli() -> None:
print(" --google-workspace export .gdoc/.gsheet/.gslides shortcuts via gws before extraction")
print(" --no-gitignore ignore .gitignore and .git/info/exclude (prioritizes .graphifyignore)")
print(" --no-cluster skip clustering, write raw extraction only")
print(" --multigraph keyed MultiDiGraph output: keep parallel relations between")
print(" the same node pair (persists across rebuilds)")
print(" --no-multigraph convert a multigraph back to simple (collapses parallel relations)")
print(" --code-only index code (local AST, no API key) and skip doc/paper/image files")
print(" --postgres DSN extract schema from a live PostgreSQL database")
print(" maps tables, views, functions + FK relationships;")
Expand All @@ -623,6 +630,9 @@ def _run_cli() -> None:
print(" global remove <tag> remove a repo's nodes from the global graph")
print(" global list list repos in the global graph")
print(" global path print path to the global graph file")
print(" cluster <subcommand> cluster graphs: link multiple repos into one connected graph")
print(" (init/add/remove/locate/build/check/status; `graphify cluster` for details)")
print(" (community detection on a single graph is `cluster-only`, above)")
print(" benchmark [graph.json] measure token reduction vs naive full-corpus approach")
print(" export callflow-html emit Mermaid-based architecture/call-flow HTML")
print(" hook install install post-commit/post-checkout git hooks (all platforms)")
Expand Down Expand Up @@ -700,9 +710,10 @@ def _run_cli() -> None:
# Universal help guard: -h/--help/-? anywhere after the command shows help
# and stops — prevents flags from silently triggering destructive subcommands
# (e.g. "cursor install --help" was silently installing into Cursor, #821).
# Exempt: free-text commands (user string may contain these tokens), and
# "install"/"uninstall" which have their own per-subcommand help handlers.
_FREE_TEXT_CMDS = {"query", "explain", "path", "save-result", "install", "uninstall"}
# Exempt: free-text commands (user string may contain these tokens),
# "install"/"uninstall" which have their own per-subcommand help handlers,
# and "cluster" whose dispatcher answers help tokens with its own USAGE.
_FREE_TEXT_CMDS = {"query", "explain", "path", "save-result", "install", "uninstall", "cluster"}
if cmd not in _FREE_TEXT_CMDS and any(a in {"-h", "--help", "-?"} for a in sys.argv[2:]):
print(f"Run 'graphify --help' for full usage.")
return
Expand Down
Loading