Skip to content

feat(cli): add cochange, check (with SARIF) and dead-code graph analyses - #4544

Open
omsenjalia wants to merge 3 commits into
Graphify-Labs:v8from
omsenjalia:feat/graph-analysis-commands
Open

omsenjalia wants to merge 3 commits into
Graphify-Labs:v8from
omsenjalia:feat/graph-analysis-commands

Conversation

@omsenjalia

Copy link
Copy Markdown

What does this PR do?

It adds three read-only analysis commands over graph.json. Each lives in its own module with its own tests, and they share only the CLI wiring. They're bundled here at my request; I'm happy to split them into three PRs if reviewers prefer.

Command Module Answers
graphify cochange graphify/cochange.py Which files change together but share no graph edge (hidden coupling)? Which files are high-churn hubs?
graphify check graphify/rules.py Does the code break the architecture rules in graphify-rules.json? Exits non-zero and can emit SARIF for GitHub code scanning
graphify dead-code graphify/deadcode.py Which callables does no live code reach?

Prior art check: I searched open and closed PRs and issues for co-change/churn/hotspots, SARIF/architecture rules/forbidden imports, and dead code/reachability. Nothing overlaps. The closest items are #2950 (ingesting external OpenGrep findings, the opposite direction from check) and #2649 (removing some unused code by hand rather than adding an analysis).

All three outputs below come from running the commands on this repository.


graphify cochange

$ graphify cochange --top 3
Co-change analysis: 489 commits (11 bulk commits skipped)

Hidden coupling (change together, no graph edge):
  CHANGELOG.md <-> pyproject.toml  26 commits, confidence 0.87
  graphify/cache.py <-> graphify/extractors/engine.py  7 commits, confidence 0.33
  graphify/cli.py <-> graphify/extractors/engine.py  7 commits, confidence 0.19

Hotspots (commits x linked files):
  graphify/extract.py  101 commits x 263 linked files
  CHANGELOG.md  35 commits x 67 linked files
  graphify/build.py  20 commits x 102 linked files
  • Runs git log --no-merges --relative --name-only as an argv list, with no shell. --relative lines up a graph built from a subdirectory with git's paths, and core.quotepath=false keeps non-ASCII paths readable. The record separator uses git's %x1e escape because a raw control byte in argv breaks on Windows.
  • Corpus boundary: only files already present as a graph source_file are considered, so detection and .graphifyignore stay the single definition of the corpus.
  • Nothing is written back to graph.json: co-change is a statistical signal, never an edge.
  • Options: --since, --max-commits (default 500), --min-count (3), --max-files-per-commit (30), --top (20), --graph, --json.
  • Limitations: renames are not followed; bulk commits are skipped; any edge between two files counts as linked, including doc→code mentions.

graphify check

{
  "forbid": [
    {"from": "src/ui/*", "to": "src/db/*", "message": "UI must go through the service layer"},
    {"from": "src/core/*", "to": "src/plugins/*", "relations": ["imports_from"]}
  ],
  "no_import_cycles": true,
  "max_cycle_length": 5,
  "include_inferred": false
}
$ graphify check --sarif graphify.sarif
graphify check: 1 violation
  src/ui/view.py:12  [graphify/forbidden-dependency] src/ui/view.py -> src/db/repo.py (calls): UI must go through the service layer
  • Exit codes: 0 no violations, 1 violations, 2 unreadable graph or invalid rules file. --sarif PATH writes SARIF 2.1.0 that github/codeql-action/upload-sarif accepts. The rules file defaults to <root>/graphify-rules.json; use --rules to point elsewhere.
  • The rules file is validated at the trust boundary (unknown keys, wrong types). Forbid rules default to affected.DEFAULT_AFFECTED_RELATIONS minus defines. Cycle checks reuse analyze.find_import_cycles, so the deferred and type-only exclusions still apply.
  • Fail-closed by default: only EXTRACTED code-to-code edges count. "include_inferred": true opts in INFERRED edges; AMBIGUOUS edges and document mentions never count. Why: on graphify's own graph, a graphify/* → tests/* rule that counted every edge produced 45 bogus violations (31 INFERRED doc references and 12 INFERRED indirect_call resolutions). The default produces 0.
  • Limitations: globs are fnmatch on repo-relative POSIX paths, and * also matches /. Unindexed files and unresolved calls can't violate a rule. SARIF URIs are relative to the scanned root.

graphify dead-code

$ graphify dead-code
161 dead-code candidates (unreached callables; verify before deleting):
  graphify/build.py
     L1947  _norm_label()
     L1955  deduplicate_by_label()
  graphify/callflow_html.py
      L911  node_in_section()
  ...
  • Walks usage edges forward (calls, references, imports, inheritance, …) starting from non-callable code nodes, such as files and modules whose top-level code runs on import, and from callables shaped like entry points. A callable the walk never reaches is a candidate. Unlike a zero-callers scan, this also catches islands: functions that only call each other.
  • On graphify itself it surfaces deduplicate_by_label, prune_dangling_edges, _auto_follow_symlinks and node_in_section; a repo-wide grep confirms none of them has a caller. This PR doesn't delete them.
  • It stays conservative wherever the AST pass is known to leave calls unresolved. I found these cases by running it on this repo: the first naive version flagged about 650 symbols, mostly nested helpers. In the current version:
    • a live function keeps its nested functions live
    • a live class keeps its methods live
    • methods that override a supertype member (via affected._overridden_members), dunders, main and test* names, and anything in a test file are never candidates
    • documents never make code live
    • AMBIGUOUS edges still count as usage
  • Limitations: the output is candidates, never proof. Framework dispatch the AST can't see makes live code look dead: decorators, reflection, functions passed as values, packaging entry points. Public library API is reported, and dead methods of a live class are not.

Verification

uv run pytest tests/test_cochange.py tests/test_check_rules.py tests/test_dead_code.py tests/test_cli_help.py tests/test_affected_cli.py -q
# 49 passed
uv run pytest tests -q
# 8056 passed, 305 skipped, 17 failed - every one of the 17 also fails on unmodified v8 (same environment); none touch these modules
uv run ruff check graphify tests/test_cochange.py tests/test_check_rules.py tests/test_dead_code.py tests/test_cli_help.py
# All checks passed
uv run pyright graphify/cochange.py graphify/rules.py graphify/deadcode.py tests/test_cochange.py tests/test_check_rules.py tests/test_dead_code.py
# 0 errors

pyright graphify/cli.py reports "dispatch_command is too complex to analyze". That error is identical on unmodified v8 and not introduced here.

Determinism: every output is sorted on stable keys (file, line, label). No command writes graph or cache state.

An AI coding assistant (Claude) materially contributed to this change; disclosed via the Co-Authored-By trailer.

🤖 Generated with Claude Code

Three read-only commands over graph.json, each in its own module:

`graphify cochange` (graphify/cochange.py)
  Compares `git log --relative --name-only` co-change with graph edges.
  Hidden coupling = file pairs that changed together >= --min-count times
  but share no graph edge; hotspots = commits x linked files. Only files
  already in the graph are considered, so the corpus boundary stays the
  detection/.graphifyignore one. Bulk commits are skipped. git runs as an
  argv list; the record separator uses git's %x1e escape because a raw
  control byte in argv breaks on Windows.

`graphify check` (graphify/rules.py)
  graphify-rules.json declares forbidden dependencies (fnmatch globs,
  optional relation filter) and whether import cycles are allowed. Exit
  0 clean / 1 violations / 2 bad input; --sarif writes SARIF 2.1.0 for
  GitHub code scanning. Fail-closed: only EXTRACTED code-to-code edges
  count unless "include_inferred": true; AMBIGUOUS edges and document
  mentions never count (counting everything produced 45 bogus hits on
  graphify's own graph, the default 0). Cycles reuse find_import_cycles.

`graphify dead-code` (graphify/deadcode.py)
  Forward reachability over usage edges from non-callable code nodes and
  entry-point-shaped callables; unreached AST callables are candidates.
  Catches islands that only call each other. Conservative where the AST
  pass leaves calls unresolved: nested functions and methods inherit
  liveness from their parent, overriding methods / dunders / main /
  test names and test files are never candidates, documents never make
  code live, AMBIGUOUS edges still count as usage. On graphify itself it
  finds deduplicate_by_label, prune_dangling_edges, _auto_follow_symlinks
  and node_in_section, which have no callers.

Heuristic limits are documented in each module docstring.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Thanks for the pull request, @omsenjalia. A maintainer will review it soon.

Want to talk it through while it is in review? Come join us on our Discord server. For longer-form discussion there is also GitHub Discussions.

A couple of things that speed up review: make sure the test suite passes on Python 3.10 and 3.13, and that the change keeps extraction deterministic.

@graphify-labs graphify-labs Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 3 advisory finding(s) below merit a look before merge.

Not checked: tests were not run; formal verification proved 0 of 2 changed function(s) (1 sampled, not proven, 1 not verified).

Formal verification. PR-changed functions: 1/2 verified (0 proven, 1 may-equivalent, 0 distinguished) · 1 not verified (1 vacuous).

Not verified on this run: dispatch\_command (vacuous: never exercised).


Graphify review — findings

Adds three graph-analysis subcommands: cochange compares git history against graph edges to surface hidden coupling and churn hotspots, check enforces architecture rules from graphify-rules.json (exit 1 on violations, 2 on an unreadable graph or rules, optional SARIF output), and dead-code lists callables no live code reaches. The check path only catches ValueError and RuntimeError from loading rules, and cochange leans on git log plus _norm path matching, so an unreadable rules file, a missing git binary, or Windows-style source_file paths can break those commands outside their documented exits. Forbid rules in check_rules do not account for edge direction on undirected graphs, so a gate that fails the build can flag the wrong side of a dependency, and the new help text in _run_cli documents all three commands.

Worth a look

  • Forbid rules ignore edge direction on undirected graphs — graphify/rules.py:86 · Escalate · high
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • check command lets unreadable rules OSErrors escape instead of exiting 2 — graphify/cli.py:1860 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • Windows-style source_file paths are not normalized to POSIX — graphify/cochange.py:39 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review

Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 496 functions depend on the 174 functions this change touches.

Health — this change adds coupling hotspots:

  • worse: dispatch_command() — 5 callers, 143 callees
  • worse: main() — 113 callers, 3 callees
  • new: dead_code_candidates() — 6 callers, 4 callees
  • new: cochange_analysis() — 6 callers, 3 callees

Verification — 496 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 496 function(s) in the blast radius were not formally verified this run

Test selection

Test selection

40 of 435 test file(s) selected (9%) via static blast radius.

  • tests/test_affected_cli.py — impact
  • tests/test_agents_platform.py — impact
  • tests/test_check_rules.py — impact, changed-test
  • tests/test_cli_export.py — impact
  • tests/test_cli_help.py — changed-test
  • tests/test_cloud_cta.py — impact
  • tests/test_cochange.py — impact, changed-test
  • tests/test_codebuddy.py — impact
  • tests/test_compact_graph_json.py — impact
  • tests/test_dead_code.py — impact, changed-test
  • tests/test_dedup_shrink_refuses_force_write.py — impact
  • tests/test_devin.py — impact
  • tests/test_explain_cli.py — impact
  • tests/test_explain_same_file_tie.py — impact
  • tests/test_extract_cli.py — impact
  • tests/test_global_add_tag_inference.py — impact
  • tests/test_god_nodes_cli.py — impact
  • tests/test_hollow_chunks_arm_shrink_guard.py — impact
  • tests/test_hook_guard_token_match.py — impact
  • tests/test_hook_out_of_project_paths.py — impact
  • tests/test_hook_strict.py — impact
  • tests/test_incomplete_build_guard.py — impact
  • tests/test_install.py — impact
  • tests/test_install_references.py — impact
  • tests/test_install_version_warning.py — impact
  • tests/test_lookup_exit_codes.py — impact
  • tests/test_merge_chunks_validation.py — impact
  • tests/test_multigraph_diagnostics.py — impact
  • tests/test_no_dedup_flag.py — impact
  • tests/test_opencode_plugin_refresh.py — impact
  • tests/test_partial_cache.py — impact
  • tests/test_path_cli.py — impact
  • tests/test_query_cli.py — impact
  • tests/test_query_induced_edges.py — impact
  • tests/test_query_seeds.py — impact
  • tests/test_skill_auto_refresh.py — impact
  • tests/test_skill_version_warning.py — impact
  • tests/test_stale_prune.py — impact
  • tests/test_truncation_full_answer.py — impact
  • tests/test_unverified_semantic_shrink.py — impact

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

Docs that may be stale (advisory)

…and 10 more.

Formal verification

No difference found (not proven): No behavior difference found in \_run\_cli (not a proof).

The verifier ran both versions of \_run\_cli on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify dispatch\_command.

The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: no capturable inputs from the test suite; property tier: not verifiable: all 40 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)

· 2 grounded finding(s) anchored inline below; 2 more finding(s) on lines outside this diff (see the check run).

Comment thread graphify/cochange.py
Comment thread graphify/deadcode.py
omsenjalia and others added 2 commits October 18, 2026 18:43
- cochange: normalize backslashes before PurePath so a Windows-written
  source_file matches git's POSIX paths when run on POSIX
- cochange: a missing git binary raises RuntimeError (exit 1 with a
  message) instead of an uncaught FileNotFoundError
- check: an unwritable --sarif path exits 2 instead of a traceback
- tests pin rules-file-is-a-directory -> exit 2, and that an undirected
  graph.json keeps dependency direction (to_json writes _src/_tgt into
  source/target; load_graph reads it back directed)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
check_rules matched `from` against G.edges()' first endpoint. build()
returns an undirected nx.Graph by default, whose edges iterate in node
insertion order, so a ui -> db call could come back as (db, ui) and the
violation was silently missed when check_rules was called on an in-memory
graph. Use the _src/_tgt attributes build() records (same idiom as
export.py). Not a both-orders match: that would flag the allowed reverse
dependency. The CLI path was unaffected (graph.json stores the true
direction and load_graph reads it directed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant