fix(graph): tie-breaks leaked os.walk order into node paths and wikilinks

[sturlese]

Tenth iteration of the autonomous bughunt sweep. A determinism claim the code did not keep.

Bug

docs/pipeline/graph.md promises "Output is deterministic for a given input" — and the input is the pages' content, not the order the filesystem happened to yield them in. Three tie-breaks read a Counter, which resolves ties by first-encounter, and build_graph fills those Counters straight from os.walk:

ms = [("Initech","company",1), ("Initech","organization",1)]
as walked -> type=company       slug=entities/company/initech
reversed  -> type=organization  slug=entities/organization/initech

That value is the node's file path — entities/<type>/<slug> — and therefore every [[wikilink]] pointing at it. build_graph's stale-removal then deletes the other one. So adding an unrelated page to brain-md could relocate an existing entity and rewrite links across the corpus.

A type tie is not exotic: two documents mentioning the same company, two LLM guesses at its type.

Two more of the same shape:

• Title tied identically — equal case-ness, equal length, equal count fell through to walk order (Acme Corp vs Acme GmbH).
• Alias order decided which aliases survive render_node's aliases[:8].

Fix

All three now state a rule rather than relying on stability:

choice	rule
entity type	most-mentioned, then alphabetical
aliases	most-mentioned, then alphabetical
title	non-ALL-CAPS → shortest → most-mentioned → alphabetical

Ordering aliases by frequency has a second benefit: the [:8] truncation now keeps the aliases that actually appear in the corpus rather than the ones the walk reached first.

docs/pipeline/graph.md states the rules next to the determinism claim it was making — and, after the gate caught me over-claiming, also names the one thing that is not guaranteed: slugs are assigned in key order, so adding an entity whose key sorts earlier can still relocate an existing node (Foo & Bar and Foo + Bar both slugify to foo-bar). Links are rewritten consistently, so nothing dangles. I had checked that loop and found it stable under reordering — stability under adding a key is a different property, and I asserted the stronger one without testing it.

Migration

A deployed corpus that has a genuine type tie will relocate that node on its next build. Verified end to end: the node moves, the old one is removed, and every wikilink is rewritten consistently — no orphan and nothing dangling. One cosmetic residue: build.py uses os.remove and never rmdir, so an empty entities/<old-type>/ directory is left behind.

Eval risk — checked, not assumed

golden.json pins graph output, so I verified the change cannot move it rather than reasoning about it:

mention names with a TYPE tie in the golden corpus: none
distinct types seen: ['organization']

No ties and a single type, so the pinned entities/organization/globex.md is unreachable by this change. placement: pages at expected paths 8/8, placement: frontmatter contract 30/30, the graph metric and the full 25/25 scorecard all unchanged.

Test

Both new tests confirmed to fail with the source change stashed.

• test_output_is_independent_of_mention_order — the property itself: build_entities(ms) == build_entities(reversed(ms)) for a type tie, a title tie, and an alias set.

• test_tie_breaks_are_alphabetical_not_incidental — that the tie-break is a stated rule, not merely stable, so a future change cannot make it deterministic-but-arbitrary.

• test_build_graph_output_is_byte_identical_under_any_walk_order — the claim asserted on the whole output tree (node pages, rewritten docs, wikilinks), not just at build_entities.

• test_title_prefers_non_all_caps_over_shortest — pins the ordered title rule this PR documents.

Three tests were passing for the wrong reason, mine included. test_dominant_type_wins picked a case where alphabetical and frequency agree, so an alphabetical-only rule passed it. My own end-to-end determinism test initially caught only one of the three reverts — the title tie needs its spellings in separate files (within one file the frontmatter fixes the order), and the alias order needs three-plus spellings because render_node excludes the title, leaving only one alias with two names. And the title rule's top two keys were unpinned entirely: reordering isupper and len passed all 50 tests, so this PR documented a rule nothing checked. All three now discriminate; the reviewer's independent mutation set is fully killed.

graph 51 (+5) · clean 256 · corpus 48 · slack 13 · fetch 33 · answer 66 · benchmark 3 — bare pytest (CI mode). ruff clean; golden scorecard 25/25.

[sturlese]

Gate verdict

VERDICT: WEAKENED. The fix itself held up under heavy attack; what did not hold up were two things this PR says. Both corrected in 7517f68.

The claim, verified hard

The reviewer ran the real build_graph with os.walk monkeypatched across forward, reverse and 6 seeded shuffles, diffing the entire output tree byte for byte, then fuzzed 40 randomised corpora × 7 walk orders including CJK/accented names, None types, registries and colliding slugs:

BRANCH: 40 random corpora x 7 walk orders: 0 order-dependent differences
MAIN:   40 random corpora x 7 walk orders: 185 order-dependent differences

With a paired control on main showing real relocations (entities/product/widget-systems.md → entities/software/…), so the harness demonstrably sees the bug. Golden graph output byte-identical between main and branch; scorecard 25/25. min never disagrees with sorted(...)[0], and an empty types Counter is unreachable because g["types"][typ or "other"] += count always creates a key.

The doc sentence I added was false

"adding an unrelated page never relocates an existing entity"

Reproduced on the branch:

before (only 'Foo + Bar')        : entities/company/foo-bar
after  (added 'Foo & Bar' pages) : foo-bar -> 'foo & bar'
                                   'foo + bar' MOVED to foo-bar-2

Two different keys can slugify identically, slugs are assigned in key order, and & (0x26) sorts before + (0x2b) — so a genuinely unrelated entity took the bare slug and the pre-existing one's wikilinks were rewritten corpus-wide. That is the same harm this PR calls the bug.

Worth being precise about my error: I did check the slug-collision loop and correctly found it stable — under reordering. Stability under adding a key is a different property, and I asserted the stronger claim without testing it. The doc now states what is actually guaranteed and names the caveat.

Also corrected: "alphabetical" is codepoint order, which for equal-length names systematically prefers the more-ASCII spelling (IPhone 15 over iPhone 15, Inigo Ruiz over Íñigo Ruiz). Any total order must pick something, but the doc shouldn't imply locale collation.

The rule I promoted to public contract was unpinned

Mutant (d) — reordering isupper and len in the title key — passed all 50 tests. So this PR documented "non-ALL-CAPS → shortest" in docs/pipeline/graph.md while nothing checked that order.

That is exactly the defect I identified for the type rule two paragraphs into my own PR body: a test passing for the wrong reason. test_dedup_variants_into_one_entity cannot pin it, because there the non-caps name is also the shortest. _best_title({"IBM": 5, "Ibm Global": 1}) discriminates — IBM is shorter and more frequent and must still lose. Both reorderings now fail.

Recorded, not fixed

• Slug-collision assignment depends on the set of keys, so a new entity whose key sorts earlier can relocate an existing node. Links are rewritten consistently, so nothing dangles — but making paths independent of sibling keys would change existing paths for every corpus, which is a migration decision rather than a bugfix. Backlog.
• Migration: a deployed corpus with a genuine type tie relocates that node on the next build. Handled correctly — no orphan .md, no dangling links — but build.py uses os.remove and never rmdir, so an empty entities/<old-type>/ directory is left behind. Cosmetic; noted.
• isupper() is False for caseless scripts, so a CJK name always reads as non-ALL-CAPS and outranks Latin caps, and 3M is penalised as shouty. Pre-existing, not introduced here.

Corrected in the PR body: the test count (graph 51, main was 47).

graph 51 · clean 256 · corpus 48 · slack 13 · fetch 33 · answer 66 · benchmark 3, bare pytest (CI mode) · ruff clean · scorecard 25/25.