Extracted pluggy's design rationale from its git/PR history into agent rules — how faithful is it?

[reasonableplan]

Hi — I build a small open-source tool (code-hijack) that mines a repository's decision history (commit bodies, closed-unmerged PRs, maintainer comments, TODO/XXX comments) and turns it into coding rules for AI agents, where every rule must carry a verbatim quote from the actual history — a rule without a verified citation is mechanically demoted. No invented citations: refs are validated against the mined pool, and code examples must be byte-verbatim excerpts of current source.

I ran it on pluggy (30 files, 216 commits scanned) and published the output: examples/pluggy. (Posting here because the pluggy repo doesn't have Discussions enabled — hope that's OK, since the maintainer group overlaps.)

Three examples of what it extracted:

1. "Re-raising a stored exception must preserve the original traceback (with_traceback)" — anchored to commit 93ac1e9 (the 1.1.0 traceback-mutation regression). We also verified this one behaviorally: a small model without the rule accumulates tb frames 4→6→8 on repeated re-raises; with the rule it stays constant.
2. "Incompatible option combinations are rejected in code, not documentation" — anchored to the XXX comment at _hooks.py:613 about firstresult + historic.
3. "Behavior changes must ship with a test that pins the behavior" — anchored to pluggy PR #648, which was closed with a comment about not reviewing test-less (agentic) PRs.

The ask: you're the only people who can judge whether these read as faithful to your actual intent, or as plausible-sounding distortions. Even a drive-by "1 and 3 are right, 2 is overfit" would be extremely valuable — I'm trying to measure exactly this failure mode (extraction that looks well-cited but misrepresents the author).

To be explicit: everything in the output is derived from pluggy's public history under its license, with quotes attributed to their commits/PRs. If this kind of post isn't welcome here, tell me and I'll close it — no hard feelings.

[RonnyPfannschmidt]

These results are overexcited and gruesome misaligned

I rate this as harmful and degenerative slop

[RonnyPfannschmidt]

Psa even opus4./gpt5. regularly completely mess up in pluggy because the its at deep framework level and even the most powerful statistical parrots simply don't have enough training data to enable them