fix(hook): skip re-review of a plan already decided this session - #1169
fix(hook): skip re-review of a plan already decided this session#1169rNoz wants to merge 4 commits into
Conversation
|
Deep review done, including end-to-end reproduction from a worktree with an isolated data dir. The store itself is careful work: fail-open is real (missing file, corrupt JSON, empty session_id all fall through to a fresh review, verified), the config resolver matches house convention with garbage values disabling the feature, hashing makes path-shaped session ids safe, and 7-day pruning bounds growth. But the risk lives at the two call sites, not in the store, and that is where this needs changes. Blocking
Fix before merge regardless of direction: plan-decision-store.ts contains a literal NUL byte (the separator at line 44 is a raw 0x00, not an escape). Verified consequences: git shows the file as Binary (the PR diff literally says Bin 0 -> 4151 bytes, so nobody can review it as a diff), grep needs -a, and rg does not list the file's own definitions. Use \u0000 in the template literal. Should-fix, if the store direction survives: label replayed feedback (e.g. "Previously requested changes (not re-reviewed): ..."); Claude-path denials are recorded with user feedback but never read (write-only prose on disk); a malformed record on the Codex path currently becomes a fabricated "Plan changes requested" denial rather than being treated as absent (validate decision === approved|denied); the 10 new tests only cover the store in isolation, with nothing on approve-then-resubmit, deny-then-resubmit, corrupt-store-reopens, or session boundaries (I hand-verified several of those; they should be CI regression tests); the env-variables reference page is missing the new knob and AGENTS.md does not say it is hook-runtimes-only (OpenCode and Pi are unaffected); the normalizer duplicates codex-session's normalizePlan and the comment claiming it mirrors version-history dedup is inaccurate (saveToHistory compares exact strings). Honest bottom line: the annoyance in #1075 is real, but the Claude half needs an occurrence/freshness bound plus a visible replay signal before auto-answering a human gate, and the Codex half is better fixed at collectPlanCandidates' turn filtering than by replaying denials. Happy to talk through the occurrence-binding design if useful, and to re-review quickly on push. |
Reuse only fresh approvals for the active Claude ExitPlanMode occurrence; never replay Codex or denial decisions. Filter Codex proposal fallbacks to the requested turn. Refs backnotprop#1169
|
Pushed the review fixes. Claude reuse is visible and bound to one active submission occurrence. Codex now filters stale plans by turn. Checks are passing. |
|
Re-verified f2c5bab in depth, including 10 independent probes and end-to-end runs through the real hook binary. This redesign is exactly right and all three blockers are resolved:
Two small asks before merge:
Benign residuals, no action needed: a turn boundary without an id inherits the previous activeTurnId (extra review, safe direction); concurrent hook writes can lose a record (extra review, verified with 8 parallel writers, no temp leaks); future record-shape changes invalidate old records via the strict two-key check (safe, worth a comment). |
A plan review re-opened for a plan that was already decided (backnotprop#1075) in two ways: an identical ExitPlanMode plan submitted twice opened two reviews, and a Codex Stop turn that did no planning still scraped the previous turn's already-decided plan and reviewed it again on every bookkeeping turn. saveToHistory already dedups identical resubmissions for version history, but nothing consulted a prior decision before opening a session. Add a per-session decision store (`plan-decision-store.ts`): plan outcomes are recorded keyed by (project, session, normalized-plan hash), one JSON file per (project, session) under the data dir, written atomically (temp + rename), pruned after seven days, entirely best-effort so a store failure never breaks review. Plans are normalized (CRLF to LF + trim) so a Windows or trailing-whitespace resubmission of the same plan still matches. Before opening a review the store is consulted: - Claude ExitPlanMode re-emits only a prior APPROVAL without re-opening; a prior denial still re-opens, so a plan denied by mistake can be reconsidered by resubmitting it. - Codex Stop re-emits a prior approval or denial, because a bookkeeping turn would otherwise re-review the same decided plan every turn. The decision is recorded after each fresh review. The Claude session id comes from the event, the Codex one from its thread/rollout. Gemini plans (file-based, different identity) are excluded. The whole feature is gated by a new `planDecisionReuse` setting (`PLANNOTATOR_PLAN_DECISION_REUSE` env or config.json), defaulting ON so the bug is fixed out of the box while remaining disable-able for anyone who prefers to always re-review.
Reuse only fresh approvals for the active Claude ExitPlanMode occurrence; never replay Codex or denial decisions. Filter Codex proposal fallbacks to the requested turn. Refs backnotprop#1169
Missing Stop turn IDs cannot safely distinguish stale assistant plans.
Missing turn identity skips are visible only with PLANNOTATOR_DEBUG.
f2c5bab to
07dbdc0
Compare
|
Rebased onto current main and resolved the environment reference conflict. Added debug-only breadcrumbs for unsafe Codex Stop skips. Local rollout markers and Claude occurrence timing were also checked. |
|
Verified the final push directly: the PLANNOTATOR_DEBUG breadcrumb covers both skip shapes (missing Stop payload turn_id, missing id-carrying rollout marker) with distinct messages on stderr and a clean exit, exactly as asked, and the rebase onto current main resolved cleanly. All 208 hook tests pass on the head. This closes out both asks from the re-review; merge-ready from my side once the post-rebase CI finishes. |
|
Thanks for fixing this! This looks like it addresses both cases from my issue without skipping reviews for genuinely new plans. |
|
Thanks for confirming both cases, that closes the loop on the report. This is queued to merge. |
Summary
Closes #1075.
A plan review re-opened for a plan already decided in the same session, in two ways:
ExitPlanMode(Claude): an identical plan submitted twice opened two separate reviews. The plan text was read straight fromevent.tool_input.planand handed tostartPlannotatorServerwith no dedup.Stop(Codex): the Stop hook scrapes the transcript for the latest plan every turn. A turn that proposes no new plan (for example one that only wrote a file) still finds the previous, already-approved plan as the latest one, so it is reviewed again. The plan-candidate collection only guarded within a turn, not across a turn boundary.Approach
The two triggers are fixed differently, because they need different guarantees.
For Claude
ExitPlanMode, a small store records an approval bound to the specific transcript occurrence (the exactExitPlanModetool-use entry), not just the plan text, using the repo's existing rewind-aware active-branch resolver, with a 5-minute TTL. A retry of that same still-active occurrence re-emits the prior allow decision together with a visiblesystemMessageso the reuse is never silent; a plan resubmitted after/rewindis a new occurrence and always gets a fresh review, even if the text is byte-identical. Only approvals are ever recorded or replayed; a denial always opens a fresh review on resubmission.For Codex
Stop, the fix is at the source rather than a replay layer: the transcript scraper that collects plan candidates now applies a turn-identity boundary, so a bookkeeping-only turn can no longer resurface an already-decided plan proposed in an earlier turn. There is no decision store and nothing is replayed on this path. When a candidate's turn identity cannot be determined, it is dropped rather than guessed at (fail closed), with aPLANNOTATOR_DEBUG-gated log line recording why, so that narrow case is diagnosable rather than silent.Design choices
planDecisionReusesetting (PLANNOTATOR_PLAN_DECISION_REUSEenv orconfig.json) gates the Claude-side occurrence-bound reuse. It defaults ON so the reported bug is fixed without configuration, while anyone who prefers to always re-review can disable it./rewind) is a different occurrence and always reviews fresh.Validation
189 targeted tests pass, covering the occurrence-bound store (record/read, TTL boundary, rewind and compact handling, fail-open on a missing or corrupt record), the Codex turn-identity gate, and the config resolver, along with a full build and the full hook test suite (208 tests) after rebasing onto current
main.Compatibility
Additive. A first-time plan review is unchanged. Only an exact resubmission of the same still-active occurrence within the same session is affected on the Claude path, and a Codex bookkeeping-only turn no longer resurfaces a stale plan. No public API, endpoint, or payload changes; the store lives under the existing data dir.
Related work
Thanks to @jpvarbed for filing #1075 and independently exploring a similar per-session decision-recording approach on a fork branch; that discussion informed the direction taken here.