Skip to content

feat(junior-profile): predict-before-reveal loop with evidence-based adaptive fading#21

Merged
DanielPodolsky merged 6 commits into
mainfrom
feat/junior-predict-before-reveal
Jun 27, 2026
Merged

feat(junior-profile): predict-before-reveal loop with evidence-based adaptive fading#21
DanielPodolsky merged 6 commits into
mainfrom
feat/junior-predict-before-reveal

Conversation

@DanielPodolsky

@DanielPodolsky DanielPodolsky commented Jun 27, 2026

Copy link
Copy Markdown
Owner

What

Reworks the Junior profile from "hand-type every line" (outdated) to "evaluate everything" — without falling into the passive-observation dependence trap. The junior now commits a prediction (approach / data structure / control flow / edge cases) before the AI reveals code, then an AI judge grades it against a rubric, names the specific gap, and grounds the feedback. Scores are logged as a judgment-growth curve.

Why it's where it is

  • The loop lives in /own:feature Phase 6 (the command that always runs), not /own:guide (rarely run) — so the gate is hard, not skippable under fatigue.
  • Friction applies only to Implementation-group tasks; Setup/Verification flow.
  • Gate 1 (Ownership) in /own:done now uses the prediction transcript as evidence, not vibes.

Evidence-based (not a hunch)

Validated against two streams (full report: docs/research/junior-profile-predict-before-reveal-validation.md):

  • Learning science (adversarially-verified deep research): Tucker et al. 2024 RCT — predicting code beats producing it on learning, transfer, and motivation; reinforced by the pretesting / errorful-generation literature.
  • Industry practice (real GitHub artifacts): ownership = accountability for understanding, not typing (kornia, psimm, github/docs); one unrelated project (hotaru) independently reinvented the same model; vapor reserves good-first-issues for human learning.

The research surfaced one gap — adaptive fading — now implemented (P0): the gate fades per-dimension after sustained MATCHes (expertise-reversal effect), driven by the Prediction Scorecard we already log. Cautious thresholds: 4 consecutive MATCHes → faded/spot-check; PARTIAL/MISS resets; ownership floor never removed.

Tested

Manually run end-to-end on a real project (HR voice-screening prototype). The loop, the gate, and per-dimension fading behave as designed; first real prediction is logged.

Known debt (tracked, not blocking)

  • Octocode grounding is retrieval, not a quality oracle — prevalence ≠ correctness. Source-vetting deferred.
  • P1 (worked-example escape hatch for cognitive-overload tasks) and P2 (self-explanation at the "Own" step) are recommended in the research but not yet built.

Files

  • profiles/junior.md — The Implementation Loop + Adaptive Fading
  • .claude/commands/own/feature.md — Phase 6 (hard gate, reads scorecard, fades per-dimension)
  • learning/LEARNING_REGISTRY.md — Prediction Scorecard
  • docs/research/… — the validation report

DanielPodolsky and others added 3 commits June 20, 2026 22:20
…l loop

Reworks the Junior profile from "hand-type every line" (outdated) to
"evaluate everything" — without falling into the passive-observation
dependence trap. The junior now commits a prediction before any AI code
is revealed, then is graded against a fixed rubric.

- profiles/junior.md: ownership reframed as evaluation, not transcription;
  adds The Implementation Loop (Predict -> Gate -> Reveal -> Judge -> Own),
  the 4-dimension rubric, anti-sycophancy + grounding guards; Gate 1 now
  uses the prediction transcript as ownership evidence.
- own/feature.md: new Phase 6 (junior-only) runs the loop task-by-task as a
  HARD gate; only Implementation-group tasks gate, Setup/Verification flow.
- LEARNING_REGISTRY.md: new Prediction Scorecard makes judgment growth a
  measurable curve across projects (Protocol E eval, built in).

Known debt: grounding leans on Octocode retrieval (prevalence != quality);
source-vetting deferred to a follow-up before real juniors use it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ghhtyQRy5nmSGBbCaNpwH
…l junior loop

Two-stream review (learning-science literature via adversarially-verified deep
research; industry practice via real GitHub artifacts) of the reworked Junior
profile. Verdict: well-grounded — direct RCT support (Tucker 2024, predict>produce
for code), errorful-generation backbone, and real-repo confirmation that ownership
= accountability-for-understanding, not typing. Surfaces the key gap (no adaptive
fading) and prioritized refinements (P0 fade, P1 overload escape, P2 self-explanation).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ghhtyQRy5nmSGBbCaNpwH
…l gate

Implements the P0 refinement the research identified: the prediction gate must
relax as competence grows (expertise-reversal effect), or it decays into a tax →
fatigue → rubber-stamping. The gate now fades PER-DIMENSION, driven by the data
we already log (Prediction Scorecard) — turning that metric from a vanity readout
into the loop's control signal.

- profiles/junior.md: new "Adaptive Fading" rule — cautious thresholds (4
  consecutive MATCHes fades a dimension to spot-check; PARTIAL/MISS resets streak;
  MISS on a re-asked faded dimension snaps it back). Ownership floor never removed.
- own/feature.md: Phase 6 reads the scorecard up front, computes per-dimension fade
  state, and gates only the still-weak dimensions; faded ones get a spot-check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ghhtyQRy5nmSGBbCaNpwH
Comment thread .claude/commands/own/feature.md Outdated
Comment thread profiles/junior.md Outdated
Comment thread profiles/junior.md
Comment thread .claude/commands/own/feature.md
@claude

claude Bot commented Jun 27, 2026

Copy link
Copy Markdown

Review — predict-before-reveal loop with adaptive fading

The substance here is strong, and I want to be clear up front: this does not weaken Gate 1 — it arguably strengthens it. The old "type every line" junior model conflated keystrokes with ownership; this reframes ownership as evaluation, not transcription, which is exactly the Active Typist protocol read correctly (comprehension, not typing volume). The Predict → Gate → Reveal → Judge → Own loop forces a committed position before the reveal, and /own:done Gate 1 now consumes the prediction transcript as real evidence and probes MISSes. That is a P2 upgrade, not a regression. The design is also genuinely evidence-grounded (P3): the validation doc cites a domain-matched RCT, names its own limits, and even records a killed claim — that intellectual honesty is the founder walking his own talk on ownership.

A few things I would want tightened, none touching the soul of the product (details inline):

  • P2 / OWN step — the ownership-evidence step asks only for acknowledgment, while the research doc you shipped in this same PR (§C2 P2) recommends a why-explanation. Implementation and its own evidence base disagree at the one step that matters most. Highest-leverage fix.
  • P4 scoping — gating the loop to profile.type = "junior" leaves Interview Prep (the audience that most needs to defend judgment live) and Career Switcher out, framed as "they do not need the gym." Make that a deliberate, softenable choice.
  • Adaptive fading floor — the "all four faded → flows like Setup" path is your max P2 exposure; the floor you wrote is load-bearing, and the 4-MATCH threshold / soft snap-back trigger deserve a "this is a conservative v1 number" caveat (your own §C3 lists it as open).

What blocks merge — shipping discipline, not substance:

  • D2 (CHANGELOG) — required. This changes the entire junior implementation experience and there is no CHANGELOG.md entry. The project rule is non-negotiable: every PR adds one, and a new user-facing capability is a MINOR version bump (2.6.02.7.0). A feature this significant cannot merge undocumented.
  • D1 (README/guide sync) — required. Juniors are now told not to run /own:guide ("the gym is built in"), but the README Commands table (line 107) still presents /own:guide as the implementation-guidance command with no note that it is bypassed for the Junior profile. Discoverability is a product feature — reconcile the two.
  • D3 (commit scope) — nit. feat(junior-profile) / docs(research) use scopes outside the valid set; these should be feat(profiles) / docs(...). Cheap to get right next time.

Fix the two required doc items (a CHANGELOG entry + a one-line README/guide note) and I will flip this. The philosophy and architecture are approved — I am holding only on the paper trail the project insists every change carries.

VERDICT: REQUEST_CHANGES

…snap-back, phased-rollout copy

Resolves the three actionable review comments on PR #21:

- P2 (ownership-critical): OWN step changed from "acknowledge the gap" to "explain
  WHY the actual approach is better and WHERE your prediction's logic broke" —
  self-explanation, not recognition. Removes the contradiction between the shipped
  research doc (which recommended P2) and the implementation. Applied in both
  junior.md and feature.md Phase 6.
- Adaptive fading: documented that the "4 consecutive MATCHes" threshold is a
  deliberately conservative v1 number (research lists it as open), and added a
  DETERMINISTIC staleness nudge (re-ask a faded dimension after 5 stale tasks) so
  snap-back no longer depends solely on a soft judgment trigger — a lightweight
  form of the research's P3 spaced-re-prediction.
- Phased-rollout copy: "Other profiles don't need the gym" reframed as deliberate
  v1 scoping, naming Interview Prep + Career Switcher as the intended next profiles
  rather than a permanent exclusion.
- Research doc: annotated implementation status (P0 ✅, P2 ✅, P3 ⚙️ partial, P1 ⏳).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ghhtyQRy5nmSGBbCaNpwH
@DanielPodolsky

Copy link
Copy Markdown
Owner Author

Addressed the three actionable comments in 9fd7be2:

  • P2 (OWN step, junior.md:158) — implemented. OWN now requires the junior to explain why the actual approach is better and where their prediction's logic broke (self-explanation), not just acknowledge the gap. Resolves the implementation-vs-shipped-doc contradiction; applied in both junior.md and feature.md Phase 6.
  • Fade threshold + soft snap-back (junior.md:194) — documented "4 consecutive MATCHes" as a deliberately conservative v1 number (research §C3 Q2 open), and added a deterministic staleness nudge: a faded dimension ≥5 tasks stale is re-asked once (lightweight P3 spaced-re-prediction), so re-engagement no longer depends solely on a soft judgment trigger.
  • "Don't need the gym" framing (feature.md:49) — reframed as deliberate v1 scoping; names Interview Prep + Career Switcher as the intended next profiles, not a permanent exclusion.

Left as tracked debt (not in this PR): P1 (worked-example escape hatch for cognitive-overload tasks) and the Octocode-quality caveat. Thanks for the catch on P2 — the doc/code disagreement at the ownership-critical step was the right thing to flag.

@claude

claude Bot commented Jun 27, 2026

Copy link
Copy Markdown

Incremental review — 8e7464e..9fd7be2

The three inline findings are resolved well, no new concerns in the diff itself:

  • P2 / OWN step ✅ — OWN now demands why the actual approach is better and where the prediction's logic broke (self-explanation, not acknowledgment), applied in both junior.md:156 and feature.md Phase 6. The highest-leverage fix, and it lands exactly where the shipped research doc said it should.
  • Snap-back determinism ✅ — the 5-task staleness nudge turns re-engagement from a soft "on a complex task" judgment call into a scorecard-driven trigger. Faded-but-rusty dimensions now re-engage on their own.
  • Fade threshold + framing ✅ — the "4" is now flagged as a conservative v1 dial (§C3 Q2 open), and "they don't need the gym" is reframed as deliberate v1 scoping naming Interview Prep + Career Switcher as next. Both copies stay consistent.

Still blocking — the two required items from the prior review are untouched in this push:

  • D2 (CHANGELOG) — still missing. No 2.7.0 entry; CHANGELOG.md still tops out at 2.6.0. Every PR adds a CHANGELOG entry, and a new user-facing capability of this size is a MINOR bump. Your reply addressed the three inline comments but not this one.
  • D1 (README/guide sync) — still missing. The Commands table (README line ~107) still lists /own:guide as the implementation-guidance command with no note that the Junior gym now lives inline in /own:feature Phase 6. Reconcile the discoverability story.

Philosophy and architecture remain approved — I'm holding solely on the paper trail. Add the CHANGELOG entry + the one-line README note and I flip this.

VERDICT: REQUEST_CHANGES

…lves review D1/D2)

Addresses the two required items the gatekeeper held on:

- D2: add the 2.7.0 CHANGELOG entry (MINOR — predict-before-reveal Junior loop,
  Phase 6 gate, Prediction Scorecard, adaptive fading, self-explanation OWN step).
- D1: README Commands table now states the Junior gym runs inline in /own:feature
  Phase 6 and that /own:guide is no longer the primary implementation path.
- Bumped version strings 2.6.0 -> 2.7.0 across installers, README badge, and the
  dashboard-data template so the release is internally consistent (.ps1 stays pure
  ASCII). The DASHBOARD_CONTRACT example value is illustrative and left as-is.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ghhtyQRy5nmSGBbCaNpwH
@claude

claude Bot commented Jun 27, 2026

Copy link
Copy Markdown

Incremental review — release cut 8e4e4b3

Both required items from the prior review are now resolved, and the release sweep is otherwise clean:

  • D2 (CHANGELOG) ✅ — 2.7.0 entry added, accurate and appropriately scoped as a MINOR bump (new user-facing capability, no schema break). It captures the substance honestly: ownership shifts from typing to evaluation, Phase 6 inline gate, scorecard, adaptive fading + deterministic staleness re-check, the OWN self-explanation, and the /own:guide demotion.
  • D1 (README sync) ✅ — Commands table updated for /own:feature Phase 6 and /own:guide, plus a clear Junior-profile note. The discoverability story now matches the behavior.
  • T1 (script parity) ✅ — .sh/.ps1 bumped symmetrically across base + project installers; dashboard-data template and README badge all consistent at 2.7.0.

One non-blocking finding (new): the version sweep is incomplete. /own:init (.claude/commands/own/init.md) still hardcodes 2.6.0 in three live spots — the standard-profile manifest write (L618), the Custom-profile meta.version (L642/L1224), and the field-mapping contract assertion "meta.version → Literal 2.6.0" (L1257). Since the installers now write 2.7.0, a freshly /own:init'd project gets a 2.6.0 manifest + dashboard under a 2.7.0 release — an internal inconsistency this commit's own job (the version bump) should have caught. Not in this PR's diff, so I can't inline it; finish the bump in a follow-up before tagging. (The 2.6.0 in DASHBOARD_CONTRACT.md is an illustrative e.g. — leave it.)

  • D3 (commit scope) — nit, again: docs(release) uses a scope outside the valid set. docs(install) or chore(core) would pass. Cheap next time.

Philosophy and architecture were approved last round and are unchanged. The two blockers are cleared; I'm flipping off REQUEST_CHANGES. Flagging the init.md version miss as a COMMENT, not a block — but land it before you cut the tag.

VERDICT: COMMENT

Completes the version bump the gatekeeper flagged as incomplete: /own:init still
wrote 2.6.0 into the manifest and dashboard meta (L618, L642, L1224) and the
field-mapping contract (L1257), so a freshly-initialized project would land a
2.6.0 manifest under a 2.7.0 release. All four now write 2.7.0, consistent with
the installers and templates. (Uses a valid commit scope per the D3 nit.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ghhtyQRy5nmSGBbCaNpwH
@claude

claude Bot commented Jun 27, 2026

Copy link
Copy Markdown

Incremental review — 8e4e4b3..de68549

This closes the one open item from my last pass: the incomplete version sweep in /own:init.

  • D2 / version consistency ✅ — all four live 2.6.0 spots I flagged are now 2.7.0: the standard-profile manifest write (L618), the Custom-profile meta.version (L642) and its dashboard window.PROJECT.meta.version (L1224), and the field-mapping contract assertion (L1257). A freshly /own:init'd project now stamps 2.7.0 end-to-end, matching the installers. The illustrative e.g. in DASHBOARD_CONTRACT.md was correctly left alone.

Nothing else in the diff. Philosophy and architecture were approved two rounds ago; the paper trail is now whole. Ship it.

VERDICT: APPROVE

@DanielPodolsky
DanielPodolsky merged commit bd1f17c into main Jun 27, 2026
1 check passed
@DanielPodolsky
DanielPodolsky deleted the feat/junior-predict-before-reveal branch June 27, 2026 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant