Skip to content

autotrain(continuous-openui-local): L01 unblock doc self-heal version-stamp gate + first two cycles - #1503

Merged
Tyler-R-Kendrick merged 8 commits into
mainfrom
claude/great-dirac-kij51s
Aug 8, 2026
Merged

autotrain(continuous-openui-local): L01 unblock doc self-heal version-stamp gate + first two cycles#1503
Tyler-R-Kendrick merged 8 commits into
mainfrom
claude/great-dirac-kij51s

Conversation

@Tyler-R-Kendrick

@Tyler-R-Kendrick Tyler-R-Kendrick commented Aug 8, 2026

Copy link
Copy Markdown
Owner

Summary

  • Loop: continuous-openui-local
  • Cycles: c1 (fixture insufficient_n, expected on smoke-scale data), c2 (primary metric win: smoke.structural_similarity 0.3267 → 0.3828, +0.0561)
  • Unblock proof: the continuous driver's document self-heal commit was hard-blocking every cycle on scripts.verify_version_stamps --check because README.md/MODEL_CARD.md changes require a matching no-bump history entry for harness.experiments.slm228_spectral_disposition. Added .agents/skills/autotrain/scripts/self_heal_doc_commit.py, a small helper that inserts the required history entry via a targeted text splice (not a json.dump rewrite, which would reformat/escape the entire 8k-line registry) and lands the commit. Replayed identical cycles c1 and c2 afterward — both completed cleanly.
  • Measured results: docs/design/continuous-loop-20260808-continuous-openui-local-8c0b60dd-c1-results.{md,json}, docs/design/continuous-loop-20260808-continuous-openui-local-8c0b60dd-c2-results.{md,json}
  • Model card / README updated with checkpoint provenance for both cycles (honesty note: fixture/scratch continuous cycles, not a ship promotion).

Why this is positive

Per autotrain-iteration-delivery.md, this qualifies as executable unblocking: a harness fix removed a hard, reproducing blocker (pre-commit version-stamp gate failure on every continuous document closeout), and the identical arm then completed with a usable scoreboard on replay (c1, then c2 with a primary-metric win).

Test plan

  • scripts.verify_version_stamps --check --staged passes for both cycle closeout commits
  • ruff check on the new helper script passes
  • Repo pre-commit hook's targeted pytest shard passes on each commit
  • Continuous supervisor (scripts.run_autotrain_supervisor --loop-id continuous-openui-local --train-version wf_smoke_v2 --steps 20) advances past cycle c1 and c2 without a human re-prompt

Generated by Claude Code

Summary by CodeRabbit

  • New Features

    • Added automation to record and verify version-history updates before committing changes.
  • Documentation

    • Added continuous training cycle notes, checkpoint details, and screening results.
    • Documented candidate and control measurements, outcomes, limitations, and fixture-only status.
    • Updated version history with two scheduled cycle records and metric information.
  • Chores

    • Added structured JSON evidence fixtures to support repeatable screening and review.

@vercel

vercel Bot commented Aug 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
slm-training Ready Ready Preview Aug 8, 2026 3:12pm

Request Review

@coderabbitai

coderabbitai Bot commented Aug 8, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@Tyler-R-Kendrick, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 42 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 34404b69-fd92-411e-93e1-f031a73b8a9b

📥 Commits

Reviewing files that changed from the base of the PR and between d473f94 and c9cfae3.

📒 Files selected for processing (13)
  • README.md
  • docs/MODEL_CARD.md
  • docs/design/continuous-loop-20260808-continuous-openui-local-8c0b60dd-c3-results.json
  • docs/design/continuous-loop-20260808-continuous-openui-local-8c0b60dd-c3-results.md
  • docs/design/continuous-loop-20260808-continuous-openui-local-8c0b60dd-c4-results.json
  • docs/design/continuous-loop-20260808-continuous-openui-local-8c0b60dd-c4-results.md
  • docs/design/continuous-loop-20260808-continuous-openui-local-8c0b60dd-c5-results.json
  • docs/design/continuous-loop-20260808-continuous-openui-local-8c0b60dd-c5-results.md
  • docs/design/continuous-loop-20260808-continuous-openui-local-8c0b60dd-c6-results.json
  • docs/design/continuous-loop-20260808-continuous-openui-local-8c0b60dd-c6-results.md
  • docs/design/continuous-loop-20260808-continuous-openui-local-8c0b60dd-c7-results.json
  • docs/design/continuous-loop-20260808-continuous-openui-local-8c0b60dd-c7-results.md
  • src/slm_training/resources/versions.json
📝 Walkthrough

Walkthrough

The change adds a staged version-history commit utility and documents two fixture-only continuous-openui-local screening cycles dated 2026-08-08. The records include JSON and Markdown results, README and model-card notes, and v3 history entries.

Changes

Continuous autotrain evidence

Layer / File(s) Summary
Staged history commit utility
.agents/skills/autotrain/scripts/self_heal_doc_commit.py
Adds a command-line utility that inserts a component history entry, validates staged version stamps, rejects empty staging areas, and commits with fixed local author settings.
Fixture screening records and version history
docs/design/continuous-loop-..., README.md, docs/MODEL_CARD.md, src/slm_training/resources/versions.json
Adds c1 and c2 screening evidence, campaign notes, and matching v3 history entries. The records mark both cycles as fixture or scratch work rather than ship promotions.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant self_heal_doc_commit.py
  participant GitIndex
  participant verify_version_stamps
  participant GitCommit
  self_heal_doc_commit.py->>GitIndex: Stage version-history entry
  self_heal_doc_commit.py->>GitIndex: Read staged paths
  self_heal_doc_commit.py->>verify_version_stamps: Validate staged version stamps
  verify_version_stamps-->>self_heal_doc_commit.py: Return validation result
  self_heal_doc_commit.py->>GitCommit: Commit staged changes
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the self-heal version-stamp fix and the documentation of the first two continuous-openui-local cycles.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/great-dirac-kij51s

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
docs/MODEL_CARD.md (2)

1-1: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Duplicate ## Continuous autotrain note (2026-08-08) heading in docs/MODEL_CARD.md and README.md. Both files add two headings with identical text for cycles c1 and c2 on the same day. markdownlint (MD024) flags this in both files, and it also causes anchor-link ambiguity for readers or tools navigating by heading id.

  • docs/MODEL_CARD.md#L1452-1463: append the cycle id to each heading, e.g. ## Continuous autotrain note (2026-08-08, cycle c1) and ## Continuous autotrain note (2026-08-08, cycle c2).
  • README.md#L796-807: apply the same cycle-id suffix to its two matching headings.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/MODEL_CARD.md` at line 1, Update the duplicate “Continuous autotrain
note (2026-08-08)” headings in both MODEL_CARD.md and README.md to include their
cycle identifiers, using “cycle c1” and “cycle c2” respectively. Preserve the
existing heading levels and note content while ensuring each heading text is
unique.

Source: Linters/SAST tools


1452-1463: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Distinguish the two same-day headings (see consolidated comment).

Lines 1452 and 1458 use the identical heading text ## Continuous autotrain note (2026-08-08) for cycles c1 and c2. This is flagged by markdownlint (MD024) and mirrors the same issue in README.md. See the consolidated comment for the proposed fix.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/MODEL_CARD.md` around lines 1452 - 1463, The two continuous autotrain
note headings in MODEL_CARD.md must be unique to satisfy MD024. Update each
heading to include its cycle identifier, distinguishing c1 from c2 while
preserving the existing date and note context.

Source: Linters/SAST tools

README.md (1)

796-807: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Distinguish the two same-day headings (see consolidated comment).

Lines 796 and 802 use the identical heading text ## Continuous autotrain note (2026-08-08) for two different cycles (c1 and c2). This is flagged by markdownlint (MD024) and causes anchor-link ambiguity between the two sections. See the consolidated comment covering this file and docs/MODEL_CARD.md for the proposed fix.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@README.md` around lines 796 - 807, Distinguish the two same-day README
headings by making each `## Continuous autotrain note (2026-08-08)` title
unique, while preserving the c1 and c2 cycle details under their respective
sections and avoiding duplicate Markdown anchors.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.agents/skills/autotrain/scripts/self_heal_doc_commit.py:
- Around line 31-48: Update _bump_component so the inserted JSON entry uses
json.dumps() for the date and note values before f-string interpolation,
producing valid escaped JSON string literals for quotes, backslashes, and other
special characters while preserving the existing text-splicing approach.
- Around line 36-38: Update the history insertion logic around hist_idx and
insert_at to detect when the targeted component’s history array is empty. For an
empty array, set the insertion position immediately after the `"history": [`
marker; otherwise preserve the existing search for the next history-entry `{`,
ensuring the new entry remains within the selected component’s history.

---

Outside diff comments:
In `@docs/MODEL_CARD.md`:
- Line 1: Update the duplicate “Continuous autotrain note (2026-08-08)” headings
in both MODEL_CARD.md and README.md to include their cycle identifiers, using
“cycle c1” and “cycle c2” respectively. Preserve the existing heading levels and
note content while ensuring each heading text is unique.
- Around line 1452-1463: The two continuous autotrain note headings in
MODEL_CARD.md must be unique to satisfy MD024. Update each heading to include
its cycle identifier, distinguishing c1 from c2 while preserving the existing
date and note context.

In `@README.md`:
- Around line 796-807: Distinguish the two same-day README headings by making
each `## Continuous autotrain note (2026-08-08)` title unique, while preserving
the c1 and c2 cycle details under their respective sections and avoiding
duplicate Markdown anchors.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 5031d1aa-d2e1-42e3-b85d-0efcda51bbed

📥 Commits

Reviewing files that changed from the base of the PR and between b12eb2c and d473f94.

📒 Files selected for processing (8)
  • .agents/skills/autotrain/scripts/self_heal_doc_commit.py
  • README.md
  • docs/MODEL_CARD.md
  • docs/design/continuous-loop-20260808-continuous-openui-local-8c0b60dd-c1-results.json
  • docs/design/continuous-loop-20260808-continuous-openui-local-8c0b60dd-c1-results.md
  • docs/design/continuous-loop-20260808-continuous-openui-local-8c0b60dd-c2-results.json
  • docs/design/continuous-loop-20260808-continuous-openui-local-8c0b60dd-c2-results.md
  • src/slm_training/resources/versions.json

Comment on lines +31 to +48
def _bump_component(component_id: str, date: str, note: str) -> bool:
data = json.loads(VERSIONS.read_text(encoding="utf-8"))
comp = data["components"][component_id]
version = comp["version"]
text = VERSIONS.read_text(encoding="utf-8")
idx = text.index(f'"{component_id}"')
hist_idx = text.index('"history": [', idx)
insert_at = text.index("{", hist_idx)
entry = (
"{\n"
f' "version": "{version}",\n'
f' "date": "{date}",\n'
f' "note": "{note}"\n'
" },\n "
)
new_text = text[:insert_at] + entry + text[insert_at:]
VERSIONS.write_text(new_text, encoding="utf-8")
return True

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🔴 Critical | ⚡ Quick win

Escape note and date before splicing them into the JSON text.

The entry template inserts note and date into a hand-written JSON fragment with plain f-string interpolation. If note or date contains a " or a \, the resulting versions.json becomes invalid JSON. This breaks verify_version_stamps, every test that parses versions.json, and any other tool that reads the registry, for every component, not only the one this script touches. The docstring states this script exists specifically "to avoid reformatting/escaping the whole registry," but the escaping is required for the inserted fragment regardless of whether the rest of the file is rewritten.

Use json.dumps() to produce the escaped string literals for these two fields.

🛡️ Proposed fix to escape inserted string values
     entry = (
         "{\n"
-        f'          "version": "{version}",\n'
-        f'          "date": "{date}",\n'
-        f'          "note": "{note}"\n'
+        f'          "version": {json.dumps(version)},\n'
+        f'          "date": {json.dumps(date)},\n'
+        f'          "note": {json.dumps(note)}\n'
         "        },\n        "
     )
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
def _bump_component(component_id: str, date: str, note: str) -> bool:
data = json.loads(VERSIONS.read_text(encoding="utf-8"))
comp = data["components"][component_id]
version = comp["version"]
text = VERSIONS.read_text(encoding="utf-8")
idx = text.index(f'"{component_id}"')
hist_idx = text.index('"history": [', idx)
insert_at = text.index("{", hist_idx)
entry = (
"{\n"
f' "version": "{version}",\n'
f' "date": "{date}",\n'
f' "note": "{note}"\n'
" },\n "
)
new_text = text[:insert_at] + entry + text[insert_at:]
VERSIONS.write_text(new_text, encoding="utf-8")
return True
def _bump_component(component_id: str, date: str, note: str) -> bool:
data = json.loads(VERSIONS.read_text(encoding="utf-8"))
comp = data["components"][component_id]
version = comp["version"]
text = VERSIONS.read_text(encoding="utf-8")
idx = text.index(f'"{component_id}"')
hist_idx = text.index('"history": [', idx)
insert_at = text.index("{", hist_idx)
entry = (
"{\n"
f' "version": {json.dumps(version)},\n'
f' "date": {json.dumps(date)},\n'
f' "note": {json.dumps(note)}\n'
" },\n "
)
new_text = text[:insert_at] + entry + text[insert_at:]
VERSIONS.write_text(new_text, encoding="utf-8")
return True
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.agents/skills/autotrain/scripts/self_heal_doc_commit.py around lines 31 -
48, Update _bump_component so the inserted JSON entry uses json.dumps() for the
date and note values before f-string interpolation, producing valid escaped JSON
string literals for quotes, backslashes, and other special characters while
preserving the existing text-splicing approach.

Comment on lines +36 to +38
idx = text.index(f'"{component_id}"')
hist_idx = text.index('"history": [', idx)
insert_at = text.index("{", hist_idx)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Handle an empty history array before searching for the next {.

insert_at = text.index("{", hist_idx) assumes comp["history"] already has at least one entry. If a component's history is [] (the empty-array case, e.g., a freshly registered component or a future --component target other than the current default), this search skips past the array's own ] and matches an unrelated { belonging to a later component or object in the file. The new entry then gets spliced into the wrong place, corrupting that other component's data and its own semantics, not just formatting.

Check whether history is empty and branch to insert right after "history": [ in that case, instead of unconditionally searching for the next {.

🛡️ Proposed fix to handle an empty history array
     comp = data["components"][component_id]
     version = comp["version"]
+    history = comp.get("history") or []
     text = VERSIONS.read_text(encoding="utf-8")
     idx = text.index(f'"{component_id}"')
     hist_idx = text.index('"history": [', idx)
-    insert_at = text.index("{", hist_idx)
-    entry = (
-        "{\n"
-        f'          "version": {json.dumps(version)},\n'
-        f'          "date": {json.dumps(date)},\n'
-        f'          "note": {json.dumps(note)}\n'
-        "        },\n        "
-    )
+    if history:
+        insert_at = text.index("{", hist_idx)
+        entry = (
+            "{\n"
+            f'          "version": {json.dumps(version)},\n'
+            f'          "date": {json.dumps(date)},\n'
+            f'          "note": {json.dumps(note)}\n'
+            "        },\n        "
+        )
+    else:
+        insert_at = text.index("]", hist_idx)
+        entry = (
+            "{\n"
+            f'          "version": {json.dumps(version)},\n'
+            f'          "date": {json.dumps(date)},\n'
+            f'          "note": {json.dumps(note)}\n'
+            "        }\n      "
+        )
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
idx = text.index(f'"{component_id}"')
hist_idx = text.index('"history": [', idx)
insert_at = text.index("{", hist_idx)
comp = data["components"][component_id]
version = comp["version"]
history = comp.get("history") or []
text = VERSIONS.read_text(encoding="utf-8")
idx = text.index(f'"{component_id}"')
hist_idx = text.index('"history": [', idx)
if history:
insert_at = text.index("{", hist_idx)
entry = (
"{\n"
f' "version": {json.dumps(version)},\n'
f' "date": {json.dumps(date)},\n'
f' "note": {json.dumps(note)}\n'
" },\n "
)
else:
insert_at = text.index("]", hist_idx)
entry = (
"{\n"
f' "version": {json.dumps(version)},\n'
f' "date": {json.dumps(date)},\n'
f' "note": {json.dumps(note)}\n'
" }\n "
)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.agents/skills/autotrain/scripts/self_heal_doc_commit.py around lines 36 -
38, Update the history insertion logic around hist_idx and insert_at to detect
when the targeted component’s history array is empty. For an empty array, set
the insertion position immediately after the `"history": [` marker; otherwise
preserve the existing search for the next history-entry `{`, ensuring the new
entry remains within the selected component’s history.

autotrain added 8 commits August 8, 2026 10:12
…oseouts

Continuous loop cycles that touch README.md/MODEL_CARD.md need a matching
no-bump version-stamp history entry before the driver's document self-heal
commit can land; without it every cycle hard-blocks on the pre-commit hook.
This helper inserts the entry via a targeted text splice (not a json.dump
rewrite, which reformats/escapes the whole 8k-line registry) and commits.
@Tyler-R-Kendrick
Tyler-R-Kendrick force-pushed the claude/great-dirac-kij51s branch from c9cfae3 to d97df9e Compare August 8, 2026 15:12
@Tyler-R-Kendrick
Tyler-R-Kendrick merged commit 3203029 into main Aug 8, 2026
1 of 2 checks passed
@Tyler-R-Kendrick
Tyler-R-Kendrick deleted the claude/great-dirac-kij51s branch August 8, 2026 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant