fix(autotrain): self-heal commit identity + continuous-openui-scheduled-0808z1 c1-c5 closeout - #1519
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Warning Review limit reached
Next review available in: 44 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (9)
📝 WalkthroughWalkthroughThe change makes autotrain self-heal commits use configured Git identity. It adds c1/c2 continuous-loop result fixtures and checkpoint notes, records non-ship outcomes, and refreshes SemanticFloorGateV1 hash and harness experiment metadata. ChangesAutotrain closeout updates
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/design/semantic-floor-gate-v1.json`:
- Line 396: Regenerate the official SemanticFloorGateV1 hash using
scripts/publish_semantic_floor_gate.py --check, then replace the stale
7839ef...4dd83d value across all hash-bound semantic-floor consumers. Update
docs/design/semantic-floor-gate-v1.json at lines 396-396 and 451-451 and
docs/design/semantic-floor-gate-v1.md at lines 5-5 with the regenerated hash,
preserving all other gate content.
In `@README.md`:
- Around line 1368-1378: The c1 and c2 continuous autotrain records are
incomplete in both model-card surfaces. Update README.md lines 1368-1378 and
docs/MODEL_CARD.md lines 2024-2034 with the required roster, evaluations, and
recipe details or canonical result-artifact links; for c2, include both the
component-plan and control arms, or explicitly document that the control result
is unavailable in both files.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 019f9102-04f4-4f4c-bf31-5c754f3abf90
📒 Files selected for processing (11)
.agents/skills/autotrain/scripts/self_heal_doc_commit.pyREADME.mddocs/MODEL_CARD.mddocs/design/continuous-loop-20260808-continuous-openui-schedu-121086e4-c1-results.jsondocs/design/continuous-loop-20260808-continuous-openui-schedu-121086e4-c1-results.mddocs/design/continuous-loop-20260808-continuous-openui-schedu-121086e4-c2-results.jsondocs/design/continuous-loop-20260808-continuous-openui-schedu-121086e4-c2-results.mddocs/design/semantic-floor-gate-v1.jsondocs/design/semantic-floor-gate-v1.mdscripts/run_autotrain_continuous.pysrc/slm_training/resources/versions.json
💤 Files with no reviewable changes (1)
- scripts/run_autotrain_continuous.py
| ], | ||
| "evidence_cutoff": "2026-07-21T04:15:30.006965Z", | ||
| "gate_hash": "4ac3cb383206e32514606b5b2540912c2ec87971ca79abbf8daa60bbf89b3d68", | ||
| "gate_hash": "d762bbbdd6222507a1c1471c85e65effe71ee85fb918ad6f8eaf7f57e950134f", |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
rtk python -m scripts.publish_semantic_floor_gate --check
rtk python -m scripts.verify_version_stamps --check
rtk rg -n -i 'SLM-215|SLM-229|d762bbbdd6222507a1c1471c85e65effe71ee85fb918ad6f8eaf7f57e950134f|"harness\.experiments": "v148"' \
docs/design src/slm_training/resources/versions.jsonRepository: Tyler-R-Kendrick/slm-training
Length of output: 208
🏁 Script executed:
#!/bin/sh
set -euo pipefail
printf 'rtk availability: '
command -v rtk || true
printf '\nRelevant file existence:\n'
for f in \
docs/design/semantic-floor-gate-v1.json \
docs/design/semantic-floor-gate-v1.md \
src/slm_training/resources/versions.json \
scripts/publish_semantic_floor_gate.py \
scripts/verify_version_stamps.py
do
if [ -f "${f}" ]; then
printf '%s\n' "${f}"
else
printf 'MISSING %s\n' "${f}"
fi
done
printf '\nGrep search without rtk prefix:\n'
rg -n -i 'SLM-215|SLM-229|d762bbbdd6222507a1c1471c85e65effe71ee85fb918ad6f8eaf7f57e950134f|gate_hash|harness\.experiments' docs/design src/slm_training/resources/versions.json scripts || trueRepository: Tyler-R-Kendrick/slm-training
Length of output: 50386
🏁 Script executed:
#!/bin/sh
set -euo pipefail
printf 'rtk availability: '
command -v rtk || true
printf '\nRelevant files:\n'
for f in \
docs/design/semantic-floor-gate-v1.json \
docs/design/semantic-floor-gate-v1.md \
src/slm_training/resources/versions.json \
scripts/p publish_semantic_floor_gate.py scripts/publish_semantic_floor_gate.py
do
:
done
printf '\nExact line snippets:\n'
if [ -f docs/design/semantic-floor-gate-v1.json ]; then
sed -n '390,400p;445,455p' docs/design/semantic-floor-gate-v1.json
fi
if [ -f docs/design/semantic-floor-gate-v1.md ]; then
sed -n '1,10p' docs/design/semantic-floor-gate-v1.md
fi
if [ -f src/slm_training/resources/versions.json ]; then
rg -n '"v148"|version_stamp|semantic.?floor|SemanticFloor|SLM-215|SLM-229' src/slm_training/resources/versions.json | head -n 80
fi
printf '\nSearch terms (limited output):\n'
rg -n -i 'SLM-215|SLM-229|semantic-floor-gate-v1|SemanticFloorGate|semantic floor' docs/design src/slm_training/resources/versions.json scripts | head -n 160 || true
printf '\nTarget content occurrences (concise):\n'
python3 - <<'PY'
import json, pathlib, re
target="d762bbbdd6222507a1c1471c85e65effe71ee85fb918ad6f8eaf7f57e950134f"
for path in [
pathlib.Path('docs/design/semantic-floor-gate-v1.json'),
pathlib.Path('docs/design/semantic-floor-gate-v1.md'),
pathlib.Path('src/slm_training/resources/versions.json')
]:
if not path.exists():
print(f"MISSING {path}")
continue
raw = path.read_text("utf-8")
hits = [m for m in re.finditer(re.escape(target) + '|"v148"|v148', raw)]
print(f"{path}: {len(hits)} hits")
for hit in hits:
line = raw.count("\n", 0, hit.start()) + 1
print(f" line {line}: {repr(hit.group(0))}")
PYRepository: Tyler-R-Kendrick/slm-training
Length of output: 50385
🏁 Script executed:
#!/bin/sh
set -euo pipefail
printf 'Exact downstream SemanticFloorGateV1 hash/token occurrences:\n'
rg -n --fixed-strings '7839ef6b6e37710d487757da9170017d7b76a9d12ca1fb314bdb0fa23a4dd83d' docs/design | head -n 200 || true
printf '\nExact new gate hash occurrences:\n'
rg -n --fixed-strings 'd762bbbdd6222507a1c1471c85e65effe71ee85fb918ad6f8eaf7f57e950134f' docs/design src/slm_training/resources/versions.json | head -n 100 || true
printf '\nSemantic-floor artifacts with exact refs/hashes:\n'
python3 - <<'PY'
import pathlib, re
old="7839ef6b6e37710d487757da9170017d7b76a9d12ca1fb314bdb0fa23a4dd83d"
new="d762bbbdd6222507a1c1471c85e65effe71ee85fb918ad6f8eaf7f57e950134f"
refs=set()
for path in pathlib.Path("docs/design").rglob("*"):
if path.is_file() and path.stat().st_size < 5_000_000:
data=path.read_text(errors="ignore")
if "semantic-floor-gate-v1.json" in data or "semantic.floor" in data or "SemanticFloor" in data or "semantic floor" in data:
has_ref="semantic-floor-gate-v1.json" in data or "docs/design/semantic-floor-gate-v1.json" in data
refs.add((str(path), has_ref, old in data, new in data))
for path, has_ref, has_old, has_new in sorted(refs):
if has_ref:
print(f"{path}: ref={has_ref}, old_hash={has_old}, new_hash={has_new}")
PYRepository: Tyler-R-Kendrick/slm-training
Length of output: 28384
Update the SemanticFloorGateV1 chain after the v148 stamp change.
docs/design/semantic-floor-gate-v1.json and docs/design/semantic-floor-gate-v1.md show the new d762bb...134f hash, but downstream SLM-215 and SLM-229 closeouts still reference 7839ef...4dd83d. The official gate hash can be derived with scripts/publish_semantic_floor_gate.py --check, then propagate the regenerated hash to all hash-bound semantic-floor consumers.
📍 Affects 2 files
docs/design/semantic-floor-gate-v1.json#L396-L396(this comment)docs/design/semantic-floor-gate-v1.json#L451-L451docs/design/semantic-floor-gate-v1.md#L5-L5
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/design/semantic-floor-gate-v1.json` at line 396, Regenerate the official
SemanticFloorGateV1 hash using scripts/publish_semantic_floor_gate.py --check,
then replace the stale 7839ef...4dd83d value across all hash-bound
semantic-floor consumers. Update docs/design/semantic-floor-gate-v1.json at
lines 396-396 and 451-451 and docs/design/semantic-floor-gate-v1.md at lines 5-5
with the regenerated hash, preserving all other gate content.
Sources: Coding guidelines, Learnings
Comment on lines
+1368
to
+1378
| ## Continuous autotrain note (2026-08-08, continuous-loop-20260808-continuous-openui-schedu-121086e4-c1) | ||
|
|
||
| - campaign: `continuous-loop-20260808-continuous-openui-schedu-121086e4-c1` | ||
| - checkpoints: `runs/c20260808-continuous-openui-schedu-121086e4-c1-bounds/checkpoints/last.pt`, `runs/c20260808-continuous-openui-schedu-121086e4-c1-control/checkpoints/last.pt` | ||
| - honesty: fixture/scratch continuous cycle — **not** a ship promotion. | ||
|
|
||
| ## Continuous autotrain note (2026-08-08, continuous-loop-20260808-continuous-openui-schedu-121086e4-c2) | ||
|
|
||
| - campaign: `continuous-loop-20260808-continuous-openui-schedu-121086e4-c2` | ||
| - checkpoints: `runs/c20260808-continuous-openui-schedu-121086e4-c2-component-plan/checkpoints/last.pt` | ||
| - honesty: fixture/scratch continuous cycle — **not** a ship promotion. |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== rtk availability =="
if command -v rtk >/dev/null 2>&1; then
rtk proxy true || true
fi
echo "== files =="
git ls-files | rg '(^README\.md$|^docs/MODEL_CARD\.md$|^resources/versions\.json$|ExperimentCampaignV1|run.*continuous|continuous.*loop|c20260808)' || true
echo "== current changed files stat =="
git status --short
echo "== README relevant section =="
sed -n '1340,1395p' README.md
echo "== MODEL_CARD relevant section =="
sed -n '2000,2050p' docs/MODEL_CARD.md
echo "== occurrences of campaign names =="
rg -n "continuous-loop-20260808-continuous-openui-schedu-121086e4-c([12]|component-plan]|bounds]|control])|c20260808-continuous-openui-schedu-121086e4-c([12]|component-plan]|bounds]|control])|continuous autotrain note" README.md docs/MODEL_CARD.md docs/MODEL_CARD.md || true
echo "== run metadata directories for campaign =="
find runs -type d -name '*c20260808-continuous-openui-schedu-121086e4*' -print 2>/dev/null | sort || true
echo "== all checkpoints last.pt under matching run paths =="
find runs -path '*c20260808-continuous-openui-schedu-121086e4*' -name 'last.pt' -print 2>/dev/null | sort || true
echo "== version invariants docs if any =="
sed -n '1,220p' docs/design/decode-invariants.md 2>/dev/null || trueRepository: Tyler-R-Kendrick/slm-training
Length of output: 36859
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
import json, os
paths = [
"docs/design/continuous-loop-20260808-continuous-openui-schedu-121086e4-c1-results.json",
"docs/design/continuous-loop-20260808-continuous-openui-schedu-121086e4-c2-results.json",
]
need_keys = ["campaign", "arm", "run", "recipe", "roster", "evaluations", "honesty", "honesty_mode", "checkpoint"]
for path in paths:
try:
data = json.load(open(path))
except Exception as e:
print(path, "ERROR", repr(e))
continue
print(f"--- {path} ---")
for key in need_keys:
print(key, type(data.get(key)).__name__, "value" if isinstance(data.get(key), str) else repr(data.get(key)))
# print available top-level keys for c2
if path.endswith("-c2-results.json"):
print("top-level keys:", ", ".join(str(k) for k in sorted(data.keys())))
PY
echo "== c1 and c2 result markdown headers =="
sed -n '1,260p' docs/design/continuous-loop-20260808-continuous-openui-schedu-121086e4-c1-results.md
printf '\n================ c2 ================\n'
sed -n '1,320p' docs/design/continuous-loop-20260808-continuous-openui-schedu-121086e4-c2-results.md
echo "== campaign names in c1/c2 result files =="
rg -n "continuous-loop-20260808-continuous-openui-schedu-121086e4-c[12]|component-plan|control|c2component|c2-control|c2 component" docs/design/continuous-loop-20260808-continuous-openui-schedu-121086e4-c*results.* || trueRepository: Tyler-R-Kendrick/slm-training
Length of output: 6841
Complete the c1 and c2 autotrain records in both model-card surfaces.
The c1 and c2 entries only contain campaign, checkpoint, and honesty text. Add the required roster, evaluations, and recipe entries, or canonical links to the documented result artifacts. For c2, link both arms (continuous-loop-20260808-continuous-openui-schedu-121086e4-c2-component-plan and continuous-loop-20260808-continuous-openui-schedu-121086e4-c2-control), or document in both files that the control checkpoint/result is unavailable.
📍 Affects 2 files
README.md#L1368-L1378(this comment)docs/MODEL_CARD.md#L2024-L2034
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@README.md` around lines 1368 - 1378, The c1 and c2 continuous autotrain
records are incomplete in both model-card surfaces. Update README.md lines
1368-1378 and docs/MODEL_CARD.md lines 2024-2034 with the required roster,
evaluations, and recipe details or canonical result-artifact links; for c2,
include both the component-plan and control arms, or explicitly document that
the control result is unavailable in both files.
Sources: Coding guidelines, Learnings
Tyler-R-Kendrick
changed the title
5 tasks
…uous-loop-20260808-continuous-openui-schedu-121086e4-c1 closeout
…uous-loop-20260808-continuous-openui-schedu-121086e4-c2 closeout
The continuous driver's document-closeout commit and the self_heal_doc_commit helper hardcoded -c user.email=autotrain@local -c user.name=autotrain, which lands every automated cycle's docs commit as GitHub-unverified regardless of the session's real git identity. Drop the override so these commits inherit the repo's configured (signed/verified) committer. Also regenerate the committed semantic-floor-gate-v1 artifact, which had drifted stale against harness.experiments v148 and was failing publish --check (blocking every commit that touches this test shard, including this one); verdict stays inconclusive, gate behavior unchanged.
…uous-loop-20260808-continuous-openui-schedu-121086e4-c3 closeout
…uous-loop-20260808-continuous-openui-schedu-121086e4-c4 closeout
…uous-loop-20260808-continuous-openui-schedu-121086e4-c5 closeout
Tyler-R-Kendrick
force-pushed
the
claude/great-dirac-qjv1cv
branch
from
August 9, 2026 04:05
7968516 to
600a948
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Scheduled
/autotraincontinuous-loop run (continuous-openui-scheduled-0808z1, fixture-scalewf_smoke_v2, 20 steps), 5 cycles.run_autotrain_continuous._git_add_and_commit_paths()and theself_heal_doc_commithelper hardcoded-c user.email=autotrain@local -c user.name=autotrainon the self-heal document-closeout commit, so every automated cycle's docs commit landed as GitHub-unverified regardless of the session's real git identity. Dropped the override so these commits inherit the repo's configured (verified) committer — confirmed working on cycles 3-5 of this run.harness.autoresearch.experiment_campaigngets a no-bump registry note (behavior-neutral, commit-identity only).docs/design/semantic-floor-gate-v1.json/.mdhad drifted stale againstharness.experiments(v147 → v148) and was failingpublish_semantic_floor_gate --check, blocking every commit that touches this test shard (including this one). Regenerated; verdict staysinconclusive, thresholds/gate behavior unchanged. No-bump registry note added.c20260808-...-c1through-c5): all non-positive at fixture scale —insufficient_n, null/sub-threshold primary-metric deltas, and twomeasurement_incompleteretries (missing scoreboard / decode timeout on the matched control), correctly self-healed by the driver (retry_measurement,thrash_bank_compose) without a hard block. No suite ever clears the honest ship gate at this scale, as expected for fixture-size smoke screening. No metric win, so per this repo's SDLC autotrain-iteration-delivery rules, no stack layer was opened for the training results themselves — only the mandatory iron-lawdocs/design/results docs plus the two harness/gate fixes discovered while running the loop.blocker_count=0throughout) to check in per this scheduled routine's cadence; no repeated-blocker condition was hit.Docs
docs/design/continuous-loop-20260808-continuous-openui-schedu-121086e4-c{1,2,3,4,5}-results.{md,json}docs/MODEL_CARD.md/README.md— checkpoint-note-only honesty stub updatessrc/slm_training/resources/versions.json— no-bump entries for both touched componentsTest plan
verify_version_stamps --check --stagedpassesgit merge-base --is-ancestor origin/main HEAD— clean rebase, no divergenceClaude <noreply@anthropic.com>authorship (rebased the first two to fix pre-fixautotrain@localattribution)