docs(autotrain): continuous-openui-scheduled-bcz5ho c1-c8 closeout (non-positive) - #1501
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Warning Review limit reached
Next review available in: 46 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (17)
📝 WalkthroughWalkthroughThe pull request adds a fixture-only continuous autotrain screening record for the 2026-08-08 OpenUI campaign. It records candidate and control metrics, rejection reasons, checkpoint provenance, non-promotion status, and unchanged SLM-228 behavior. ChangesContinuous autotrain screening
Estimated code review effort: 1 (Trivial) | ~5 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@docs/design/continuous-loop-20260808-openui-scheduled-bcz5ho-5720c848-c1-results.json`:
- Around line 37-41: Preserve the numeric sample-size gate inputs in both result
artifacts: in
docs/design/continuous-loop-20260808-openui-scheduled-bcz5ho-5720c848-c1-results.json
lines 37-41, add canonical observed and required sample-count fields for both
arms, including the rejected values actual=3 and need>=20; in
docs/design/continuous-loop-20260808-openui-scheduled-bcz5ho-5720c848-c1-results.md
lines 9-11, render those same counts in the historical narrative.
In `@README.md`:
- Around line 796-800: Update README.md lines 796-800 to add campaign
continuous-loop-20260808-openui-scheduled-bcz5ho-5720c848-c1 to the model-card
summary with roster, evaluations, recipe metadata, honesty/no-sync status,
non-promotion status, and both exact checkpoint paths. Update docs/MODEL_CARD.md
lines 1452-1456 with newest-first checkpoint-history coverage and the
corresponding roster and evaluation sections, preserving the same metadata and
resolvable paths at both canonical model-card surfaces.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 3e7c1782-295b-49a0-985a-764da6a6c446
📒 Files selected for processing (5)
README.mddocs/MODEL_CARD.mddocs/design/continuous-loop-20260808-openui-scheduled-bcz5ho-5720c848-c1-results.jsondocs/design/continuous-loop-20260808-openui-scheduled-bcz5ho-5720c848-c1-results.mdsrc/slm_training/resources/versions.json
| "reasons": [ | ||
| "fixture_insufficient_n:c20260808-openui-scheduled-bcz5ho-5720c848-c1-bounds", | ||
| "fixture_insufficient_n:c20260808-openui-scheduled-bcz5ho-5720c848-c1-control", | ||
| "primary_metric_null_or_worse:smoke.structural_similarity:control=0.057499999999999996 candidate=0.057499999999999996 improvement=0.0", | ||
| "fixture_insufficient_n_alone" |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Preserve the numeric sample-size gate inputs in both result artifacts.
The rejection currently depends on actual=3 and need>=20, but neither artifact stores those values. Add canonical sample-count fields to the JSON and render the same values in the Markdown.
docs/design/continuous-loop-20260808-openui-scheduled-bcz5ho-5720c848-c1-results.json#L37-L41: add observed and required sample counts for both arms.docs/design/continuous-loop-20260808-openui-scheduled-bcz5ho-5720c848-c1-results.md#L9-L11: include the same counts in the historical narrative.
📍 Affects 2 files
docs/design/continuous-loop-20260808-openui-scheduled-bcz5ho-5720c848-c1-results.json#L37-L41(this comment)docs/design/continuous-loop-20260808-openui-scheduled-bcz5ho-5720c848-c1-results.md#L9-L11
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@docs/design/continuous-loop-20260808-openui-scheduled-bcz5ho-5720c848-c1-results.json`
around lines 37 - 41, Preserve the numeric sample-size gate inputs in both
result artifacts: in
docs/design/continuous-loop-20260808-openui-scheduled-bcz5ho-5720c848-c1-results.json
lines 37-41, add canonical observed and required sample-count fields for both
arms, including the rejected values actual=3 and need>=20; in
docs/design/continuous-loop-20260808-openui-scheduled-bcz5ho-5720c848-c1-results.md
lines 9-11, render those same counts in the historical narrative.
Tyler-R-Kendrick
changed the title
Tyler-R-Kendrick
changed the title
482995c to
a390a5e
Compare
…0260808-openui-scheduled-bcz5ho-5720c848-c3 closeout
…0260808-openui-scheduled-bcz5ho-5720c848-c6 closeout
2 participants
Summary
Tracks incremental commits from a scheduled
/autotraincontinuous-loop session (loop-idopenui-scheduled-bcz5ho), run fixture-scale and local (no paid/remote compute — no session authority was granted for that; environment has no GPU). This was a fresh container (no venv, nonode_modules), so this firing also bootstrapped the Python venv (.venv,pip install -e ".[dev,hf]") and the OpenUI/DESIGN.md JS bridges before the first cycle.Eight cycles ran this firing, all non-positive per the SDLC Phase A gate (no primary-metric win, no ship-quality win, no unblock) — so this PR is documentation/provenance only, not a model or harness change:
c1(bounds/control, TwoTower, fixturewf_smoke_v2): honest ship-gate reject on fixture n (insufficient_n actual=3 need>=20); control/bounds landed on an identical primary metric (structural_similarity=0.0575).c2(control/component-plan): evaluation measurement incomplete —decode_timeout_count=3on control (CPU-only, no-GPU, 3-minute wall cap), missing scoreboard on the component-plan candidate.c3(frozen replay ofc2): retried the identical incomplete arm; control hit the same decode-timeout signature again (2nd consecutive occurrence on that arm — one more identical occurrence would have crossed the loop's repeated-hard-block threshold; it did not recur on that specific arm again this firing).c4(control/bounds): honest ship-gate reject on fixture n; null primary-metric delta (0.4167both arms).c5(control/component-plan): measurement incomplete, missing scoreboard on the candidate (soft failure, different arm than c2/c3).c6(frozen replay ofc5, retry_measurement): component-plan hit a decode-timeout this time (decode_timeout_count=3); still measurement-incomplete.c7(control/canvas): honest ship-gate reject on fixture n; null primary-metric delta (0.0575both arms).c8(control/component-plan): honest ship-gate reject on fixture n; a smallmpr_per_msefficiency gain (7.89e-5→8.18e-5,+3.6%) was measured but rejected as sub-threshold (needs ≥5%).No cycle crossed the 3-consecutive-identical-hard-block threshold, so the loop kept going in-process rather than reporting
blocked. Two distinct arms (c2/c3's control, c6's component-plan) each hit CPU-only decode-timeout signatures — worth watching on a future firing since a 3rd identical occurrence on the same arm would be a real infra signal to route throughimprove-openui-harnesses.Per
sdlc/autotrain-iteration-delivery, non-positive cycles only get local commits + iron-law docs, never a new stack layer. Each cycle's doc note to README/MODEL_CARD also required ano-bump:version-stamp history entry forharness.experiments.slm228_spectral_disposition(behavior unchanged) perscripts.verify_version_stamps.Note: this environment does not have the
ghCLI available, so this PR is opened directly againstmain(nogh stack) — matching how prior non-positive screening batches in this repo have landed when a stack tool wasn't available. Commit authorship on the driver's self-heal commits was rebased/amended to the session's identity before each push (GitHub verified-commit requirement).This firing is now wrapping up (no positive-result cycle landed, so there is no stack layer to close out per Phase B — only this docs-only PR). The next scheduled firing will continue the loop from a fresh container.
Test plan
python -m scripts.verify_version_stamps --check --staged→ok(each cycle)tests/test_scripts/test_verify_checkpoint_references.py,tests/test_versioning/*)