Add deterministic skill selection to Impeccable reviewer - #54575
Merged
Conversation
Contributor
|
PR Triage: category=docs · risk=low · priority=low · score=9/100
|
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Add concrete skill-selection table to improve output density
Add deterministic skill selection to Impeccable reviewer
Aug 21, 2026
pelikhan
approved these changes
Aug 21, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
Adds deterministic Impeccable skill selection to produce more targeted PR reviews.
Changes:
- Pins the official Impeccable skill and adds selection guidance.
- Adds fallback behavior and contract coverage.
- Regenerates the compiled workflow.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/impeccable-skills-reviewer.md |
Adds skill installation and deterministic review selection. |
.github/workflows/impeccable-skills-reviewer.lock.yml |
Regenerates the compiled workflow. |
pkg/cli/pr_prefetch_cache_contract_test.go |
Adds prompt contract assertions. |
Review details
Suppressed comments (2)
.github/workflows/impeccable-skills-reviewer.md:130
- This tests-only signal omits Go's standard
*_test.gonaming convention, even though this repository is predominantly Go and the sibling reviewer handles that pattern (.github/workflows/mattpocock-skills-reviewer.md:144). A Go test-only PR will therefore fall through to a keyword or catch-all classification instead oftests_only.
| Only test files (`*.test.*`, `*.spec.*`, `test/**`, `tests/**`) | `tests_only` | `audit` |
.github/workflows/impeccable-skills-reviewer.md:134
- Because rows are evaluated top-down, this copy-only rule loses to the broad keyword rules above it. For example, a PR that only fixes a button-label typo and has “fix” in its title is classified as
bug_fixand getsharden/auditrather thanclarify. Evaluate the exclusive documentation/copy rule before the keyword-based rows, or explicitly exclude copy-only changes from those rows.
| Only documentation or copy changes | `documentation` | `clarify` |
- Files reviewed: 3/3 changed files
- Comments generated: 4
- Review effort level: Balanced
| 4. Select the most relevant skills for the detected change type and risk areas. | ||
| Use the inline mode guidance above by default. Read the installed `SKILL.md` only when that guidance is insufficient. | ||
|
|
||
| 4. Select 1–2 Impeccable modes using the first matching row: |
Comment on lines
+65
to
+72
| assert.Contains(t, text, "using the first matching row", "Impeccable reviewer should select modes deterministically") | ||
| assert.Contains(t, text, "`tests_only`") | ||
| assert.Contains(t, text, "`bug_fix`") | ||
| assert.Contains(t, text, "`new_feature`") | ||
| assert.Contains(t, text, "`refactor_cleanup`") | ||
| assert.Contains(t, text, "`documentation`") | ||
| assert.Contains(t, text, "`mixed_unclear`") | ||
| assert.Contains(t, text, "Select 1–2 Impeccable modes", "Impeccable reviewer should limit selected modes") |
Comment on lines
+89
to
+91
| ## Available Impeccable Review Modes | ||
|
|
||
| The installed `/impeccable` skill provides these review-relevant modes: |
| assert.Contains(t, text, "Test Quality Sentinel skipped because pre-fetch PR data was unavailable", "Test Quality Sentinel prompt should instruct the agent to noop with the fallback reason") | ||
| } | ||
|
|
||
| func TestImpeccableSkillsReviewerHasDeterministicSkillSelectionGuidance(t *testing.T) { |
Contributor
|
🎉 This pull request is included in a new release. Release: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Impeccable reviewer lacked concrete skill-selection guidance, leading to generic reviews and fewer actionable comments than comparable workflows.
Deterministic selection
Reliable skill guidance
Regression coverage