Skip to content

MAINT: Make ScorerEvaluator registry update dispatch exhaustive (P311-03) - #4045

Merged
Roman Lutz (romanlutz) merged 3 commits into
microsoft:mainfrom
Shizoqua:maint/scorer-eval-registry-dispatch
Nov 8, 2026
Merged

Roman Lutz (romanlutz) merged 3 commits into
microsoft:mainfrom
Shizoqua:maint/scorer-eval-registry-dispatch

Conversation

@Shizoqua

Copy link
Copy Markdown
Contributor

Description

Implements the P311-03 case for RegistryUpdateBehavior dispatch in ScorerEvaluator.run_evaluation_async, claimed and assigned in #3166.

The method checked SKIP_IF_EXISTS and NEVER_UPDATE with == and !=, so ALWAYS_UPDATE and any future member fell through implicitly. It now matches each member explicitly and adds a TYPE_CHECKING-only assert_never guard, following the existing pattern in ScorerEvaluator.from_scorer (#3924).

Behavior is unchanged, including the runtime fallback for values outside the enum (run the evaluation and write to the registry).

Tests and Documentation

Added a parametrized test in tests/unit/score/test_scorer_evaluator.py covering all three members and the out-of-enum fallback: whether existing metrics are checked and whether the registry is written. It passes on both the old and new code, confirming no behavior change.

No docs or notebooks changed, so JupyText was not needed.

Copilot AI balanced review requested due to automatic review settings November 7, 2026 18:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the registry-update dispatch change.

No open issues 🎉

What changed in this PR

Makes registry-update dispatch in ScorerEvaluator.run_evaluation_async explicit and statically exhaustive for P311-03, following the existing from_scorer pattern while preserving runtime behavior.

Changes:

  • Handles all three RegistryUpdateBehavior members explicitly, with a type-checking-only assert_never guard and the existing out-of-enum fallback.
  • Adds parametrized coverage for existing-metrics checks and registry writes across all supported modes and the fallback.
File Description
tests/​unit/​score/​test_scorer_evaluator.py Adds registry-update behavior regression coverage.
pyrit/​score/​scorer_evaluation/​scorer_evaluator.py Replaces implicit dispatch with exhaustive matching and explicit behavior flags.

Review effort: Balanced

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@romanlutz
Roman Lutz (romanlutz) added this pull request to the merge queue Nov 8, 2026
Merged via the queue into microsoft:main with commit 551fdfd Nov 8, 2026
59 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants