feat: better scorers

[c-ehrlich]

No description provided.

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/axiomhq/ai/axiom@106

commit: 7668499

[Copilot]

Pull Request Overview

This PR refactors the scorer system to improve type inference and ergonomics. The key changes separate scorer definitions from usage requirements, introduce automatic type inference from function arguments, and simplify the API by removing the need for explicit type parameters in most cases.

• Introduces ScorerLike for flexible scorer consumption and Scorer for strict scorer definitions with a name property
• Implements automatic type inference in createScorer based on the callback's argument types
• Updates scorer factory to attach the name as a property instead of embedding it in the returned score

Reviewed Changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
packages/ai/src/evals/scorers.ts	Separates Score and ScoreWithName types, introduces ScorerLike vs Scorer distinction with TExtra support
packages/ai/src/evals/scorer.factory.ts	Refactors createScorer to infer types from arguments and attach name as function property
packages/ai/test/evals/scorer.types.test.ts	Adds comprehensive type inference tests for the new scorer system
packages/ai/src/evals/eval.types.ts	Updates type references and strengthens OutputOf type constraints
packages/ai/src/evals/eval.ts	Updates to use ScorerLike and extract scorer name from function property
packages/ai/src/evals/builder.ts	Adds TODO comment about unused function
packages/ai/README.md	Documents Node version requirement for evals
examples/example-evals-nextjs/test/feature.eval.ts	Migrates to new Scorer factory pattern
examples/example-evals-nextjs/src/lib/scorers.ts	Updates scorers to use typed arguments and removes optional expected handling
examples/example-evals-nextjs/src/lib/capabilities/classify-ticket/evaluations/ticket-classification.eval.ts	Adds wrapped autoevals scorer example
.github/workflows/ci.yaml	Reorders CI steps to run format check before build

Files not reviewed (1)

• pnpm-lock.yaml: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Using any type annotation bypasses type safety. Consider using the inferred return type or a more specific type annotation for the scorer function.

Suggested change

	const scorer: any = (args: TArgs) => {
	const scorer = (args: TArgs) => {

[Copilot]

Using as any type assertion bypasses type checking. This could hide type mismatches between the data's expected value and what the scorer expects.

Suggested change

	expected: data.expected as any,
	expected: data.expected,