feat(evaluations): add evaluation API reporting #91

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

gabrielelpidio merged 15 commits into main from gabriel/evals-api

Nov 4, 2025

Collaborator

gabrielelpidio commented Sep 30, 2025

No description provided.

gabrielelpidio marked this pull request as draft

September 30, 2025 03:20

pkg-pr-new bot commented Sep 30, 2025 •

edited

Loading

Open in StackBlitz

npm i https://pkg.pr.new/axiomhq/ai/axiom@91

commit: ced999e

thesollyz reviewed

View reviewed changes

packages/ai/src/evals/eval.ts Outdated Show resolved Hide resolved

thesollyz reviewed

View reviewed changes

packages/ai/src/evals/eval.ts Outdated Show resolved Hide resolved

thesollyz reviewed

View reviewed changes

packages/ai/src/evals/eval.ts Outdated Show resolved Hide resolved

c-ehrlich reviewed

View reviewed changes

packages/ai/src/evals/eval.ts Outdated Show resolved Hide resolved

thesollyz reviewed

View reviewed changes

packages/ai/src/evals/eval.ts Outdated Show resolved Hide resolved

c-ehrlich force-pushed the main branch from e60875e to 7fe0fb8 Compare

October 10, 2025 05:50

gabrielelpidio force-pushed the gabriel/evals-api branch 3 times, most recently from 5d7817f to 53279cf Compare

October 15, 2025 19:37


          feat: (evals) add axiom reporting step

474d31b

gabrielelpidio force-pushed the gabriel/evals-api branch from 53279cf to 474d31b Compare

October 15, 2025 22:21


          feat:(evals): add resources url builder and override

da2a525

gabrielelpidio force-pushed the gabriel/evals-api branch from e9e3256 to da2a525 Compare

October 15, 2025 23:17

gabrielelpidio and others added 4 commits

October 15, 2025 19:25


          feat(evals): send version to api reporting

29480f6


          fix(evals): correct evaluation status logic in registerEval function

00ecfb4


          fix eval name

5ac56b5


          Merge branch 'main' into gabriel/evals-api

7ed7850

gabrielelpidio requested a review from c-ehrlich

October 24, 2025 02:10


          Merge branch 'main' into gabriel/evals-api

d0bd298

gabrielelpidio marked this pull request as ready for review

October 24, 2025 02:10

gabrielelpidio changed the title ~~feat(ai): add evaluation API reporting~~ feat(evaluations): add evaluation API reporting

c-ehrlich reviewed

View reviewed changes

packages/ai/src/evals/eval.service.ts Outdated Show resolved Hide resolved

packages/ai/src/config/resolver.ts Outdated Show resolved Hide resolved

packages/ai/src/evals/eval.ts Show resolved Hide resolved

packages/ai/src/evals/eval.ts Outdated Show resolved Hide resolved

packages/ai/src/config/index.ts Outdated Show resolved Hide resolved

gabrielelpidio added 6 commits

October 24, 2025 00:00


          refactor(evals): remove resourcesUrl from types and rename to console…

ace514a

…EndpointUrl


          refactor(evals): remove unnecessary readonly

60907d9


          fix(evals): rename __overrideResourcesUrl to __overrideEndpointUrl

6b3b63c


          fix(evals): set evaluation status to 'completed' in registerEval func…

d136871

…tion


          refactor(evals): rename buildResourcesUrl to buildConsoleUrl

d39ca73


          refactor(evals): update AxiomEvalInstrumentationOptions to remove res…

e238475

…ourcesUrl

gabrielelpidio requested a review from c-ehrlich

October 27, 2025 15:54

c-ehrlich approved these changes

View reviewed changes

packages/ai/src/config/index.ts Outdated Show resolved Hide resolved

packages/ai/src/config/resolver.ts Show resolved Hide resolved

packages/ai/src/evals/eval.ts Outdated Show resolved Hide resolved

packages/ai/src/evals/eval.ts Outdated Show resolved Hide resolved

gabrielelpidio added 2 commits

November 3, 2025 18:30


          refactor(evals): remove region property from EvaluationApiPayloadBase…

b06ca2f

… and update scorers mapping


          Merge remote-tracking branch 'origin/main' into gabriel/evals-api

ced999e

c-ehrlich approved these changes

View reviewed changes

gabrielelpidio merged commit e50e125 into main

6 checks passed

gabrielelpidio deleted the gabriel/evals-api branch

November 4, 2025 04:36

axiom-automation mentioned this pull request

chore(main): release axiom 0.23.0 #113

Merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet