Skip to content

Latest commit

 

History

History
53 lines (47 loc) · 4.26 KB

File metadata and controls

53 lines (47 loc) · 4.26 KB

Evaluation

Generated by npm run eval:report on 2026-08-27 from the latest skillgrade results under evals/results/ (git-ignored — run npm run eval:smoke locally first; see ../evals/README.md).

Tasks: 37/37 · trials/task: 1 · mean reward 0.99 · skill discovered in 97% of trials · baseline (no skills) mean 0.82

category skill task reward (skills) pass loaded baseline (no skills) Δ outcome avg s
android ns-android-edge-to-edge android-edge-to-edge 1.00 1/1 1/1 0.27 +0.73 44
android ns-android-window-insets android-modal-insets 1.00 1/1 1/1 1.00 +0.00 62
core ns-binding-expressions binding-fixes 1.00 1/1 1/1 1.00 +0.00 218
core ns-color-formats theme-color-utils 1.00 1/1 1/1 1.00 +0.00 77
core ns-http-gotchas status-aware-http 1.00 1/1 1/1 1.00 +0.00 46
core ns-multi-window second-window 1.00 1/1 1/1 1.00 +0.00 412
core ns-navigation-transitions navigation-transitions 1.00 1/1 1/1 0.50 +0.50 20
core ns-observable-array-change-events feed-store-events 1.00 1/1 1/1 1.00 +0.00 203
core ns-observable-set-semantics profile-vm-bugs 1.00 1/1 1/1 1.00 +0.00 36
core ns-page-navigation-lifecycle detail-page-lifecycle 1.00 1/1 1/1 1.00 +0.00 234
core ns-web-globals cancellable-fetch 1.00 1/1 1/1 1.00 +0.00 42
core ns-xml-custom-components rating-component 1.00 1/1 1/1 0.67 +0.33 142
ios ns-ios-corespotlight ios-corespotlight 1.00 1/1 1/1 1.00 +0.00 183
ios ns-ios-safe-area-smudge ios-safe-area-smudge 1.00 1/1 1/1 0.00 +1.00 33
ios ns-ios-scroll-under-bars ios-scroll-under-bars 1.00 1/1 1/1 1.00 +0.00 16
ios ns-ios-widgets ios-widgets-live-activity 1.00 1/1 1/1 0.64 +0.36 80
testing ns-trace-debugging trace-bootstrap 1.00 1/1 1/1 1.00 +0.00 42
testing ns-unit-testing unit-testing-setup 1.00 1/1 1/1 1.00 +0.00 441
tooling ns-file-qualifiers tablet-layout-variant 1.00 1/1 1/1 1.00 +0.00 18
tooling ns-snippets snippets-search-tool 1.00 1/1 1/1 1.00 +0.00 51
tooling ns-uncaught-error-policy crash-policy-config 1.00 1/1 1/1 1.00 +0.00 44
ui ns-code-built-view-styling code-built-toast 1.00 1/1 1/1 0.10 +0.90 173
ui ns-css-keyframes card-entrance-css 1.00 1/1 1/1 1.00 +0.00 64
ui ns-css-selectors css-selector-fixes 1.00 1/1 1/1 1.00 +0.00 683
ui ns-css-variables-calc css-vars-calc-fixes 1.00 1/1 1/1 1.00 +0.00 19
ui ns-formatted-string-spans formatted-status-fix 0.75 1/1 0/1 1.00 -0.00 15
ui ns-gridlayout-spec-strings grid-spec-fix 1.00 1/1 1/1 0.33 +0.67 18
ui ns-input-accessory-dialogs composer-over-sheet 1.00 1/1 1/1 1.00 +0.00 251
ui ns-keyboard-input chat-composer 1.00 1/1 1/1 0.54 +0.46 164
ui ns-line-height wrapped-label-gaps 1.00 1/1 1/1 0.64 +0.36 26
ui ns-listview-recycling listview-jank 1.00 1/1 1/1 0.60 +0.40 32
ui ns-root-css-classes conditional-theming-css 0.86 1/1 1/1 0.82 +0.00 33
ui ns-root-layout-overlays sheet-overlay 1.00 1/1 1/1 1.00 +0.00 223
ui ns-tabview-lazy-loading tab-fetch-android 1.00 1/1 1/1 0.50 +0.50 26
ui ns-top-nav native-top-bar 1.00 1/1 1/1 1.00 +0.00 271
ui ns-touch-animations press-feedback 1.00 1/1 1/1 1.00 +0.00 38
ui ns-view-animations animation-helpers 1.00 1/1 1/1 0.73 +0.27 250

Reward = 0.25 × (skill loaded) + 0.75 × (weighted outcome checks). "Δ outcome" compares outcome checks only, skills on vs. off.

Checks that did not always pass (skills on)

task check pass example
conditional-theming-css light default on bare selector, not .ns-light 0/1 .ns-light is absent on iOS <= 12 — defaults belong on bare selectors