Skip to content

Backfill benchmark scores for ~1,000 CPUs and ~900 GPUs - #288

Merged
Seungpyo1007 merged 3 commits into
developfrom
data/benchmarks-new-records
Sep 26, 2026
Merged

Backfill benchmark scores for ~1,000 CPUs and ~900 GPUs#288
Seungpyo1007 merged 3 commits into
developfrom
data/benchmarks-new-records

Conversation

@Seungpyo1007

@Seungpyo1007 Seungpyo1007 commented Sep 26, 2026 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Member

Benchmark backfill with the existing TechEngine app.ingest.enrich runner, focused on the CPUs and GPUs added on 2026-09-25. The runner only fills null fields: no existing score was overwritten (checked on the diff).

Source Component Records filled
topcpu.net (Cinebench R23/2024, Geekbench 6, PassMark) CPU 395
spec.org CPU2006 result tables (max base) CPU 646
cgdirector Cinebench CPU 1
opendata.blender.org (CC0) GPU 151
videocardbenchmark.net PassMark G3D GPU 764
topcpu.net Time Spy (2011+ cards only) GPU 338
topcpu.net Time Spy Extreme / Speed Way / OctaneBench / FP32 (2011+ only) GPU 334

Checks:

  • CPU: no record has a single-thread score above its multi-thread score.
  • GPU (G3D vs Time Spy): a log-ratio MAD check found 13 outliers. They are mostly TeraScale/Tesla-era cards, which can't run DirectX 12 benchmarks, so topcpu serves estimates for them.
  • Cleanup: Time Spy / Time Spy Extreme / Speed Way are now nulled on every pre-DX12 architecture plus those outliers (14 records, a few pre-existing; the P102-100 mining card is kept).
  • GPU (G3D vs FP32): 11 outliers. All are expected (datacenter cards with high TFLOPS, or 2007-era low-end parts), not variant mismatches.

Closes #1

@Seungpyo1007
Fill Cinebench R23/2024, Geekbench 6 and PassMark scores for 395 CPUs…
6c9c45b 
… from topcpu.net
@Seungpyo1007
Fill SPEC CPU2006 base scores for 646 CPUs from spec.org result tables
98f447d
@Seungpyo1007
Fill Blender, PassMark G3D, Time Spy and topcpu GPU scores; drop DX12…
d1589ef 
… scores on pre-DX12 GPUs

Blender (opendata.blender.org), PassMark G3D (videocardbenchmark.net), Time Spy / Speed Way /
OctaneBench / FP32 (topcpu.net, 2011+ cards only). Time Spy, Time Spy Extreme and Speed Way need
DirectX 12, so values on TeraScale/Tesla-and-older cards are aggregator estimates, not runs;
those are nulled (14 records, a few of them pre-existing).
@Seungpyo1007 Seungpyo1007 added data Dataset changes enhancement New feature or request labels Sep 26, 2026
@github-actions github-actions Bot mentioned this pull request Sep 26, 2026
Open
@TechEngineBot

TechEngineBot commented Sep 26, 2026

Copy link
Copy Markdown
Member

🔎 Data verification — Tier 0 (offline existence/trust)

Scored by app.verify; posted by TechEngineBot. Informational only — the structural gate (app.validate) is separate and authoritative for merge.

Changed records in this PR

2007 record(s) scored.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 1915
    "Yellow" : 92
    "Red" : 0
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
soc 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 65 1 64 0 1.5%
tablet 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 4 0 4 0 0.0%
gpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨 924 909 15 0 98.4%
cpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨 1014 1005 9 0 99.1%
All 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨 2007 1915 92 0 95.4%

Full-dataset baseline

106343 record(s) scored.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 24527
    "Yellow" : 80734
    "Red" : 1082
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
brand 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨 207 122 85 0 58.9%
soc 🟩🟨🟨🟨🟨🟨🟥🟥🟥🟥🟥🟥 2105 254 843 1008 12.1%
smartphone 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 93407 17956 75383 68 19.2%
tablet 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 3459 292 3166 1 8.4%
watch 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 442 80 358 4 18.1%
pda 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨 140 83 57 0 59.3%
gpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨 2076 1529 547 0 73.7%
cpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟥 4507 4211 295 1 93.4%
All 🟩🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟥 106343 24527 80734 1082 23.1%

Hard violations (forced red):

Count Check
1 !boost_ge_base

green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to verified runs in the scheduled verify-network workflow.

@Seungpyo1007
Seungpyo1007 merged commit 0e92656 into develop Sep 26, 2026
5 checks passed
@Seungpyo1007
Seungpyo1007 deleted the data/benchmarks-new-records branch September 26, 2026 13:06
@TechEngineBot

TechEngineBot commented Sep 26, 2026

Copy link
Copy Markdown
Member

TechEngine change review: PASS

Check Result
python -m app.validate PASS
New hard integrity anomalies vs PR base PASS

Changed data

Category Added Modified Deleted Added verified Added unverified Added Kaggle-sourced
brand 0 0 0 0 0 0
soc 0 0 0 0 0 0
smartphone 0 0 0 0 0 0
tablet 0 0 0 0 0 0
watch 0 0 0 0 0 0
pda 0 0 0 0 0 0
gpu 0 924 0 0 0 0
cpu 0 1014 0 0 0 0

Changed record examples

gpu modified

  • gpu/amd/2007/enterprise/firestream-9170.json - FireStream 9170
  • gpu/amd/2008/enterprise/firestream-9250.json - FireStream 9250
  • gpu/amd/2008/enterprise/firestream-9270.json - FireStream 9270
  • gpu/amd/2009/consumer/radeon-hd-5850.json - Radeon HD 5850
  • gpu/amd/2010/consumer/radeon-hd-5670.json - Radeon HD 5670
  • gpu/amd/2010/consumer/radeon-hd-6370m.json - Radeon HD 6370M
  • gpu/amd/2010/consumer/radeon-hd-6850.json - Radeon HD 6850
  • gpu/amd/2010/enterprise/firepro-rg220.json - FirePro RG220
  • gpu/amd/2010/enterprise/firestream-9370.json - FireStream 9370
  • gpu/amd/2011/consumer/radeon-e6460.json - Radeon E6460
  • gpu/amd/2011/consumer/radeon-hd-6230.json - Radeon HD 6230
  • gpu/amd/2011/consumer/radeon-hd-6250.json - Radeon HD 6250
  • gpu/amd/2011/consumer/radeon-hd-6290.json - Radeon HD 6290
  • gpu/amd/2011/consumer/radeon-hd-6350.json - Radeon HD 6350
  • gpu/amd/2011/consumer/radeon-hd-6450.json - Radeon HD 6450
  • ... 909 more

cpu modified

  • cpu/amd/2009/consumer/amd-athlon-64-fx-60-dual-core.json - AMD Athlon 64 FX-60 Dual Core
  • cpu/amd/2009/consumer/amd-athlon-64-fx-62-dual-core.json - AMD Athlon 64 FX-62 Dual Core
  • cpu/amd/2009/consumer/amd-phenom-ii-x3-710.json - AMD Phenom II X3 710
  • cpu/amd/2009/consumer/amd-phenom-ii-x3-720.json - AMD Phenom II X3 720
  • cpu/amd/2009/consumer/amd-phenom-ii-x4-805.json - AMD Phenom II X4 805
  • cpu/amd/2009/consumer/amd-phenom-ii-x4-810.json - AMD Phenom II X4 810
  • cpu/amd/2009/consumer/amd-phenom-ii-x4-910.json - AMD Phenom II X4 910
  • cpu/amd/2009/consumer/amd-phenom-ii-x4-920.json - AMD Phenom II X4 920
  • cpu/amd/2009/consumer/amd-phenom-ii-x4-925.json - AMD Phenom II X4 925
  • cpu/amd/2009/consumer/amd-phenom-ii-x4-940.json - AMD Phenom II X4 940
  • cpu/amd/2009/consumer/amd-phenom-ii-x4-955.json - AMD Phenom II X4 955
  • cpu/amd/2009/enterprise/amd-opteron-146.json - AMD Opteron 146
  • cpu/amd/2009/enterprise/amd-opteron-2218.json - AMD Opteron 2218
  • cpu/amd/2009/enterprise/amd-opteron-2220-se.json - AMD Opteron 2220 SE
  • cpu/amd/2009/enterprise/amd-opteron-2220.json - AMD Opteron 2220
  • ... 999 more

Heuristic review

  • Heuristic warnings: none found.

@TechEngineBot

TechEngineBot commented Sep 26, 2026

Copy link
Copy Markdown
Member

TechEngine validation stats: PASS

Data summary

Category Total Verified Unverified Missing verified Tracked Verified % of tracked
brand 207 122 85 0 207 58.9%
soc 2105 193 1912 0 2105 9.2%
smartphone 93407 17501 75906 0 93407 18.7%
tablet 3459 273 3186 0 3459 7.9%
watch 442 39 403 0 442 8.8%
pda 140 74 66 0 140 52.9%
gpu 2076 1026 1050 0 2076 49.4%
cpu 4507 2869 1638 0 4507 63.7%
all 106343 22097 84246 0 106343 20.8%

Warning

Tracked verified coverage is below 50% for tablet 7.9% (273/3459), watch 8.8% (39/442), soc 9.2% (193/2105), smartphone 18.7% (17501/93407), all 20.8% (22097/106343), gpu 49.4% (1026/2076).
Tracked coverage excludes records missing the verified field; see the Missing verified column for those records.
This does not fail validation. Keep imported records verified: false until manual audit, but treat this as follow-up verification work before relying on the affected categories as curated data.

Validation notes

  • The integrity gate blocks only hard anomalies introduced by this PR relative to its merge base.
  • Full advisory outlier listings are suppressed on successful runs because they are dataset-wide and mostly stable between PRs.
  • Failure runs still include a detailed log excerpt for debugging.

Key output:

## app.validate
## integrity_check.py (PR head compared with PR base)
loaded CPU=4507 GPU=2076
Integrity section Flagged lines
structural 18
CPU name/tier consistency (desktop mainstream only) 0
CPU single>multi (cinebench/geekbench — should be multi>=single) 0
CPU era-vs-score outliers 8
CPU cross-source ratio outliers (possible wrong-variant) 131
GPU cross-source ratio outliers + sanity 51

This was referenced Sep 26, 2026
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@Seungpyo1007 Seungpyo1007

@TechEngineBot TechEngineBot

Labels

data Dataset changes enhancement New feature or request

Projects

TechAPI-Project
Status: Done

Milestone

Massive dataset rebuild (1989-2026)

Development

Successfully merging this pull request may close these issues.

TechAPI dataset roadmap and status: all categories (1989-2026)

2 participants

@Seungpyo1007 @TechEngineBot