Skip to content

ci: send the PR base to TechEngine checks and stop tracking scores.jsonl - #294

Merged
Seungpyo1007 merged 1 commit into
developfrom
ci/bot-full-coverage-relay
Sep 26, 2026
Merged

Seungpyo1007 merged 1 commit into
developfrom
ci/bot-full-coverage-relay

Conversation

@Seungpyo1007

Copy link
Copy Markdown
Member

TechAPI side of GetTechAPI/TechEngine#95 (TechEngineBot full coverage), from the 2026-09-27 PR/issue audit.

With #95, bot comments cover all 12 categories (191,797 records) instead of 8 categories, and the integrity gate covers everything instead of CPU/GPU only (3.4%).

Closes #1

TechEngine #95 diffs against the PR's real base instead of a hard-coded
origin/develop (release PRs reported no changes) or origin/main (develop
PRs over-counted). Pass base_ref/base_sha from the validation relay,
verify-report and /verify. scores.jsonl is a 43 MB generated cache that
TechEngine no longer reads from git; untrack and ignore it.

Refs #1
@TechEngineBot

Copy link
Copy Markdown
Member

🔎 Data verification — Tier 0 (offline existence/trust)

Scored by app.verify; posted by TechEngineBot. Informational only — the structural gate (app.validate) is separate and authoritative for merge.

12/12 categories ? 191,797 records ? diff base develop@c7f4a36

Changed records in this PR

No records scored.

Full-dataset baseline

191797 record(s) assessed.

Laptop, monitor, software and website assess required fields and sources only; domain consistency rules are unavailable and these categories cannot earn green.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 24605
    "Yellow" : 163367
    "Red" : 3825
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
brand 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨 207 122 85 0 58.9%
soc 🟩🟩🟨🟨🟨🟨🟨🟥🟥🟥🟥🟥 2105 282 827 996 13.4%
smartphone 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 93407 18003 75350 54 19.3%
tablet 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 3459 292 3166 1 8.4%
watch 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 442 83 356 3 18.8%
pda 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨 140 83 57 0 59.3%
gpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨 2076 1529 547 0 73.7%
cpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟥 4507 4211 295 1 93.4%
laptop 🟨🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 1995 0 107 1888 0.0%
monitor 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 882 0 0 882 0.0%
software 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 42493 0 42493 0 0.0%
website 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 40084 0 40084 0 0.0%
All 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 191797 24605 163367 3825 12.8%

Hard violations (forced red):

Count Check
1 !boost_ge_base

green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to verified runs in the scheduled verify-network workflow.

@Seungpyo1007
Seungpyo1007 merged commit 7ce54ef into develop Sep 26, 2026
5 checks passed
@Seungpyo1007
Seungpyo1007 deleted the ci/bot-full-coverage-relay branch September 26, 2026 17:27
@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine change review: PASS

12/12 categories ? 191,797 records ? diff base develop@c7f4a36

Check Result
python -m app.validate PASS
Changed dump JSON and manifest/index counts PASS
New hard integrity anomalies vs PR base PASS

Changed data

Category Added Modified Deleted Added verified Added unverified Added Kaggle-sourced
brand 0 0 0 0 0 0
soc 0 0 0 0 0 0
smartphone 0 0 0 0 0 0
tablet 0 0 0 0 0 0
watch 0 0 0 0 0 0
pda 0 0 0 0 0 0
gpu 0 0 0 0 0 0
cpu 0 0 0 0 0 0
laptop 0 0 0 0 0 0
monitor 0 0 0 0 0 0
software 0 0 0 0 0 0
website 0 0 0 0 0 0

Changed record examples

  • No data file changes detected.

Heuristic review

  • Heuristic warnings: none found.

@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine validation stats: PASS

12/12 categories ? 191,797 records ? diff base develop@c7f4a36

Data summary

Category Total Verified Unverified Missing verified Tracked Verified % of tracked
brand 207 122 85 0 207 58.9%
soc 2105 193 1912 0 2105 9.2%
smartphone 93407 17501 75906 0 93407 18.7%
tablet 3459 273 3186 0 3459 7.9%
watch 442 39 403 0 442 8.8%
pda 140 74 66 0 140 52.9%
gpu 2076 1026 1050 0 2076 49.4%
cpu 4507 2869 1638 0 4507 63.7%
laptop 1995 0 1995 0 1995 0.0%
monitor 882 0 882 0 882 0.0%
software 42493 0 42493 0 42493 0.0%
website 40084 0 40084 0 40084 0.0%
all 191797 22097 169700 0 191797 11.5%

Warning

Tracked verified coverage is below 50% for laptop 0.0% (0/1995), monitor 0.0% (0/882), software 0.0% (0/42493), website 0.0% (0/40084), tablet 7.9% (273/3459), watch 8.8% (39/442), soc 9.2% (193/2105), all 11.5% (22097/191797), and 2 more.
Tracked coverage excludes records missing the verified field; see the Missing verified column for those records.
This does not fail validation. Keep imported records verified: false until manual audit, but treat this as follow-up verification work before relying on the affected categories as curated data.

Validation notes

  • The integrity gate blocks only hard anomalies introduced by this PR relative to its merge base.
  • Full advisory outlier listings are suppressed on successful runs because they are dataset-wide and mostly stable between PRs.
  • Failure runs still include a detailed log excerpt for debugging.

Key output:

## app.validate
## integrity_check.py (PR head compared with PR base)
loaded CPU=4507 GPU=2076
Integrity section Flagged lines
structural 18
CPU name/tier consistency (desktop mainstream only) 0
CPU single>multi (cinebench/geekbench — should be multi>=single) 0
CPU era-vs-score outliers 8
CPU cross-source ratio outliers (possible wrong-variant) 131
GPU cross-source ratio outliers + sanity 51

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci CI and workflow changes data Dataset changes enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

TechAPI dataset roadmap and status: all categories (1989-2026)

2 participants