Skip to content

data(smartphone): import PhoneDB raw variants (batch 2-3) + normalize names - #28

Merged
Seungpyo1007 merged 20 commits into
mainfrom
data/import-staging
Jun 19, 2026
Merged

Seungpyo1007 merged 20 commits into
mainfrom
data/import-staging

Conversation

@Seungpyo1007

@Seungpyo1007 Seungpyo1007 commented Jun 19, 2026 •

Copy link
Copy Markdown
Member

Summary

Continues the PhoneDB variant-level import (batch 1 landed in #27). Adds ~5,450 new variant-level smartphones + 86 SoC seeds, normalizes 7,168 device names (fixes the #27 double-space heuristic warnings), and refreshes the public dump. All raw seed (verified: false) with per-record source_urls to phonedb.net.

Changes vs main

  • +5,450 smartphones (PhoneDB device ids ~25,227→16,228; years skew 2020–2023).
  • +86 SoC seeds (auto-detected chipsets, verified: false).
  • 2,596 existing (batch-1) smartphone names normalized (whitespace collapse).
  • site/public/v1 dump refreshed as the final commit → now 24,655 smartphones / 1,773 SoC (was stale at 19,205 / 1,560).
  • Distinct per-batch commit messages (brand/year); Refs #1 in each commit body.

Validation (local)

  • python -m app.validate → ✅ passed
  • python TechEngine/integrity_check.py data --strict → ✅ no hard anomalies
  • Dump generated from an isolated git-archive snapshot + fresh DB (so a concurrent session's in-progress writes to the shared data/ dir could not contaminate it).

Closes #1

@Seungpyo1007

Copy link
Copy Markdown
Member Author

Validation review — PASS ✅

Commands run (local):

  • python -m app.validate → ✅ passed
  • python TechEngine/integrity_check.py data --strict → ✅ no hard anomalies

Data changes vs main:

category added modified
smartphone 5,450 2,596 (name normalize)
soc 86 —

Example record — data/smartphone/samsung/2026/samsung-sm-s9480-galaxy-s26-ultra-5g-...json:

{ "slug": "samsung-sm-s9480-galaxy-s26-ultra-5g-dual-sim-td-lte-cn-hk-tw-512gb-samsung-miracle-3",
  "name": "Samsung SM-S9480 Galaxy S26 Ultra 5G Dual SIM TD-LTE CN HK TW 512GB (Samsung Miracle 3)",
  "brand": "samsung", "soc": "qualcomm-snapdragon-8-elite-gen-5-sm8850-1-ad-for-galaxy",
  "release_date": "2026-03-11", "ram_gb": 12, "battery_mah": 5000, "os": "Android",
  "verified": false, "source_urls": ["https://phonedb.net/index.php?m=device&id=25688"] }

Heuristic warnings: the double-space names flagged on #27 are fixed (7,168 normalized). Remaining integrity-gate ratio outliers are pre-existing GPU/CPU benchmark notes, unrelated to this PR.

Site build: N/A (no site/ changes).

@Seungpyo1007

Copy link
Copy Markdown
Member Author

Dataset stats (post-merge projection)

category total verified unverified verified %
smartphone 24,655 184 24,471 0.7%
soc 1,773 58 1,715 3.3%
brand 189 — — —
cpu 3,977 — — —
gpu 2,030 — — —

Warning

Verified coverage is far below 50% (smartphone 0.7%, soc 3.3%) — expected for bulk raw seed imports. Verification is deferred to TechEngine / manual audit per the import workflow.

@Seungpyo1007
Seungpyo1007 merged commit a427fd6 into main Jun 19, 2026
4 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in TechAPI-Project Jun 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data Dataset changes enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

TechAPI dataset roadmap and status: all categories (1989-2026)

1 participant