Skip to content

chore(benchmarks): add 5000-route production-build stress scenario#2388

Closed
hyf0 wants to merge 1 commit into
cloudflare:mainfrom
hyf0:bench/large-scale-fixture
Closed

chore(benchmarks): add 5000-route production-build stress scenario#2388
hyf0 wants to merge 1 commit into
cloudflare:mainfrom
hyf0:bench/large-scale-fixture

Conversation

@hyf0

@hyf0 hyf0 commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

No dependencies (standalone benchmark infra). ⚠️ Needs a small perf.yml wiring change (see the bottom of this description) before the scenario runs in CI.

What

  • A vinext-only perf scenario "Production build time (5000 routes)" that builds a generated 5000-route app, via a separate benchmarks/vinext-large fixture.
  • generate-app.mjs gains --extra <n> / --target <name> (and the VINEXT_BENCH_EXTRA_ROUTES env) to scale and target the generated app.

Why

The default benchmark app is ~33 routes. At that size, super-linear build-time effects — e.g. route-graph directory scanning that re-reads shared ancestor directories once per descendant route — are smaller than run-to-run noise, so regressions (or fixes) in them are invisible to the perf bot. 5000 is the smallest scale where such effects clearly clear the ±1.5% band while keeping CI cost moderate (a scaling sweep showed the relevant delta going ~noise at 1k → ~−32% at 5k → ~−52% at 10k, with signal flattening past ~5–8k).

Notes

  • vinext-only and a separate fixture, so the headline vinext-vs-Next comparison and the existing scenarios are unchanged.
  • Adds benchmarks/vinext-large to the workspace (lockfile updated).
  • The generate step is intentionally always-run (no implementationId), so the 5000-route app is generated even on PR runs that skip Next.js via --implementation=vinext.

⚠️ CI wiring needed (for a maintainer)

This scenario builds a new project (benchmarks/vinext-large) under the isolated vinext-perf-head user, which perf.yml doesn't yet know about. I've left the perf.yml change to whoever owns the perf workflow, since it touches the isolated-user ACL model and I couldn't exercise that path locally. To run green it needs the same treatment benchmarks/vinext already gets:

  • grant_write_paths — add benchmarks/vinext-large to project_roots, and benchmarks/vinext-large/{dist,.vite,node_modules/.vite,node_modules/.vite-temp} to paths. Without the write-ACL, vp build (run as vinext-perf-head) hits EACCES writing dist.
  • "Validate performance workspace paths" — add benchmarks/vinext-large (+ its app/dist) to the directory / symlink checks, for parity.

No change is needed to the trusted-manifest/harness-copy steps (they only gate vinext/nextjs): the build reads the fixture from the target checkout and its scripts from the trusted harness.

@hyf0 hyf0 force-pushed the bench/large-scale-fixture branch from 214427f to 989ed50 Compare June 27, 2026 18:58
@hyf0 hyf0 marked this pull request as ready for review June 27, 2026 19:23
@hyf0 hyf0 force-pushed the bench/large-scale-fixture branch from 989ed50 to 8fe3d54 Compare June 28, 2026 04:23
@hyf0 hyf0 changed the title chore(benchmarks): add opt-in large-scale fixture via VINEXT_BENCH_EXTRA_ROUTES chore(benchmarks): add 5000-route production-build stress scenario Jun 28, 2026
Adds an app-scaling knob to generate-app.mjs (--extra/--target + VINEXT_BENCH_EXTRA_ROUTES) and a vinext-only 'Production build time (5000 routes)' perf scenario that builds a generated 5000-route app via a separate benchmarks/vinext-large fixture. Purpose: catch super-linear build-time regressions (e.g. route-graph directory scanning) that are invisible at the default ~33-route app. Separate fixture keeps the headline vinext-vs-Next comparison unchanged.
@pkg-pr-new

pkg-pr-new Bot commented Jun 28, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@vinext/cloudflare@2388
npm i https://pkg.pr.new/vinext@2388

commit: 3cb128e

@hyf0 hyf0 closed this Jun 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant