Skip to content

Commit d91e78a

Browse files
timneutkenstimneutkens
authored
Rename turbopack-tests-manifest to turbopack-dev-tests-manifest (vercel#63409)
## What? - Renames the Turbopack tests manifest to reflect that it only holds development tests. - Creates initial plumbing for Turbopack build tests manifest (currently empty) - Added running tests in the Turbopack builds test manifest on PRs - Implements uploading the Turbopack builds manifest to areweturboyet What this doesn't implement: - Updating the Turbopack builds manifest Open questions: - Since the manifest is empty there are no test results, I had to add handling for that in `run-tests.js`: https://github.com/vercel/next.js/pull/63409/files#diff-269567847b110d8ecaaade3ab592df207bba02c7b446db23d72f18ff0e61d335R359 but not sure if that exit case was added for a specific special reason. <!-- Thanks for opening a PR! Your contribution is much appreciated. To make sure your PR is handled as smoothly as possible we request that you follow the checklist sections below. Choose the right checklist for the change(s) that you're making: ## For Contributors ### Improving Documentation - Run `pnpm prettier-fix` to fix formatting issues before opening the PR. - Read the Docs Contribution Guide to ensure your contribution follows the docs guidelines: https://nextjs.org/docs/community/contribution-guide ### Adding or Updating Examples - The "examples guidelines" are followed from our contributing doc https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md - Make sure the linting passes by running `pnpm build && pnpm lint`. See https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md ### Fixing a bug - Related issues linked using `fixes #number` - Tests added. See: https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs - Errors have a helpful link attached, see https://github.com/vercel/next.js/blob/canary/contributing.md ### Adding a feature - Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR. (A discussion must be opened, see https://github.com/vercel/next.js/discussions/new?category=ideas) - Related issues/discussions are linked using `fixes #number` - e2e tests added (https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs) - Documentation added - Telemetry added. In case of a feature if it's used or not. - Errors have a helpful link attached, see https://github.com/vercel/next.js/blob/canary/contributing.md ## For Maintainers - Minimal description (aim for explaining to someone not on the team to understand the PR) - When linking to a Slack thread, you might want to share details of the conclusion - Link both the Linear (Fixes NEXT-xxx) and the GitHub issues - Add review comments if necessary to explain to the reviewer the logic behind a change ### What? ### Why? ### How? Closes NEXT- Fixes # --> Closes NEXT-2837
1 parent 0c62367 commit d91e78a

9 files changed

+136
-30
lines changed

‎.github/CODEOWNERS

+2-2
Original file line numberDiff line numberDiff line change
@@ -41,5 +41,5 @@ Cargo.toml @timneutkens @i
4141
Cargo.lock @timneutkens @ijjk @shuding @huozhi @vercel/turbopack
4242
/.cargo/config.toml @timneutkens @ijjk @shuding @huozhi @vercel/turbopack
4343
/.config/nextest.toml @timneutkens @ijjk @shuding @huozhi @vercel/turbopack
44-
/test/build-turbopack-tests-manifest.js @timneutkens @ijjk @shuding @huozhi @vercel/turbopack
45-
/test/turbopack-tests-manifest.json @timneutkens @ijjk @shuding @huozhi @vercel/turbopack
44+
/test/build-turbopack-dev-tests-manifest.js @timneutkens @ijjk @shuding @huozhi @vercel/turbopack
45+
/test/turbopack-dev-tests-manifest.json @timneutkens @ijjk @shuding @huozhi @vercel/turbopack

‎.github/actions/upload-turboyet-data/src/main.js

+81-20
Original file line numberDiff line numberDiff line change
@@ -3,22 +3,64 @@ const path = require('path')
33

44
const { createClient } = require('@vercel/kv')
55

6-
async function main() {
7-
try {
8-
const file = path.join(process.cwd(), 'test/turbopack-tests-manifest.json')
6+
async function collectResults(manifestFile) {
7+
const file = path.join(process.cwd(), manifestFile)
8+
9+
let passingTests = ''
10+
let failingTests = ''
11+
let passCount = 0
12+
let failCount = 0
13+
14+
const contents = await fs.readFile(file, 'utf-8')
15+
const results = JSON.parse(contents)
16+
17+
const currentDate = new Date()
18+
const isoString = currentDate.toISOString()
19+
const timestamp = isoString.slice(0, 19).replace('T', ' ')
20+
21+
if (results.version === 2) {
22+
for (const [testFileName, result] of Object.entries(results.suites)) {
23+
let suitePassCount = 0
24+
let suiteFailCount = 0
25+
26+
suitePassCount += result.passed.length
27+
suiteFailCount += result.failed.length
28+
29+
if (suitePassCount > 0) {
30+
passingTests += `${testFileName}\n`
31+
}
32+
33+
if (suiteFailCount > 0) {
34+
failingTests += `${testFileName}\n`
35+
}
36+
37+
for (const passed of result.passed) {
38+
const passedName = passed.replaceAll('`', '\\`')
39+
passingTests += `* ${passedName}\n`
40+
}
41+
42+
for (const passed of result.failed) {
43+
const failedName = passed.replaceAll('`', '\\`')
44+
failingTests += `* ${failedName}\n`
45+
}
946

10-
let passingTests = ''
11-
let failingTests = ''
12-
let passCount = 0
13-
let failCount = 0
47+
passCount += suitePassCount
48+
failCount += suiteFailCount
1449

15-
const contents = await fs.readFile(file, 'utf-8')
16-
const results = JSON.parse(contents)
50+
if (suitePassCount > 0) {
51+
passingTests += `\n`
52+
}
1753

18-
const currentDate = new Date()
19-
const isoString = currentDate.toISOString()
20-
const timestamp = isoString.slice(0, 19).replace('T', ' ')
54+
if (suiteFailCount > 0) {
55+
failingTests += `\n`
56+
}
57+
}
2158

59+
const testRun = `${process.env.GITHUB_SHA}\t${timestamp}\t${passCount}/${
60+
passCount + failCount
61+
}`
62+
return { testRun, passingTests, failingTests }
63+
} else {
2264
for (const [testFileName, result] of Object.entries(results)) {
2365
let suitePassCount = 0
2466
let suiteFailCount = 0
@@ -55,26 +97,45 @@ async function main() {
5597
failingTests += `\n`
5698
}
5799
}
100+
const testRun = `${process.env.GITHUB_SHA}\t${timestamp}\t${passCount}/${
101+
passCount + failCount
102+
}`
103+
104+
return { testRun, passingTests, failingTests }
105+
}
106+
}
107+
108+
async function main() {
109+
try {
110+
const developmentResult = await collectResults(
111+
'test/turbopack-dev-tests-manifest.json'
112+
)
113+
114+
const productionResult = await collectResults(
115+
'test/turbopack-build-tests-manifest.json'
116+
)
58117

59118
const kv = createClient({
60119
url: process.env.TURBOYET_KV_REST_API_URL,
61120
token: process.env.TURBOYET_KV_REST_API_TOKEN,
62121
})
63122

64-
const testRun = `${process.env.GITHUB_SHA}\t${timestamp}\t${passCount}/${
65-
passCount + failCount
66-
}`
123+
console.log('TEST RESULT DEVELOPMENT')
124+
console.log(developmentResult.testRun)
67125

68-
console.log('TEST RESULT')
69-
console.log(testRun)
126+
console.log('TEST RESULT PRODUCTION')
127+
console.log(productionResult.testRun)
70128

71-
await kv.rpush('test-runs', testRun)
129+
await kv.rpush('test-runs', developmentResult.testRun)
130+
await kv.rpush('test-runs-production', productionResult.testRun)
72131
console.log('SUCCESSFULLY SAVED RUNS')
73132

74-
await kv.set('passing-tests', passingTests)
133+
await kv.set('passing-tests', developmentResult.passingTests)
134+
await kv.set('passing-tests-production', productionResult.passingTests)
75135
console.log('SUCCESSFULLY SAVED PASSING')
76136

77-
await kv.set('failing-tests', failingTests)
137+
await kv.set('failing-tests', developmentResult.failingTests)
138+
await kv.set('failing-tests-production', productionResult.failingTests)
78139
console.log('SUCCESSFULLY SAVED FAILING')
79140
} catch (error) {
80141
console.log(error)

‎.github/workflows/build_and_test.yml

+38-4
Original file line numberDiff line numberDiff line change
@@ -163,12 +163,12 @@ jobs:
163163
group: [1/5, 2/5, 3/5, 4/5, 5/5]
164164
uses: ./.github/workflows/build_reusable.yml
165165
with:
166-
afterBuild: RUST_BACKTRACE=0 NEXT_EXTERNAL_TESTS_FILTERS="$(pwd)/test/turbopack-tests-manifest.json" TURBOPACK=1 NEXT_E2E_TEST_TIMEOUT=240000 NEXT_TEST_MODE=dev node run-tests.js --test-pattern '^(test\/(development|e2e))/.*\.test\.(js|jsx|ts|tsx)$' --timings -g ${{ matrix.group }} -c ${TEST_CONCURRENCY}
166+
afterBuild: RUST_BACKTRACE=0 NEXT_EXTERNAL_TESTS_FILTERS="$(pwd)/test/turbopack-dev-tests-manifest.json" TURBOPACK=1 NEXT_E2E_TEST_TIMEOUT=240000 NEXT_TEST_MODE=dev node run-tests.js --test-pattern '^(test\/(development|e2e))/.*\.test\.(js|jsx|ts|tsx)$' --timings -g ${{ matrix.group }} -c ${TEST_CONCURRENCY}
167167
stepName: 'test-turbopack-dev-${{ matrix.group }}'
168168
secrets: inherit
169169

170170
test-turbopack-integration:
171-
name: test turbopack integration
171+
name: test turbopack development integration
172172
needs: ['changes', 'build-next']
173173
if: ${{ needs.changes.outputs.docs-only == 'false' }}
174174

@@ -179,10 +179,42 @@ jobs:
179179
uses: ./.github/workflows/build_reusable.yml
180180
with:
181181
nodeVersion: 18.17.0
182-
afterBuild: RUST_BACKTRACE=0 NEXT_EXTERNAL_TESTS_FILTERS="$(pwd)/test/turbopack-tests-manifest.json" TURBOPACK=1 node run-tests.js --timings -g ${{ matrix.group }} -c ${TEST_CONCURRENCY} --type integration
182+
afterBuild: RUST_BACKTRACE=0 NEXT_EXTERNAL_TESTS_FILTERS="$(pwd)/test/turbopack-dev-tests-manifest.json" TURBOPACK=1 node run-tests.js --timings -g ${{ matrix.group }} -c ${TEST_CONCURRENCY} --type integration
183183
stepName: 'test-turbopack-integration-${{ matrix.group }}'
184184
secrets: inherit
185185

186+
test-turbopack-production:
187+
name: test turbopack production
188+
needs: ['changes', 'build-next']
189+
if: ${{ needs.changes.outputs.docs-only == 'false' }}
190+
191+
strategy:
192+
fail-fast: false
193+
matrix:
194+
group: [1/5, 2/5, 3/5, 4/5, 5/5]
195+
uses: ./.github/workflows/build_reusable.yml
196+
with:
197+
nodeVersion: 18.17.0
198+
afterBuild: RUST_BACKTRACE=0 NEXT_EXTERNAL_TESTS_FILTERS="$(pwd)/test/turbopack-build-tests-manifest.json" TURBOPACK=1 TURBOPACK_BUILD=1 node run-tests.js --timings -g ${{ matrix.group }} -c ${TEST_CONCURRENCY} --type start
199+
stepName: 'test-turbopack-production-${{ matrix.group }}'
200+
secrets: inherit
201+
202+
test-turbopack-production-integration:
203+
name: test turbopack production integration
204+
needs: ['changes', 'build-next']
205+
if: ${{ needs.changes.outputs.docs-only == 'false' }}
206+
207+
strategy:
208+
fail-fast: false
209+
matrix:
210+
group: [1/5, 2/5, 3/5, 4/5, 5/5]
211+
uses: ./.github/workflows/build_reusable.yml
212+
with:
213+
nodeVersion: 18.17.0
214+
afterBuild: RUST_BACKTRACE=0 NEXT_EXTERNAL_TESTS_FILTERS="$(pwd)/test/turbopack-build-tests-manifest.json" TURBOPACK=1 TURBOPACK_BUILD=1 node run-tests.js --timings -g ${{ matrix.group }} -c ${TEST_CONCURRENCY} --type integration
215+
stepName: 'test-turbopack-production-integration-${{ matrix.group }}'
216+
secrets: inherit
217+
186218
test-next-swc-wasm:
187219
name: test next-swc wasm
188220
needs: ['changes', 'build-next']
@@ -339,7 +371,7 @@ jobs:
339371
stepName: 'test-ppr-prod-${{ matrix.group }}'
340372
secrets: inherit
341373

342-
report-test-results:
374+
report-test-results-to-datadog:
343375
needs:
344376
[
345377
'changes',
@@ -352,6 +384,8 @@ jobs:
352384
'test-ppr-integration',
353385
'test-turbopack-dev',
354386
'test-turbopack-integration',
387+
'test-turbopack-production',
388+
'test-turbopack-production-integration',
355389
]
356390
if: ${{ always() && needs.changes.outputs.docs-only == 'false' && !github.event.pull_request.head.repo.fork }}
357391

‎.github/workflows/turbopack-update-tests-manifest.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,6 @@ jobs:
3838
env:
3939
GITHUB_TOKEN: ${{ secrets.GH_TOKEN_PULL_REQUESTS }}
4040
BRANCH_NAME: turbopack-manifest
41-
SCRIPT: test/build-turbopack-tests-manifest.js
41+
SCRIPT: test/build-turbopack-dev-tests-manifest.js
4242
PR_TITLE: Update Turbopack test manifest
4343
PR_BODY: This auto-generated PR updates the integration test manifest used when testing Turbopack.

‎.github/workflows/turbopack-upload-tests-manifest.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ jobs:
2929

3030
# Always run build manifest script to get the latest value
3131
- run: |
32-
node ./test/build-turbopack-tests-manifest.js
32+
node ./test/build-turbopack-dev-tests-manifest.js
3333
env:
3434
GITHUB_TOKEN: ${{ secrets.GH_TOKEN_PULL_REQUESTS }}
3535

‎run-tests.js

+4-1
Original file line numberDiff line numberDiff line change
@@ -356,9 +356,12 @@ async function main() {
356356
}
357357
}
358358

359+
if (!tests) {
360+
tests = []
361+
}
362+
359363
if (tests.length === 0) {
360364
console.log('No tests found for', options.type, 'exiting..')
361-
return cleanUpAndExit(1)
362365
}
363366

364367
console.log(`${GROUP}Running tests:

‎test/build-turbopack-tests-manifest.js renamed to ‎test/build-turbopack-dev-tests-manifest.js

+1-1
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ async function format(text) {
1313

1414
const override = process.argv.includes('--override')
1515

16-
const PASSING_JSON_PATH = `${__dirname}/turbopack-tests-manifest.json`
16+
const PASSING_JSON_PATH = `${__dirname}/turbopack-dev-tests-manifest.json`
1717
const WORKING_PATH = '/root/actions-runner/_work/next.js/next.js/'
1818

1919
const INITIALIZING_TEST_CASES = [

‎test/turbopack-build-tests-manifest.json

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"version": 2,
3+
"suites": {},
4+
"rules": {
5+
"include": [],
6+
"exclude": []
7+
}
8+
}

‎test/turbopack-tests-manifest.json renamed to ‎test/turbopack-dev-tests-manifest.json

File renamed without changes.

0 commit comments

Comments
 (0)