Skip to content

test: stabilize codspeed memory benchmarks - #7988

Open
Sheraff wants to merge 1 commit into
mainfrom
test-codspeed-memory-stabilization
Open

test: stabilize codspeed memory benchmarks#7988
Sheraff wants to merge 1 commit into
mainfrom
test-codspeed-memory-stabilization

Conversation

@Sheraff

@Sheraff Sheraff commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • Benchmark Improvements

    • Memory benchmarks now run in isolated, fresh processes for more consistent results.
    • Client and server scenarios across React, Solid, and Vue use standardized benchmark execution.
    • Churn and request benchmarks provide improved cleanup, state isolation, and memory-floor validation.
    • Failures and workload errors are reported more reliably.
  • Documentation

    • Added guidance for deterministic execution, process isolation, cleanup, and benchmark interpretation.
  • Tests

    • Added coverage for process restarts, workload failures, cleanup, and invalid benchmark selections.

@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The PR adds isolated child-process execution for memory benchmarks. It introduces shared client and server registration helpers, migrates existing scenarios to those helpers, adds IPC and process-isolation tests, updates benchmark configuration, and documents the execution model.

Changes

Memory benchmark isolation

Layer / File(s) Summary
Process runtime and IPC contracts
benchmarks/memory/shared/*
Adds IsolatedMemoryProcess and its child runner. The runtime validates workloads, applies deterministic V8 settings, executes indexed workloads through IPC, settles measurements, and handles cleanup and failures.
Client benchmark registration and migration
benchmarks/memory/client/isolated-benchmark.ts, benchmarks/memory/client/scenarios/*, benchmarks/memory/client/package.json, benchmarks/memory/client/tsconfig.json
Adds shared client registration and migrates client scenarios to setup URLs and isolated execution. Vitest uses the Node environment without the previous setup file.
Server benchmark registration and validation
benchmarks/memory/server/isolated-benchmark.ts, benchmarks/memory/server/scenarios/*, benchmarks/memory/server/isolated-process.test.ts, benchmarks/memory/server/test-fixtures/*, benchmarks/memory/server/package.json, benchmarks/memory/server/tsconfig.json
Adds shared server registration, migrates server scenarios, and adds process-isolation fixtures, integration tests, package targets, and TypeScript coverage.
Execution model documentation
benchmarks/memory/README.md, benchmarks/memory/client/scenarios/*/shared.ts, benchmarks/memory/server/scenarios/*/shared.ts
Documents fresh child processes, deterministic setup, cleanup collections, measured loops, heap-floor checks, and identifier scope.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Vitest
  participant BenchmarkRegistrar
  participant IsolatedMemoryProcess
  participant isolated-process-child
  participant Workload
  Vitest->>BenchmarkRegistrar: register benchmark
  BenchmarkRegistrar->>IsolatedMemoryProcess: start setup process
  IsolatedMemoryProcess->>isolated-process-child: send run request
  isolated-process-child->>Workload: execute workload
  Workload-->>isolated-process-child: return or throw
  isolated-process-child-->>IsolatedMemoryProcess: send completion or error
  IsolatedMemoryProcess-->>BenchmarkRegistrar: resolve or reject benchmark
  BenchmarkRegistrar-->>Vitest: finish benchmark lifecycle
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: stabilizing CodSpeed memory benchmarks through isolated-process benchmark infrastructure.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test-codspeed-memory-stabilization

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (2)
benchmarks/memory/shared/isolated-process.ts (2)

192-196: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Await child exit in the failure path.

child.kill() only requests termination. start() returns before the child exits, so a failed start can leave a live process that competes for memory with the next run. #waitForExit already handles the exit transition safely.

♻️ Proposed change
     } catch (error) {
+      const exit = this.#waitForExit(child)
       child.kill()
       this.#child = undefined
+      await exit.catch(() => {})
       throw error
     }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@benchmarks/memory/shared/isolated-process.ts` around lines 192 - 196, Update
the catch block in start() to await `#waitForExit` after requesting child
termination, ensuring the failed child has fully exited before rethrowing the
original error. Preserve the existing child reference cleanup and error
propagation.

295-347: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Consider a timeout for IPC waits.

#waitForMessage settles only on a message, an error, or child exit. If a workload stalls, run() and stop() never settle, and the failure surfaces later as an opaque runner timeout. A bounded wait that kills the child and rejects with the workload name would make the failure diagnosable.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@benchmarks/memory/shared/isolated-process.ts` around lines 295 - 347, Add a
bounded timeout to `#waitForMessage` so stalled IPC waits terminate
deterministically. On timeout, clean up listeners, kill the child process, and
reject with an error that includes the workload name; also clear the timer
whenever the wait settles through a message, error, or exit.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@benchmarks/memory/client/isolated-benchmark.ts`:
- Around line 31-38: Update the memory benchmark suite around isolatedProcess to
remove or neutralize the describe-level beforeEach and afterEach hooks, leaving
lifecycle management exclusively to the setup and teardown callbacks in bench
options. Preserve the existing isolatedProcess.start() and
isolatedProcess.stop() calls in those Tinybench callbacks.

In `@benchmarks/memory/server/isolated-process.test.ts`:
- Around line 24-28: Update the afterEach cleanup around runner.stop() so
environment-variable deletion and temporary-directory removal always execute in
a finally block, even when stop() rejects. Also reset the runner reference
during cleanup, using the existing runner and tempDirectory symbols.

In `@benchmarks/memory/shared/isolated-process-child.ts`:
- Around line 222-250: Update the commandQueue chain around the message handler
so failures from either the main operation or the catch-block send are contained
at every link. Reuse the handler for both fulfillment and rejection, e.g. attach
it as both callbacks to commandQueue.then, and ensure the error-reporting send
cannot leave the chain rejected so later run and stop messages continue
processing.

---

Nitpick comments:
In `@benchmarks/memory/shared/isolated-process.ts`:
- Around line 192-196: Update the catch block in start() to await `#waitForExit`
after requesting child termination, ensuring the failed child has fully exited
before rethrowing the original error. Preserve the existing child reference
cleanup and error propagation.
- Around line 295-347: Add a bounded timeout to `#waitForMessage` so stalled IPC
waits terminate deterministically. On timeout, clean up listeners, kill the
child process, and reject with an error that includes the workload name; also
clear the timer whenever the wait settles through a message, error, or exit.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: fc9d0b9f-7700-4377-a465-1174a1582619

📥 Commits

Reviewing files that changed from the base of the PR and between abf9b81 and 300581b.

📒 Files selected for processing (64)
  • benchmarks/memory/README.md
  • benchmarks/memory/client/isolated-benchmark.ts
  • benchmarks/memory/client/package.json
  • benchmarks/memory/client/scenarios/interrupted-navigations/react/memory.bench.ts
  • benchmarks/memory/client/scenarios/interrupted-navigations/react/vite.config.ts
  • benchmarks/memory/client/scenarios/interrupted-navigations/solid/memory.bench.ts
  • benchmarks/memory/client/scenarios/interrupted-navigations/solid/vite.config.ts
  • benchmarks/memory/client/scenarios/interrupted-navigations/vue/memory.bench.ts
  • benchmarks/memory/client/scenarios/interrupted-navigations/vue/vite.config.ts
  • benchmarks/memory/client/scenarios/loader-data-retention/react/memory.bench.ts
  • benchmarks/memory/client/scenarios/loader-data-retention/react/vite.config.ts
  • benchmarks/memory/client/scenarios/loader-data-retention/solid/memory.bench.ts
  • benchmarks/memory/client/scenarios/loader-data-retention/solid/vite.config.ts
  • benchmarks/memory/client/scenarios/loader-data-retention/vue/memory.bench.ts
  • benchmarks/memory/client/scenarios/loader-data-retention/vue/vite.config.ts
  • benchmarks/memory/client/scenarios/mount-unmount/react/memory.bench.ts
  • benchmarks/memory/client/scenarios/mount-unmount/react/vite.config.ts
  • benchmarks/memory/client/scenarios/mount-unmount/solid/memory.bench.ts
  • benchmarks/memory/client/scenarios/mount-unmount/solid/vite.config.ts
  • benchmarks/memory/client/scenarios/mount-unmount/vue/memory.bench.ts
  • benchmarks/memory/client/scenarios/mount-unmount/vue/vite.config.ts
  • benchmarks/memory/client/scenarios/navigation-churn/react/memory.bench.ts
  • benchmarks/memory/client/scenarios/navigation-churn/react/vite.config.ts
  • benchmarks/memory/client/scenarios/navigation-churn/solid/memory.bench.ts
  • benchmarks/memory/client/scenarios/navigation-churn/solid/vite.config.ts
  • benchmarks/memory/client/scenarios/navigation-churn/vue/memory.bench.ts
  • benchmarks/memory/client/scenarios/navigation-churn/vue/vite.config.ts
  • benchmarks/memory/client/scenarios/preload-churn/react/memory.bench.ts
  • benchmarks/memory/client/scenarios/preload-churn/react/vite.config.ts
  • benchmarks/memory/client/scenarios/preload-churn/shared.ts
  • benchmarks/memory/client/scenarios/preload-churn/solid/memory.bench.ts
  • benchmarks/memory/client/scenarios/preload-churn/solid/vite.config.ts
  • benchmarks/memory/client/scenarios/preload-churn/vue/memory.bench.ts
  • benchmarks/memory/client/scenarios/preload-churn/vue/vite.config.ts
  • benchmarks/memory/client/scenarios/unique-location-churn/react/memory.bench.ts
  • benchmarks/memory/client/scenarios/unique-location-churn/react/vite.config.ts
  • benchmarks/memory/client/scenarios/unique-location-churn/shared.ts
  • benchmarks/memory/client/scenarios/unique-location-churn/solid/memory.bench.ts
  • benchmarks/memory/client/scenarios/unique-location-churn/solid/vite.config.ts
  • benchmarks/memory/client/scenarios/unique-location-churn/vue/memory.bench.ts
  • benchmarks/memory/client/scenarios/unique-location-churn/vue/vite.config.ts
  • benchmarks/memory/client/tsconfig.json
  • benchmarks/memory/server/isolated-benchmark.ts
  • benchmarks/memory/server/isolated-process.test.ts
  • benchmarks/memory/server/package.json
  • benchmarks/memory/server/scenarios/aborted-requests/react/memory.bench.ts
  • benchmarks/memory/server/scenarios/aborted-requests/solid/memory.bench.ts
  • benchmarks/memory/server/scenarios/aborted-requests/vue/memory.bench.ts
  • benchmarks/memory/server/scenarios/error-paths/react/memory.bench.ts
  • benchmarks/memory/server/scenarios/error-paths/shared.ts
  • benchmarks/memory/server/scenarios/error-paths/solid/memory.bench.ts
  • benchmarks/memory/server/scenarios/error-paths/vue/memory.bench.ts
  • benchmarks/memory/server/scenarios/request-churn/react/memory.bench.ts
  • benchmarks/memory/server/scenarios/request-churn/shared.ts
  • benchmarks/memory/server/scenarios/request-churn/solid/memory.bench.ts
  • benchmarks/memory/server/scenarios/request-churn/vue/memory.bench.ts
  • benchmarks/memory/server/scenarios/server-fn-churn/react/memory.bench.ts
  • benchmarks/memory/server/scenarios/server-fn-churn/solid/memory.bench.ts
  • benchmarks/memory/server/scenarios/server-fn-churn/vue/memory.bench.ts
  • benchmarks/memory/server/test-fixtures/isolated-process-setup.ts
  • benchmarks/memory/server/tsconfig.json
  • benchmarks/memory/shared/isolated-process-child.ts
  • benchmarks/memory/shared/isolated-process.ts
  • benchmarks/memory/shared/tsconfig.json

Comment on lines +31 to +38
describe('memory', () => {
beforeEach(() => isolatedProcess.start())
afterEach(() => isolatedProcess.stop())

bench(options.name, run, {
...memoryBenchOptions,
setup: () => isolatedProcess.start(),
teardown: () => isolatedProcess.stop(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

For Vitest 4.1.4 benchmark files, verify whether suite-level beforeEach/afterEach hooks run around bench() tasks and whether Tinybench setup/teardown run for each benchmark cycle.

💡 Result:

In Vitest 4.1.4, the behavior of suite-level hooks and Tinybench setup/teardown in benchmark files is as follows: Suite-Level Hooks (beforeEach/afterEach) Historically, Vitest's suite-level beforeEach and afterEach hooks did not run for bench tasks [1]. While there have been community efforts and pull requests (such as PR #7541) aimed at enabling this functionality, these hooks are generally not standardly supported or guaranteed to execute around benchmark tasks in the same way they do for test blocks [1][2][3]. Benchmark execution in Vitest has historically followed a distinct lifecycle from standard tests [1][3]. Tinybench Setup/Teardown Tinybench, which powers Vitest's benchmarking, explicitly provides setup and teardown options [4][5][6]. These are configured as part of the third argument (options object) passed to the bench function [4][5]. According to Tinybench documentation, these setup and teardown functions are designed to run for each benchmark cycle (task execution) [6][7][8]. They are intended for operations that need to occur immediately before and after the specific code being measured, allowing users to handle stateful setups that should not be included in the timed performance result [3][6]. In summary, you should rely on the explicit setup and teardown options provided in the bench function's configuration object to manage cycle-specific logic, rather than relying on suite-level beforeEach/afterEach hooks [3][4][6]. Top Results: [1][2][3][4][6]

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "Locate files:"
fd -a 'isolated-benchmark\.ts$|isolated.*memory.*\.ts$|memory.*\.ts$' . | sed 's#^\./##' | head -100

echo
echo "Target file outline:"
ast-grep outline benchmarks/memory/client/isolated-benchmark.ts --view compact || true

echo
echo "Target file lines:"
cat -n benchmarks/memory/client/isolated-benchmark.ts | sed -n '1,120p'

echo
echo "Search isolated process definitions/usages:"
rg -n "class IsolatedMemoryProcess|IsolatedMemoryProcess|start\(|stop\(|isolatedProcess|bench\\(" -S benchmarks . --glob '*.ts' --glob '!node_modules' | head -200

Repository: TanStack/router

Length of output: 25556


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "Vitest version declarations:"
for f in package.json pnpm-lock.yaml yarn.lock package-lock.json; do
  if [ -f "$f" ]; then
    echo "--- $f ---"
    if [ "$f" = "package.json" ]; then
      node -e "const p=require('./package.json'); console.log({devDependencies:p.devDependencies?.vitest, dependencies:p.dependencies?.vitest, scripts:p.scripts});" 2>/dev/null || sed -n '/vitest/p' "$f"
    else
      rg -n "vitest(@|:)|version:" "$f" | head -80
    fi
  fi
done

echo
echo "Shared process start/stop implementation:"
cat -n benchmarks/memory/shared/isolated-process.ts | sed -n '140,235p'

echo
echo "Isolated process tests around duplicate start:"
cat -n benchmarks/memory/server/isolated-process.test.ts | sed -n '1,130p'

echo
echo "Memory bench options:"
cat -n benchmarks/memory/client/bench-utils.ts | sed -n '1,120p'

echo
echo "Server isolated benchmark counterpart:"
cat -n benchmarks/memory/server/isolated-benchmark.ts | sed -n '1,70p'

Repository: TanStack/router

Length of output: 10199


Use Tinybench setup/teardown for the isolated lifecycle.

In Vitest 4 benchmark mode, keep the setup and teardown in the bench() options, but remove the suite beforeEach/afterEach hooks or make them no-ops. The suite hooks can start the process before bench() setup/teardown runs, while IsolatedMemoryProcess.start() rejects a second start because a child process already exists.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@benchmarks/memory/client/isolated-benchmark.ts` around lines 31 - 38, Update
the memory benchmark suite around isolatedProcess to remove or neutralize the
describe-level beforeEach and afterEach hooks, leaving lifecycle management
exclusively to the setup and teardown callbacks in bench options. Preserve the
existing isolatedProcess.start() and isolatedProcess.stop() calls in those
Tinybench callbacks.

Comment on lines +24 to +28
afterEach(async () => {
await runner?.stop()
delete process.env.TSR_MEMORY_ISOLATION_TEST_LOG
await rm(tempDirectory, { recursive: true })
})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Make cleanup independent of stop() success.

If runner.stop() rejects, afterEach skips the env-var deletion and the temp-directory removal. The stale TSR_MEMORY_ISOLATION_TEST_LOG value then leaks into later tests, and temp directories accumulate. Run the cleanup in a finally block, and reset runner.

🧹 Proposed fix
   afterEach(async () => {
-    await runner?.stop()
-    delete process.env.TSR_MEMORY_ISOLATION_TEST_LOG
-    await rm(tempDirectory, { recursive: true })
+    try {
+      await runner?.stop()
+    } finally {
+      runner = undefined
+      delete process.env.TSR_MEMORY_ISOLATION_TEST_LOG
+      await rm(tempDirectory, { recursive: true, force: true })
+    }
   })
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
afterEach(async () => {
await runner?.stop()
delete process.env.TSR_MEMORY_ISOLATION_TEST_LOG
await rm(tempDirectory, { recursive: true })
})
afterEach(async () => {
try {
await runner?.stop()
} finally {
runner = undefined
delete process.env.TSR_MEMORY_ISOLATION_TEST_LOG
await rm(tempDirectory, { recursive: true, force: true })
}
})
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@benchmarks/memory/server/isolated-process.test.ts` around lines 24 - 28,
Update the afterEach cleanup around runner.stop() so environment-variable
deletion and temporary-directory removal always execute in a finally block, even
when stop() rejects. Also reset the runner reference during cleanup, using the
existing runner and tempDirectory symbols.

Comment on lines +222 to +250
commandQueue = commandQueue.then(async () => {
try {
if (message.type === 'run') {
const workload = loaded.workloads[message.workloadIndex]

if (!workload) {
throw new Error(
`Invalid isolated memory workload index ${message.workloadIndex}`,
)
}

await workload.run()
await settle(completionSettleTurns)
await send({ type: 'complete', requestId: message.requestId })
return
}

stopping = true
await loaded.cleanup()
await send({ type: 'stopped', requestId: message.requestId })
process.exit(0)
} catch (error) {
await send({
type: 'error',
requestId: message.requestId,
error: serializeError(error),
})
}
})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

A failed send breaks the command queue permanently.

The catch block awaits send(...). If that send rejects, for example when the IPC channel is closing, the callback rejects and commandQueue becomes a rejected promise. The next commandQueue.then(cb) has no rejection handler, so cb never executes. The child then ignores all later 'run' and 'stop' messages while staying alive, and Node reports an unhandled rejection. The parent waits until exit or a runner timeout.

Terminate the rejection at each link.

🐛 Proposed fix
-    commandQueue = commandQueue.then(async () => {
+    commandQueue = commandQueue.then(async () => {
       try {
         if (message.type === 'run') {
           const workload = loaded.workloads[message.workloadIndex]
 
           if (!workload) {
             throw new Error(
               `Invalid isolated memory workload index ${message.workloadIndex}`,
             )
           }
 
           await workload.run()
           await settle(completionSettleTurns)
           await send({ type: 'complete', requestId: message.requestId })
           return
         }
 
         stopping = true
         await loaded.cleanup()
         await send({ type: 'stopped', requestId: message.requestId })
         process.exit(0)
       } catch (error) {
         await send({
           type: 'error',
           requestId: message.requestId,
           error: serializeError(error),
-        })
+        }).catch(() => {})
       }
-    })
+    })

Additionally, guard the chain itself so a rejection cannot block later commands:

commandQueue = commandQueue.then(handler, handler)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
commandQueue = commandQueue.then(async () => {
try {
if (message.type === 'run') {
const workload = loaded.workloads[message.workloadIndex]
if (!workload) {
throw new Error(
`Invalid isolated memory workload index ${message.workloadIndex}`,
)
}
await workload.run()
await settle(completionSettleTurns)
await send({ type: 'complete', requestId: message.requestId })
return
}
stopping = true
await loaded.cleanup()
await send({ type: 'stopped', requestId: message.requestId })
process.exit(0)
} catch (error) {
await send({
type: 'error',
requestId: message.requestId,
error: serializeError(error),
})
}
})
commandQueue = commandQueue.then(async () => {
try {
if (message.type === 'run') {
const workload = loaded.workloads[message.workloadIndex]
if (!workload) {
throw new Error(
`Invalid isolated memory workload index ${message.workloadIndex}`,
)
}
await workload.run()
await settle(completionSettleTurns)
await send({ type: 'complete', requestId: message.requestId })
return
}
stopping = true
await loaded.cleanup()
await send({ type: 'stopped', requestId: message.requestId })
process.exit(0)
} catch (error) {
await send({
type: 'error',
requestId: message.requestId,
error: serializeError(error),
}).catch(() => {})
}
})
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@benchmarks/memory/shared/isolated-process-child.ts` around lines 222 - 250,
Update the commandQueue chain around the message handler so failures from either
the main operation or the catch-block send are contained at every link. Reuse
the handler for both fulfillment and rejection, e.g. attach it as both callbacks
to commandQueue.then, and ensure the error-reporting send cannot leave the chain
rejected so later run and stop messages continue processing.

@codspeed-hq

codspeed-hq Bot commented Aug 6, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 49.3%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 6 improved benchmarks
❌ 33 regressed benchmarks
✅ 141 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Memory mem client interrupted-navigations (react) 250.1 KB 2,255 KB -88.91%
Memory mem client loader-data-retention (react) 155.5 KB 1,164.2 KB -86.64%
Memory mem client loader-data-retention (solid) 153.4 KB 1,079.8 KB -85.8%
Memory mem client unique-location-churn (solid) 342.1 KB 2,373.7 KB -85.59%
Memory mem client interrupted-navigations (solid) 341.1 KB 2,263.8 KB -84.93%
Memory mem client preload-churn (solid) 311.8 KB 1,548.1 KB -79.86%
Memory mem client navigation-churn (solid) 538.1 KB 2,486.5 KB -78.36%
Memory mem client interrupted-navigations (vue) 501.8 KB 2,284.8 KB -78.04%
Memory mem client navigation-churn (react) 542.4 KB 2,178.2 KB -75.1%
Memory mem client unique-location-churn (react) 762.5 KB 2,444.6 KB -68.81%
Memory mem server request-churn (solid) 425.4 KB 1,261.4 KB -66.28%
Memory mem client mount-unmount (solid) 475.5 KB 1,262.4 KB -62.33%
Memory mem server error-paths not-found (vue) 328.8 KB 872.2 KB -62.3%
Memory mem client unique-location-churn (vue) 1,003.9 KB 2,431 KB -58.71%
Memory mem server error-paths unmatched (vue) 479.9 KB 1,046.4 KB -54.14%
Memory mem server aborted-requests (react) 562.8 KB 1,225.1 KB -54.06%
Memory mem server error-paths not-found (react) 250.7 KB 539.2 KB -53.5%
Memory mem client mount-unmount (vue) 832 KB 1,628.7 KB -48.91%
Memory mem client navigation-churn (vue) 1.2 MB 2.2 MB -45.16%
Memory mem server server-fn-churn (solid) 274.9 KB 458.7 KB -40.07%
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing test-codspeed-memory-stabilization (300581b) with main (abf9b81)

Open in CodSpeed

@nx-cloud

nx-cloud Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

View your CI Pipeline Execution ↗ for commit 300581b

Command Status Duration Result
nx affected --targets=test:eslint,test:unit,tes... ✅ Succeeded 2m 23s View ↗
nx run-many --target=build --exclude=examples/*... ✅ Succeeded 2m 11s View ↗

☁️ Nx Cloud last updated this comment at 2026-08-07 06:31:54 UTC

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

🚀 Changeset Version Preview

No changeset entries found. Merging this PR will not cause a version bump for any packages.

@pkg-pr-new

pkg-pr-new Bot commented Aug 7, 2026

Copy link
Copy Markdown
More templates

@tanstack/arktype-adapter

npm i https://pkg.pr.new/@tanstack/arktype-adapter@7988

@tanstack/eslint-plugin-router

npm i https://pkg.pr.new/@tanstack/eslint-plugin-router@7988

@tanstack/eslint-plugin-start

npm i https://pkg.pr.new/@tanstack/eslint-plugin-start@7988

@tanstack/history

npm i https://pkg.pr.new/@tanstack/history@7988

@tanstack/nitro-v2-vite-plugin

npm i https://pkg.pr.new/@tanstack/nitro-v2-vite-plugin@7988

@tanstack/react-router

npm i https://pkg.pr.new/@tanstack/react-router@7988

@tanstack/react-router-devtools

npm i https://pkg.pr.new/@tanstack/react-router-devtools@7988

@tanstack/react-router-ssr-query

npm i https://pkg.pr.new/@tanstack/react-router-ssr-query@7988

@tanstack/react-start

npm i https://pkg.pr.new/@tanstack/react-start@7988

@tanstack/react-start-client

npm i https://pkg.pr.new/@tanstack/react-start-client@7988

@tanstack/react-start-rsc

npm i https://pkg.pr.new/@tanstack/react-start-rsc@7988

@tanstack/react-start-server

npm i https://pkg.pr.new/@tanstack/react-start-server@7988

@tanstack/router-cli

npm i https://pkg.pr.new/@tanstack/router-cli@7988

@tanstack/router-core

npm i https://pkg.pr.new/@tanstack/router-core@7988

@tanstack/router-devtools

npm i https://pkg.pr.new/@tanstack/router-devtools@7988

@tanstack/router-devtools-core

npm i https://pkg.pr.new/@tanstack/router-devtools-core@7988

@tanstack/router-generator

npm i https://pkg.pr.new/@tanstack/router-generator@7988

@tanstack/router-plugin

npm i https://pkg.pr.new/@tanstack/router-plugin@7988

@tanstack/router-ssr-query-core

npm i https://pkg.pr.new/@tanstack/router-ssr-query-core@7988

@tanstack/router-utils

npm i https://pkg.pr.new/@tanstack/router-utils@7988

@tanstack/router-vite-plugin

npm i https://pkg.pr.new/@tanstack/router-vite-plugin@7988

@tanstack/solid-router

npm i https://pkg.pr.new/@tanstack/solid-router@7988

@tanstack/solid-router-devtools

npm i https://pkg.pr.new/@tanstack/solid-router-devtools@7988

@tanstack/solid-router-ssr-query

npm i https://pkg.pr.new/@tanstack/solid-router-ssr-query@7988

@tanstack/solid-start

npm i https://pkg.pr.new/@tanstack/solid-start@7988

@tanstack/solid-start-client

npm i https://pkg.pr.new/@tanstack/solid-start-client@7988

@tanstack/solid-start-server

npm i https://pkg.pr.new/@tanstack/solid-start-server@7988

@tanstack/start-client-core

npm i https://pkg.pr.new/@tanstack/start-client-core@7988

@tanstack/start-fn-stubs

npm i https://pkg.pr.new/@tanstack/start-fn-stubs@7988

@tanstack/start-plugin-core

npm i https://pkg.pr.new/@tanstack/start-plugin-core@7988

@tanstack/start-server-core

npm i https://pkg.pr.new/@tanstack/start-server-core@7988

@tanstack/start-static-server-functions

npm i https://pkg.pr.new/@tanstack/start-static-server-functions@7988

@tanstack/start-storage-context

npm i https://pkg.pr.new/@tanstack/start-storage-context@7988

@tanstack/valibot-adapter

npm i https://pkg.pr.new/@tanstack/valibot-adapter@7988

@tanstack/virtual-file-routes

npm i https://pkg.pr.new/@tanstack/virtual-file-routes@7988

@tanstack/vue-router

npm i https://pkg.pr.new/@tanstack/vue-router@7988

@tanstack/vue-router-devtools

npm i https://pkg.pr.new/@tanstack/vue-router-devtools@7988

@tanstack/vue-router-ssr-query

npm i https://pkg.pr.new/@tanstack/vue-router-ssr-query@7988

@tanstack/vue-start

npm i https://pkg.pr.new/@tanstack/vue-start@7988

@tanstack/vue-start-client

npm i https://pkg.pr.new/@tanstack/vue-start-client@7988

@tanstack/vue-start-server

npm i https://pkg.pr.new/@tanstack/vue-start-server@7988

@tanstack/zod-adapter

npm i https://pkg.pr.new/@tanstack/zod-adapter@7988

commit: 300581b

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant