Description
The compiled-caller branch of resolveRuntimeTasksRoot() deliberately returns a normalized forward-slash path, but its unit test builds the expected value with the host-specific path.join().
The two values agree on POSIX. On Windows, the implementation returns /repo/packages/evals/dist/esm/tasks while the assertion expects \repo\packages\evals\dist\esm\tasks, so the evals unit suite has a deterministic platform-only failure.
Code reference
packages/evals/runtimePaths.ts:111-118 normalizes the caller path and constructs the compiled tasks root with forward slashes.packages/evals/tests/runtimePaths.test.ts:14-20 supplies POSIX fixture paths but uses host-native path.join() for the compiled expectation.- The source-mode case at
packages/evals/tests/runtimePaths.test.ts:6-11 happens to use path.join() in both implementation and expectation, so it passes.
Reproduction
On Windows:
.\node_modules\.bin\vitest.CMD run packages/evals/tests/runtimePaths.test.ts
The uses compiled tasks for built CLI callers case fails with the equivalent of:
expected "\repo\packages\evals\dist\esm\tasks"
received "/repo/packages/evals/dist/esm/tasks"
Expected behavior
A fixture written as a POSIX path should have a platform-independent expected result, or both sides should be normalized by the same helper.
Actual behavior
The assertion changes with the host OS while the function's compiled branch intentionally returns normalized separators.
Why it matters
This makes the evals unit suite fail on Windows for test-fixture formatting rather than product behavior. It also makes full-suite failures noisier and can mask genuine eval runtime-path regressions.
Duplicate/history check
I searched open and closed issues and PRs for resolveRuntimeTasksRoot, runtime path tests, Windows separators, and the compiled tasks directory. The code came through the evals/v4 history, but I found no report or PR addressing this assertion.
Description
The compiled-caller branch of
resolveRuntimeTasksRoot()deliberately returns a normalized forward-slash path, but its unit test builds the expected value with the host-specificpath.join().The two values agree on POSIX. On Windows, the implementation returns
/repo/packages/evals/dist/esm/taskswhile the assertion expects\repo\packages\evals\dist\esm\tasks, so the evals unit suite has a deterministic platform-only failure.Code reference
packages/evals/runtimePaths.ts:111-118normalizes the caller path and constructs the compiled tasks root with forward slashes.packages/evals/tests/runtimePaths.test.ts:14-20supplies POSIX fixture paths but uses host-nativepath.join()for the compiled expectation.packages/evals/tests/runtimePaths.test.ts:6-11happens to usepath.join()in both implementation and expectation, so it passes.Reproduction
On Windows:
The
uses compiled tasks for built CLI callerscase fails with the equivalent of:Expected behavior
A fixture written as a POSIX path should have a platform-independent expected result, or both sides should be normalized by the same helper.
Actual behavior
The assertion changes with the host OS while the function's compiled branch intentionally returns normalized separators.
Why it matters
This makes the evals unit suite fail on Windows for test-fixture formatting rather than product behavior. It also makes full-suite failures noisier and can mask genuine eval runtime-path regressions.
Duplicate/history check
I searched open and closed issues and PRs for
resolveRuntimeTasksRoot, runtime path tests, Windows separators, and the compiled tasks directory. The code came through the evals/v4 history, but I found no report or PR addressing this assertion.