`resolveSnapshotPath` Logs an Error, But Tests Continue

[htho]

Environment (please complete the following information):

• Node.js version: v20.15.1
• NPM version: 10.8.2
• Browser name and version: Chrome 131
• Platform name and version: Windows 10
• WebdriverIO version: 9.5.7
• @wdio/visual-service version: 6.3.0

Config of WebdriverIO + @wdio/visual-service

I created a repository where I tried to reproduce the problems I found, when setting up visual testing: https://github.com/htho/wdio-repro-visual-service

Config of WebdriverIO + @wdio/visual-service

export const config: WebdriverIO.Config = {
    ...defaultConfig,

    logLevels: {
        "webdriver": "error",
        "chromedriver": "error",
    },

    reporters: [],
    
    // TODO: REPORT setting up this, throws an error, but tests continue...
    // ERROR @wdio/utils:shim: Error: EISDIR: illegal operation on a directory, read
    resolveSnapshotPath(testPath, _snapExtension) {
        const cwd = process.cwd();
        const testPathRel = path.relative(cwd, testPath);
        const joined = path.join(cwd, "screenshots", testPathRel);
        return joined;
    },

    services: [["visual", {}]],
}

Describe the bug

The tests run, but there is an error EISDIR: illegal operation on a directory, read

To Reproduce
Steps to reproduce the behavior:

1. clone repository
2. npm run test:resolveSnapshotPath

Expected behavior

There shouldn't be an error, because (it seems) everything works as expected.

Log

C:\dev\github.com\htho\wdio-repro-visual-service> npm run test:resolveSnapshotPath

> test:resolveSnapshotPath
> wdio wdio-resolveSnapshotPath.conf.ts


Execution of 1 workers started at 2025-01-16T12:24:51.540Z

2025-01-16T12:24:51.977Z INFO @wdio/cli:launcher: Run onPrepare hook
2025-01-16T12:24:51.979Z INFO @wdio/utils: Setting up browser driver for: chrome@stable
2025-01-16T12:24:51.980Z INFO @wdio/utils: Setting up browser binaries for: chrome@stable
2025-01-16T12:24:51.987Z INFO @wdio/cli:launcher: Run onWorkerStart hook
2025-01-16T12:24:51.988Z INFO @wdio/local-runner: Start worker 0-0 with arg: wdio-resolveSnapshotPath.conf.ts
[0-0] 2025-01-16T12:24:53.049Z INFO @wdio/local-runner: Run worker command: run
[0-0] RUNNING in chrome - file:///C:/dev/github.com/htho/wdio-repro-visual-service/test/specs/test.e2e.ts
[0-0] 2025-01-16T12:24:54.904Z INFO @wdio/utils: Started Chromedriver v131.0.6778.109 with params --port=52506 --allowed-origins=* --allowed-ips=0.0.0.0 in 1039ms on port 52506

DevTools listening on ws://127.0.0.1:52512/devtools/browser/53dd3443-c262-4f60-8c7b-2a16b038b537
[0-0] 2025-01-16T12:24:55.689Z INFO webdriverio:PolyfillManager: Adding polyfill script to context with id 2730DB4E88F41FD2FAB717AEC7898A4A
[0-0] 2025-01-16T12:24:55.693Z INFO @wdio/visual-service: Adding commands to global browser
[0-0] 2025-01-16T12:24:55.694Z INFO @wdio/visual-service: Adding element command "saveElement" to browser object
[0-0] 2025-01-16T12:24:55.695Z INFO @wdio/visual-service: Adding element command "checkElement" to browser object
[0-0] 2025-01-16T12:24:55.695Z INFO @wdio/visual-service: Adding browser command "saveScreen" to browser object
[0-0] 2025-01-16T12:24:55.696Z INFO @wdio/visual-service: Adding browser command "saveFullPageScreen" to browser object
[0-0] 2025-01-16T12:24:55.696Z INFO @wdio/visual-service: Adding browser command "saveTabbablePage" to browser object
[0-0] 2025-01-16T12:24:55.696Z INFO @wdio/visual-service: Adding browser command "checkScreen" to browser object
[0-0] 2025-01-16T12:24:55.697Z INFO @wdio/visual-service: Adding browser command "checkFullPageScreen" to browser object
[0-0] 2025-01-16T12:24:55.697Z INFO @wdio/visual-service: Adding browser command "checkTabbablePage" to browser object
[0-0] 2025-01-16T12:24:55.697Z INFO @wdio/visual-service: Adding browser command "waitForStorybookComponentToBeLoaded" to browser object
[0-0] 2025-01-16T12:24:55.872Z ERROR @wdio/utils:shim: Error: EISDIR: illegal operation on a directory, read
[0-0]     at async readFileHandle (node:internal/fs/promises:553:24)
[0-0]     at Function.create (C:\dev\github.com\htho\wdio-repro-visual-service\node_modules\@vitest\snapshot\dist\index.js:1835:21)
[0-0]     at SnapshotClient.startCurrentRun (C:\dev\github.com\htho\wdio-repro-visual-service\node_modules\@vitest\snapshot\dist\index.js:2105:11)    
[0-0]     at async SnapshotService.beforeTest (file:///C:/dev/github.com/htho/wdio-repro-visual-service/node_modules/expect-webdriverio/lib/snapshot.js:66:9)
[0-0]     at async Promise.all (index 0)
[0-0]     at executeHooksWithArgs (C:\dev\github.com\htho\wdio-repro-visual-service\node_modules\@wdio\utils\build\index.js:876:18)
[0-0]     at Context.testFrameworkFnWrapper (C:\dev\github.com\htho\wdio-repro-visual-service\node_modules\@wdio\utils\build\index.js:1079:39)        
[0-0] Error in "BeforeTest Hook"
EISDIR: illegal operation on a directory, read
[0-0] PASSED in chrome - file:///C:/dev/github.com/htho/wdio-repro-visual-service/test/specs/test.e2e.ts
2025-01-16T12:24:56.225Z INFO @wdio/cli:launcher: Run onWorkerEnd hook

Spec Files:      1 passed, 1 total (100% completed) in 00:00:04

2025-01-16T12:24:56.227Z INFO @wdio/local-runner: Shutting down spawned worker
2025-01-16T12:24:56.479Z INFO @wdio/local-runner: Waiting for 0 to shut down gracefully
2025-01-16T12:24:56.480Z INFO @wdio/local-runner: shutting down
2025-01-16T12:24:56.480Z INFO @wdio/cli:launcher: Run onComplete hook

Additional context
Add any other context about the problem here.

[wswebcreation]

Hi @htho

Thanks for submitting this issue. Are you able to create a PR for this? Help is much appreciated

[wswebcreation]

@htho

There are two things, first of all your solution results in a strange path like /Users/wswebcreation/Git/wdio/issues/wdio-repro-visual-service/screenshots/test/specs/test.e2e.ts
Secondly, the first time this work, the second time you run this the error EISDIR: illegal operation on a directory, read is thrown. This seems to be an issue in the expect-webdriverio-library. I'll move this issue to the repo

[christian-bromann]

Sending it back to you @wswebcreation 😉

The current way how the visual service detects the snapshot directory is not correct if there is a custom resolveSnapshotPath option provided. The #getBaselineFolder method:

https://github.com/webdriverio/visual-testing/blob/main/packages/visual-service/src/service.ts#L120-L138

has to return a directory and not a file path. Because of this, the service creates a folder with the name of the snapshot. Hence, when Vitest tries to read the snapshot file, it throws the error EISDIR: illegal operation on a directory, read because it can't read a directory.