Skip to content

XCTest results aren't correct when you have identical XCTestSuites in different test targets #1036

Open
@plemarquand

Description

@plemarquand

On non Darwin platforms, XCTest results aren't correct when you have identical XCTestSuites different test targets.

To Reproduce

Create a test project with two test targets defined.

// Package.swift
...
targets: [
  .target(name: "TestApp"),
  .testTarget(name: "TestTarget1"),
  .testTarget(name: "TestTarget2"),
]

And create two test suites, one in each target.

// TestTarget1/TestFoo.swift
final class MyTestSuite: XCTestCase {
  func testFoo() {}
}
// TestTarget2/TestFoo.swift
final class MyTestSuite: XCTestCase {
  func testFoo() {}
}

Now select both test suites in the Test Explorer and run both. Notice only one test is marked as passing. This stems from the fact that unlike on macOS, on non Darwin platforms the test target is not included in the test run output.

Expected behavior
Both tests show as passing.

Environment

  • OS: Ubuntu 20.04
  • Swift version: Swift version 6.0-dev (LLVM 31f0cdc6d5ff8a6, Swift 497b266c2188eae)
  • Visual Studio Code version: 1.92.2
  • vscode-swift version: 1.10.7

Additional context

Here is the console output for a single suite run on Ubuntu

Test Suite 'Selected tests' started at 2024-08-27 20:23:17.187
Test Suite 'MyTestSuite' started at 2024-08-27 20:23:17.188
Test Case 'MyTestSuite.testFoo' started at 2024-08-27 20:23:17.188
Test Case 'MyTestSuite.testFoo' passed (0.0 seconds)
Test Suite 'MyTestSuite' passed at 2024-08-27 20:23:17.188

And the same run on macOS:

Test Suite 'Selected tests' started at 2024-08-28 09:13:42.501.
Test Suite 'TestTarget1.xctest' started at 2024-08-28 09:13:42.502.
Test Suite 'MyTestSuite' started at 2024-08-28 09:13:42.502.
Test Case '-[TestTarget1.MyTestSuite testFoo]' started.
Test Case '-[TestTarget1.MyTestSuite testFoo]' passed (0.001 seconds).

Metadata

Metadata

Assignees

No one assigned

    Labels

    XCTestIssues related to testing integration for XCTestbugSomething isn't working

    Type

    No type

    Projects

    Status

    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions