Attachment lifetimes #1319
Open
Attachment lifetimes #1319
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR introduces a new experimental trait, `.savingAttachments(if:)`, that can be used to control whether a test's attachments are saved or not. XCTest has API around the [`XCTAttachment.Lifetime`](https://developer.apple.com/documentation/xctest/xctattachment/lifetime-swift.enum) enumeration that developers can use to control whether attachments are saved to a test report in Xcode. This enumeration has two cases: ```objc /* * Attachment will be kept regardless of the outcome of the test. */ XCTAttachmentLifetimeKeepAlways = 0, /* * Attachment will only be kept when the test fails, and deleted otherwise. */ XCTAttachmentLifetimeDeleteOnSuccess = 1 ``` I've opted to implement something a bit more granular. A developer can specify `.savingAttachments(if: .testFails)` and `.savingAttachments(if: .testPasses)` or can call some custom function of their own design like `runningInCI` or `hasPlentyOfFloppyDiskSpace`. The default behaviour if this trait is not used is to always save attachments, which is equivalent to `XCTAttachmentLifetimeKeepAlways`. `XCTAttachmentLifetimeDeleteOnSuccess` is, in effect, equivalent to `.savingAttachments(if: .testFails)`, but I hope reads a bit more clearly in context. Here's a usage example: ```swift @test(.savingAttachments(if: .testFails)) func `best test ever`() { Attachment.record("...") // only saves to the test report or to disk if the // next line is uncommented. // Issue.record("sadness") } ``` I've taken the opportunity to update existing documentation for `Attachment` and `Attachable` to try to use more consistent language: a test records an attachment and then the testing library saves it (somewhere). I'm sure I've missed some spots, so please point them out if you see them. Resolves rdar://138921461.
grynspan
requested review from
briancroom,
suzannaratcliff and
jerryjrchen
as code owners
grynspan
added
public-api
grynspan
commented
Sep 16, 2025
| @@ -47,9 +47,19 @@ extension Runner { | |||
| return | |||
| } | |||
|
|
|||
| configuration.eventHandler = { [eventHandler = configuration.eventHandler] event, context in | |||
| configuration.eventHandler = { [oldEventHandler = configuration.eventHandler] event, context in | |||
There was a problem hiding this comment.
This code isn't new, but needed to move from handleEvent as it imposed the wrong layering (saving the attachment before any event handlers actually ran.)
Comment on lines
+47
to
+49
| public static var testFails: Self { | ||
| Self { $0.hasFailed } | ||
| } |
There was a problem hiding this comment.
I think we should include a built-in condition representing a test which did not fail but recorded one or more issues with warning severity. If a test recorded a warning issue, preserving the attachments might be useful to help diagnose the problem.
There was a problem hiding this comment.
Sure, gimme a symbol name and I can add it.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces a new experimental trait,
.savingAttachments(if:), that can be used to control whether a test's attachments are saved or not.XCTest has API around the
XCTAttachment.Lifetimeenumeration that developers can use to control whether attachments are saved to a test report in Xcode. This enumeration has two cases:I've opted to implement something a bit more granular. A developer can specify
.savingAttachments(if: .testFails)and.savingAttachments(if: .testPasses)or can call some custom function of their own design likerunningInCIorhasPlentyOfFloppyDiskSpace. The default behaviour if this trait is not used is to always save attachments, which is equivalent toXCTAttachmentLifetimeKeepAlways.XCTAttachmentLifetimeDeleteOnSuccessis, in effect, equivalent to.savingAttachments(if: .testFails), but I hope reads a bit more clearly in context.Here's a usage example:
I've taken the opportunity to update existing documentation for
AttachmentandAttachableto try to use more consistent language: a test records an attachment and then the testing library saves it (somewhere). I'm sure I've missed some spots, so please point them out if you see them.Resolves rdar://138921461.
Checklist: