chore: Try out nextest in CI #11692
Merged
chore: Try out nextest in CI #11692
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
✔️ Deploy Preview for vector-project canceled. 🔨 Explore the source changes: 1715fdb 🔍 Inspect the deploy log: https://app.netlify.com/sites/vector-project/deploys/62268c5a45671d0007a413d1 |
github-actions
bot
added
the
domain: ci
jszwedko
force-pushed
the
try-out-nextest
branch
from
1e47b47 to
098e84c
Compare
github-actions
bot
added
the
domain: vrl
Starting with flakey test retries. Signed-off-by: Jesse Szwedko <[email protected]>
`nextest` doesn't work with criterion benches: nextest-rs/nextest#96 Signed-off-by: Jesse Szwedko <[email protected]>
Since criterion doesn't support the flags mentioned on https://nexte.st/book/custom-test-harnesses.html Signed-off-by: Jesse Szwedko <[email protected]>
jszwedko
force-pushed
the
try-out-nextest
branch
from
31bc075 to
d48dad8
Compare
Signed-off-by: Jesse Szwedko <[email protected]>
Signed-off-by: Jesse Szwedko <[email protected]>
Signed-off-by: Jesse Szwedko <[email protected]>
Signed-off-by: Jesse Szwedko <[email protected]>
Signed-off-by: Jesse Szwedko <[email protected]>
jszwedko
commented
Mar 7, 2022
| @@ -816,44 +816,37 @@ enrichment-tables-benches = ["enrichment-tables-file"] | |||
| [[bench]] | |||
| name = "default" | |||
| harness = false | |||
| test = true | |||
There was a problem hiding this comment.
I dropped these because criterion doesn't play well with nextest yet: bheisler/criterion.rs#562
There was a problem hiding this comment.
I am okay with this, considering that we'll pick these up with clippy.
Signed-off-by: Jesse Szwedko <[email protected]>
spencergilbert
approved these changes
Mar 7, 2022
blt
approved these changes
Mar 7, 2022
| @@ -816,44 +816,37 @@ enrichment-tables-benches = ["enrichment-tables-file"] | |||
| [[bench]] | |||
| name = "default" | |||
| harness = false | |||
| test = true | |||
There was a problem hiding this comment.
I am okay with this, considering that we'll pick these up with clippy.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I added flakey test retrying but otherwise haven't taken advantage of any other nextest features yet. I hope to expand this to output JUnit compatible output to integrate with Datadog's CI product. Even without that integration, I think this is a worthwhile change for the flakey test behavior and improved output so I think this can safely go in first.
There are a couple of things to be aware of:
cargo-nextestdoesn't support running docs tests so these are split out intomake test-docscargo-nextestandcriteriondon't play well yet so this PR drops testing of benches.make check-clippyshould still ensure they compile though.Assuming we like this, I'll expand it out to the integration tests and see if I can get the JUnit parts wired up.
Closes: #11377
Signed-off-by: Jesse Szwedko [email protected]