reclassify prepare failure #770
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Skgland
force-pushed
the
reclassify-no-space-on-fetch
branch
2 times, most recently
from
29fb04b to
e2cd63d
Compare
…ious fixed fixing the non-spurious build failure is a non-spurious fix even if we then spuriously fail the tests clean up the comparison logic a bit while at it
Prepare failure are likely spurious and as such are included in the retry-regressed-list.txt to give the crates another change in a retry. This is different to a crate being Broken which is probably non-spurious. Iif either base or test toolchain is a prepare failure the runs can't give any insight as we are missing either a base or a test result.
Skgland
force-pushed
the
reclassify-no-space-on-fetch
branch
from
dd0acaf to
ec7f45e
Compare
Skgland
changed the title
no space left on device during fetch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently prepare failure gets classified as
Errorunless the crate is determined to be broken in which case its classified asBroken. This PR reclassifies the former.Most if not all of the perpare errors are spurious and would benefit from being included in the
retry-regressed-list.txtso that they can be retried.This should cause instances such as https://crater-reports.s3.amazonaws.com/beta-1.86-rustdoc-1/1.85.0/reg/ssstar-0.7.3/log.txt to be classified as prepare-fail instead of error which means the are included in the retry-regressed-list.txt and have a change at properly regressing in a retry.
This should help prevent regression in beta crater run being missed due to running out of disk space as was the case with the above example.
The first commit fixes existing clippy problems.
The second commit removes a no longer existing crate from the config.toml (broke config check)
The third commit cleans-up the compare match.
Basically a regression is spurious if the test toolchain result is spurious
and a fix is spurious if the base toolchain result is spurious.
This only changes the classification of
BuildFail(non-spurious) -> TestFail(spurious)fromSpuriousFixedtoFixed.I think the new classification is correct and the old one was wrong.
The fourth commit adds a new
ComparisonandTestResultvariant to properly classify the failure mode of preparation failing for a non-broken crate.It also adds crates with this new Comparison variant to the
retry-regressed-list.txt.I expect this to significantly increase the size of the
retry-regressed-list.txt.Edit: Including
PrepareFailin theretry-regressed-list.txtmakes the file a bit misnamed, maybe it should be renamed toretry-list.txtthe function for filtering the cratesgen_retry_listalready doesn't contain regressed in its name.