Skip to content

[TD] Script to determine which reverts are caused by bad TD #6911

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

clee2000
Copy link
Contributor

@clee2000 clee2000 commented Jul 9, 2025

Honestly a pretty messy script

Determines a commit was caused by bad TD, basically:

  1. find revert
  2. find originally merged commit
  3. find last commit on the PR before the merge
  4. check merged commit for failures
  5. check if failure was excluded by TD on the last commit on the PR

Obviously not perfect, but spot checking it does seem to be ok

Counts are pretty volatile if granularity is too small (week granularity is very volatile), so I'm not sure if this can really be displayed. Maybe should make this periodic and still upload to clickhouse? idk

End of the output looks like this but theres a lot of extra output before it to help debug

CAUSED BY BAD TD: 27 / 184 = 14.67%
Unable to check (lack run id) on PR: 1 / 184 = 0.54%
Total caused by bad TD: 27 / 184 = 14.67%
Month 674: 13 bad TD / 77 total = 16.88%
Month 675: 14 bad TD / 107 total = 13.08%

Also make clickhouse.py client able to be used in thread pool executor (http client didn't like having 1 http client for multiple threads i think)

@pytorch-bot pytorch-bot bot added the ci-no-td label Jul 9, 2025
Copy link

vercel bot commented Jul 9, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Updated (UTC)
torchci ⬜️ Ignored (Inspect) Visit Preview Jul 9, 2025 9:54pm

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 9, 2025
@clee2000 clee2000 marked this pull request as ready for review July 9, 2025 21:45
@clee2000 clee2000 requested a review from a team July 9, 2025 21:45
Copy link

@github-advanced-security github-advanced-security bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lintrunner found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-no-td CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants