Skip to content

[do not merge] CI experiments #112049

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

[do not merge] CI experiments #112049

wants to merge 1 commit into from
+2 −2

Conversation

Kobzol
Copy link
Member

@Kobzol Kobzol commented May 28, 2023 •
edited
Loading

Various CI experiments for try/dist builds.

r? @ghost

@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels May 28, 2023
@Kobzol
Copy link
Member Author

Kobzol commented May 28, 2023

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

Sign in to view
@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 28, 2023
@bors
Copy link
Collaborator

bors commented May 28, 2023

⌛ Trying commit c42e33cf36e728937b9e7d9bbb16c17d0b364e23 with merge c60ecf63ab7a2a0a260b0a8ee02adff4f4dbfe87...

@bors
Copy link
Collaborator

bors commented May 28, 2023

☀️ Try build successful - checks-actions
Build commit: c60ecf63ab7a2a0a260b0a8ee02adff4f4dbfe87 (c60ecf63ab7a2a0a260b0a8ee02adff4f4dbfe87)

@rust-timer

This comment has been minimized.

Sign in to view
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (c60ecf63ab7a2a0a260b0a8ee02adff4f4dbfe87): comparison URL.

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 0.3% [0.3%, 0.3%] 2
Regressions ❌
(secondary)		 0.3% [0.3%, 0.3%] 2
Improvements ✅
(primary)		 -0.3% [-0.3%, -0.3%] 1
Improvements ✅
(secondary)		 -0.4% [-0.7%, -0.2%] 9
All ❌✅ (primary) 0.1% [-0.3%, 0.3%] 3

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 3.4% [3.4%, 3.4%] 1
Regressions ❌
(secondary)		 1.8% [1.8%, 1.8%] 1
Improvements ✅
(primary)		 -3.5% [-3.5%, -3.5%] 1
Improvements ✅
(secondary)		 -3.4% [-3.7%, -3.1%] 2
All ❌✅ (primary) -0.1% [-3.5%, 3.4%] 2

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 - - 0
Improvements ✅
(primary)		 - - 0
Improvements ✅
(secondary)		 -2.0% [-2.2%, -1.7%] 2
All ❌✅ (primary) - - 0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 646.44s -> 646.666s (0.03%)

@rustbot rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels May 29, 2023
@Kobzol
Copy link
Member Author

Kobzol commented May 29, 2023

Looks like removing libcore from the profile set doesn't hurt much. Now I'll try to also completely remove rustc-perf gathering when gathering LLVM PGO profiles.

Looks like after the initial stage 2 build with an instrumented LLVM, there is already 6 GiB of profiles (after benchmarks it's usually 20 GiB). So instrumenting stage 2 build provides a lot of data! I'll try what happens if we delete the profiles.

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

Sign in to view
@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 29, 2023
@bors
Copy link
Collaborator

bors commented May 29, 2023

⌛ Trying commit 31c917d813e3461b6babe1220a1ac15fc66eae53 with merge 3004a6cee0725e571674b0efdefcd9b82b1e11d6...

@bors
Copy link
Collaborator

bors commented May 29, 2023

☀️ Try build successful - checks-actions
Build commit: 3004a6cee0725e571674b0efdefcd9b82b1e11d6 (3004a6cee0725e571674b0efdefcd9b82b1e11d6)

@rust-timer

This comment has been minimized.

Sign in to view
@Kobzol
Copy link
Member Author

Kobzol commented May 29, 2023

@bors try

@bors
Copy link
Collaborator

bors commented May 29, 2023

⌛ Trying commit 042e02535bb5a95c8d6d77fc5954d762b73079e0 with merge ed48ba9307a0a1b523e3519499c3918be2dba056...

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (3004a6cee0725e571674b0efdefcd9b82b1e11d6): comparison URL.

Overall result: ❌ regressions - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 3.2% [0.3%, 7.3%] 51
Regressions ❌
(secondary)		 3.1% [0.4%, 5.9%] 24
Improvements ✅
(primary)		 - - 0
Improvements ✅
(secondary)		 - - 0
All ❌✅ (primary) 3.2% [0.3%, 7.3%] 51

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 2.5% [2.5%, 2.5%] 1
Improvements ✅
(primary)		 - - 0
Improvements ✅
(secondary)		 - - 0
All ❌✅ (primary) - - 0

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 2.6% [0.8%, 4.1%] 32
Regressions ❌
(secondary)		 2.8% [1.8%, 3.6%] 12
Improvements ✅
(primary)		 - - 0
Improvements ✅
(secondary)		 - - 0
All ❌✅ (primary) 2.6% [0.8%, 4.1%] 32

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 642.999s -> 642.878s (-0.02%)

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 29, 2023
@bors
Copy link
Collaborator

bors commented May 29, 2023

☀️ Try build successful - checks-actions
Build commit: ed48ba9307a0a1b523e3519499c3918be2dba056 (ed48ba9307a0a1b523e3519499c3918be2dba056)

@Kobzol
Copy link
Member Author

Kobzol commented May 29, 2023

@rust-timer build ed48ba9307a0a1b523e3519499c3918be2dba056

@rust-timer

This comment has been minimized.

Sign in to view
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (ed48ba9307a0a1b523e3519499c3918be2dba056): comparison URL.

Overall result: ✅ improvements - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 - - 0
Improvements ✅
(primary)		 -1.0% [-2.0%, -0.3%] 33
Improvements ✅
(secondary)		 -1.1% [-1.5%, -0.4%] 12
All ❌✅ (primary) -1.0% [-2.0%, -0.3%] 33

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 2.5% [2.5%, 2.5%] 1
Improvements ✅
(primary)		 - - 0
Improvements ✅
(secondary)		 - - 0
All ❌✅ (primary) - - 0

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 - - 0
Improvements ✅
(primary)		 - - 0
Improvements ✅
(secondary)		 -2.8% [-2.8%, -2.8%] 1
All ❌✅ (primary) - - 0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 642.999s -> 644.35s (0.21%)

@rustbot rustbot removed the perf-regression Performance regression. label May 29, 2023
@rust-bors

This comment has been minimized.

Sign in to view
@rust-bors
Copy link

rust-bors bot commented Sep 6, 2025

☀️ Try build successful (CI)
Build commit: 71ea9a0 (71ea9a0cacc3473c7b6852c17453259f74635c62, parent: 0d0f4eac8b98133e5da6d3604d86a8f3b5a67844)

@rust-timer

This comment has been minimized.

Sign in to view
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (71ea9a0): comparison URL.

Overall result: ✅ improvements - no action needed

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 - - 0
Improvements ✅
(primary)		 -8.4% [-36.7%, -0.2%] 67
Improvements ✅
(secondary)		 -4.5% [-18.5%, -0.1%] 88
All ❌✅ (primary) -8.4% [-36.7%, -0.2%] 67

Max RSS (memory usage)

Results (primary -5.8%, secondary -2.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 3.3% [2.3%, 4.9%] 3
Improvements ✅
(primary)		 -5.8% [-16.9%, -0.6%] 27
Improvements ✅
(secondary)		 -4.2% [-8.6%, -2.3%] 11
All ❌✅ (primary) -5.8% [-16.9%, -0.6%] 27

Cycles

Results (primary -14.6%, secondary -9.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 - - 0
Improvements ✅
(primary)		 -14.6% [-37.1%, -3.0%] 40
Improvements ✅
(secondary)		 -9.4% [-21.2%, -2.1%] 41
All ❌✅ (primary) -14.6% [-37.1%, -3.0%] 40

Binary size

Results (primary -30.9%, secondary -25.5%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 - - 0
Improvements ✅
(primary)		 -30.9% [-62.4%, -0.2%] 79
Improvements ✅
(secondary)		 -25.5% [-69.9%, -0.0%] 86
All ❌✅ (primary) -30.9% [-62.4%, -0.2%] 79

Bootstrap: 469.522s -> 467.253s (-0.48%)
Artifact size: 390.39 MiB -> 387.98 MiB (-0.62%)

@rustbot rustbot removed S-waiting-on-perf Status: Waiting on a perf run to be completed. perf-regression Performance regression. labels Sep 6, 2025
@Kobzol Kobzol force-pushed the pgo-omit-benchmarks branch 2 times, most recently from 042b55c to f8ab528 Compare September 6, 2025 19:52
@Kobzol
Copy link
Member Author

Kobzol commented Sep 6, 2025

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

Sign in to view
@rust-bors

This comment has been minimized.

Sign in to view
rust-bors bot added a commit that referenced this pull request Sep 6, 2025
@rust-bors
Auto merge of #112049 - Kobzol:pgo-omit-benchmarks, r=<try>
87c3e1e 
[do not merge] CI experiments
@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Sep 6, 2025
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@rust-bors
Copy link

rust-bors bot commented Sep 6, 2025

☀️ Try build successful (CI)
Build commit: 87c3e1e (87c3e1ecd699573f7cb4c9074b8727956bd37a74, parent: bea625f3275e3c897dc965ed97a1d19ef7831f01)

@rust-timer

This comment has been minimized.

Sign in to view
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (87c3e1e): comparison URL.

Overall result: ✅ improvements - no action needed

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 - - 0
Improvements ✅
(primary)		 -12.4% [-42.8%, -0.1%] 80
Improvements ✅
(secondary)		 -8.6% [-34.3%, -0.1%] 102
All ❌✅ (primary) -12.4% [-42.8%, -0.1%] 80

Max RSS (memory usage)

Results (primary -6.6%, secondary -5.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 3.8% [3.8%, 3.8%] 1
Improvements ✅
(primary)		 -6.6% [-23.0%, -1.3%] 50
Improvements ✅
(secondary)		 -6.0% [-25.1%, -2.0%] 23
All ❌✅ (primary) -6.6% [-23.0%, -1.3%] 50

Cycles

Results (primary -19.2%, secondary -14.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 - - 0
Improvements ✅
(primary)		 -19.2% [-42.7%, -2.1%] 50
Improvements ✅
(secondary)		 -14.6% [-35.1%, -2.3%] 54
All ❌✅ (primary) -19.2% [-42.7%, -2.1%] 50

Binary size

Results (primary -45.2%, secondary -56.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 - - 0
Improvements ✅
(primary)		 -45.2% [-88.5%, -0.2%] 83
Improvements ✅
(secondary)		 -56.6% [-88.6%, -0.0%] 88
All ❌✅ (primary) -45.2% [-88.5%, -0.2%] 83

Bootstrap: 468.001s -> 465.788s (-0.47%)
Artifact size: 387.44 MiB -> 379.64 MiB (-2.01%)

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Sep 7, 2025
@Kobzol Kobzol force-pushed the pgo-omit-benchmarks branch from f8ab528 to b5e0064 Compare September 11, 2025 10:50
@Kobzol
Copy link
Member Author

Kobzol commented Sep 11, 2025

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

Sign in to view
rust-bors bot added a commit that referenced this pull request Sep 11, 2025
@rust-bors
Auto merge of #112049 - Kobzol:pgo-omit-benchmarks, r=<try>
3cbcac7 
[do not merge] CI experiments
@rust-bors

This comment has been minimized.

Sign in to view
@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Sep 11, 2025
ranger-ross
ranger-ross reviewed Sep 11, 2025
View reviewed changes
Copy link
Contributor

@ranger-ross ranger-ross left a comment •
edited by rustbot
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

View changes since this review

.gitmodules
@@ -4,7 +4,7 @@
shallow = true
[submodule "src/tools/cargo"]
path = src/tools/cargo
url = https://github.com/rust-lang/cargo.git
url = https://github.com/ranger-ross/cargo.git
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should point to the branch with the layout changes?

Suggested change
url = https://github.com/ranger-ross/cargo.git
url = https://github.com/ranger-ross/cargo.git
branch = impl-build-dir-layout

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It shouldn't matter too much, the important thing is the commit of the submodule, that is not determined by .gitmodules. I changed it to the commit SHA of your PR.

@rust-bors
Copy link

rust-bors bot commented Sep 11, 2025

☀️ Try build successful (CI)
Build commit: 3cbcac7 (3cbcac7f3839d94528ac716e67d4f99f2619720e, parent: 5e33838ccad070f1536ed82336dd0133e2681233)

@rust-timer

This comment has been minimized.

Sign in to view
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (3cbcac7): comparison URL.

Overall result: no relevant changes - no action needed

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results (secondary -2.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 - - 0
Improvements ✅
(primary)		 - - 0
Improvements ✅
(secondary)		 -2.6% [-2.6%, -2.6%] 1
All ❌✅ (primary) - - 0

Cycles

Results (secondary 2.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 2.3% [2.3%, 2.3%] 1
Improvements ✅
(primary)		 - - 0
Improvements ✅
(secondary)		 - - 0
All ❌✅ (primary) - - 0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 468.387s -> 469.16s (0.17%)
Artifact size: 387.81 MiB -> 387.87 MiB (0.02%)

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Sep 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
1 more reviewer

@ranger-ross ranger-ross ranger-ross left review comments

Reviewers whose approvals may not affect merge requirements
Assignees

@Kobzol Kobzol

Labels
A-CI Area: Our Github Actions CI A-compiletest Area: The compiletest test runner A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. A-meta Area: Issues & PRs about the rust-lang/rust repository itself A-query-system Area: The rustc query system (https://rustc-dev-guide.rust-lang.org/query.html) A-testsuite Area: The testsuite used to check the correctness of rustc A-translation Area: Translation infrastructure, and migrating existing diagnostics to SessionDiagnostic S-experimental Status: Ongoing experiment that does not require reviewing and won't be merged in its current state. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. WG-trait-system-refactor The Rustc Trait System Refactor Initiative (-Znext-solver)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.