Bump rustc-perf and update PGO crates

[compiler-errors]

Updates rustc-perf to rust-lang/rustc-perf@8158f78, and updates the crates.

r? @Kobzol

[rustbot]

⚠️ Warning ⚠️

• Some commits in this PR modify submodules.

[Kobzol]

@bors try @rust-timer queue

[bors]

⌛ Trying commit d6c4ab8 with merge 4c4ffc4...

[bors]

☀️ Try build successful - checks-actions
Build commit: 4c4ffc4 (4c4ffc4ef78313dc32c25e917372a415ab446035)

[rust-timer]

Finished benchmarking commit (4c4ffc4): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	0.3%	[0.1%, 1.7%]	70
Regressions ❌ (secondary)	0.6%	[0.1%, 2.2%]	89
Improvements ✅ (primary)	-0.4%	[-1.9%, -0.1%]	84
Improvements ✅ (secondary)	-2.5%	[-14.4%, -0.1%]	65
All ❌✅ (primary)	-0.1%	[-1.9%, 1.7%]	154

Max RSS (memory usage)

Results (primary -1.6%, secondary -1.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.4%	[1.1%, 1.7%]	2
Regressions ❌ (secondary)	4.9%	[2.2%, 9.9%]	15
Improvements ✅ (primary)	-1.8%	[-4.4%, -0.8%]	30
Improvements ✅ (secondary)	-4.5%	[-7.8%, -1.1%]	25
All ❌✅ (primary)	-1.6%	[-4.4%, 1.7%]	32

Cycles

Results (primary -1.0%, secondary -8.5%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.0%	[-1.1%, -0.9%]	2
Improvements ✅ (secondary)	-8.5%	[-17.1%, -1.5%]	24
All ❌✅ (primary)	-1.0%	[-1.1%, -0.9%]	2

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 776.203s -> 774.104s (-0.27%)
Artifact size: 366.33 MiB -> 371.89 MiB (1.52%)

[compiler-errors]

new solver perf improvements even without the crates being included is really funny 😆

i'm not missing anything here, right?

[Kobzol]

Sorry, it's the same bug 🤦 Since some of the used benchmarks are prefixes of the new solver benchmarks, the new solver benchmarks are included even though we don't want them to be. I will fix it next week.

CC @nnethercote I think I will just change the behavior of --include to be exactly matching, I feel like we had enough trouble with the prefix/substring matching logic..

[lqd]

Can we --exclude-suffix new-solver maybe?

[Kobzol]

But we actually want to add a few -new-solver benchmarks in a follow-up 🫠

[nnethercote]

CC @nnethercote I think I will just change the behavior of --include to be exactly matching, I feel like we had enough trouble with the prefix/substring matching logic..

It's a shame to have to type the extra chars, but yeah, the false matches are really annoying.

I wonder if we could allow "exact match" or "exact match but without the version number at the end". E.g. nalgebra-0.33.0 or nalgebra would match, but not nalg. Because it's the version number at the end that's the most annoying part, I usually have to look that up.

[Kobzol]

I thought about this a bit more, and maybe to avoid breaking existing users it would be better to just add a new flag, e.g. --benchmarks or --match, and use that for programmatic usage of rustc-perf where you don't want ambiguities, while keeping --include and others for interactive usage, to make it simpler for humans. We could disallow combining the new flag with the old ones to avoid dealing with the edge cases.

[Kobzol]

rust-lang/rustc-perf#2139 should hopefully fix this.

[Kobzol]

Bumping rustc-perf to 9863312596d1106a96fa54f88ee65f8839d4397a and changing --include to --exact-match here should fix the benchmark matching issue.

[rustbot]

Some changes occurred in src/tools/opt-dist

cc @Kobzol

[compiler-errors]

@bors try @rust-timer queue

[rust-timer]

Finished benchmarking commit (b7413ea): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	0.3%	[0.1%, 2.0%]	64
Regressions ❌ (secondary)	0.7%	[0.0%, 3.3%]	101
Improvements ✅ (primary)	-0.4%	[-3.3%, -0.1%]	68
Improvements ✅ (secondary)	-0.9%	[-4.8%, -0.1%]	58
All ❌✅ (primary)	-0.1%	[-3.3%, 2.0%]	132

Max RSS (memory usage)

Results (primary -0.3%, secondary 1.5%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.6%	[1.2%, 4.3%]	20
Regressions ❌ (secondary)	3.0%	[1.3%, 7.0%]	60
Improvements ✅ (primary)	-1.7%	[-4.6%, -0.5%]	39
Improvements ✅ (secondary)	-4.4%	[-7.2%, -1.5%]	15
All ❌✅ (primary)	-0.3%	[-4.6%, 4.3%]	59

Cycles

Results (primary -1.5%, secondary -3.3%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.5%	[-3.1%, -0.7%]	4
Improvements ✅ (secondary)	-3.3%	[-7.3%, -1.6%]	16
All ❌✅ (primary)	-1.5%	[-3.1%, -0.7%]	4

Binary size

Results (primary -1.1%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.1%	[-1.1%, -1.1%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-1.1%	[-1.1%, -1.1%]	1

Bootstrap: 780.121s -> 777.226s (-0.37%)
Artifact size: 366.40 MiB -> 370.24 MiB (1.05%)

[Kobzol]

We changed both the PGO training set and the benchmarks that we measure, so we have to take what we get. The binary size regression of the toolchain is a bit annoying, but I guess there's not much to do here.

@bors r+

[bors]

📌 Commit fd2242c has been approved by Kobzol

It is now in the queue for this repository.

[bors]

⌛ Testing commit fd2242c with merge c455ff2...

[bors]

💔 Test failed - checks-actions

[lqd]

@Kobzol we’ve had the same issue when trying individual benchmarks with cargo, maybe they need [workspace] in their Cargo.toml.

We used to do that when adding the benchmarks before, but I don’t know if it’s been done for this round of updates?

[Kobzol]

Hmm, I wonder why distcheck tries to load these files. But yeah, this is a bug, fixed in rust-lang/rustc-perf#2144.

[compiler-errors]

Ok, bumped to master once again.

[Kobzol]

Thanks!

@bors r+

[bors]

📌 Commit 73382e4 has been approved by Kobzol

It is now in the queue for this repository.

[bors]

⌛ Testing commit 73382e4 with merge 1bbd62e...

[bors]

☀️ Test successful - checks-actions
Approved by: Kobzol
Pushing 1bbd62e to master...

[github-actions]

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing 13718eb (parent) -> 1bbd62e (this PR)

Test differences

No test diffs found

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard 1bbd62e547ba5cc08ccb44c27def3d33195d2dd5 --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

1. aarch64-gnu: 6742.0s -> 8338.7s (23.7%)
2. dist-aarch64-linux: 5450.8s -> 6553.9s (20.2%)
3. dist-x86_64-apple: 8277.5s -> 9516.4s (15.0%)
4. x86_64-apple-2: 4811.2s -> 5131.0s (6.6%)
5. x86_64-msvc-2: 7171.1s -> 6711.2s (-6.4%)
6. x86_64-gnu-llvm-20-1: 4147.2s -> 3920.3s (-5.5%)
7. x86_64-mingw-2: 7457.5s -> 7084.0s (-5.0%)
8. aarch64-apple: 5398.9s -> 5657.1s (4.8%)
9. dist-powerpc-linux: 5247.4s -> 5484.4s (4.5%)
10. x86_64-gnu-llvm-20-3: 7162.4s -> 7482.1s (4.5%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

[rust-timer]

Finished benchmarking commit (1bbd62e): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Our benchmarks found a performance regression caused by this PR.
This might be an actual regression, but it can also be just noise.

Next Steps:

• If the regression was expected or you think it can be justified,
  please write a comment with sufficient written justification, and add
  @rustbot label: +perf-regression-triaged to it, to mark the regression as triaged.
• If you think that you know of a way to resolve the regression, try to create
  a new PR with a fix for the regression.
• If you do not understand the regression or you think that it is just noise,
  you can ask the @rust-lang/wg-compiler-performance working group for help (members of this group
  were already notified of this PR).

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	0.3%	[0.1%, 2.0%]	46
Regressions ❌ (secondary)	0.7%	[0.1%, 3.2%]	110
Improvements ✅ (primary)	-0.4%	[-0.9%, -0.1%]	72
Improvements ✅ (secondary)	-0.9%	[-4.8%, -0.2%]	62
All ❌✅ (primary)	-0.1%	[-0.9%, 2.0%]	118

Max RSS (memory usage)

Results (primary -2.0%, secondary -2.8%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.6%	[0.4%, 4.9%]	20
Improvements ✅ (primary)	-2.0%	[-4.3%, -0.8%]	31
Improvements ✅ (secondary)	-5.8%	[-11.2%, -0.7%]	30
All ❌✅ (primary)	-2.0%	[-4.3%, -0.8%]	31

Cycles

Results (primary -1.0%, secondary -1.4%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.8%	[0.4%, 2.3%]	10
Improvements ✅ (primary)	-1.0%	[-1.1%, -0.8%]	4
Improvements ✅ (secondary)	-2.5%	[-6.7%, -0.5%]	20
All ❌✅ (primary)	-1.0%	[-1.1%, -0.8%]	4

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 780.508s -> 778.428s (-0.27%)
Artifact size: 368.46 MiB -> 370.31 MiB (0.50%)