Only work-steal in the main loop

[Zoxc]

This PR removes work stealing from other threads, except when in the main loop. This avoids situations where we steal work we can't complete due to waiting on queries. This should fix bugs like rust-lang/rust#111521 and rust-lang/rust#115223, but it does have a negative performance impact.

Tests for 6 threads:

Benchmark	Before	After
	Time	Time	%
🟣 clap:check	0.6502s	0.7641s	💔 17.52%
🟣 hyper:check	0.1396s	0.1508s	💔 8.00%
🟣 regex:check	0.4033s	0.4413s	💔 9.43%
🟣 syn:check	0.6749s	0.7749s	💔 14.81%
🟣 syntex_syntax:check	2.1708s	2.4696s	💔 13.76%
Total	4.0389s	4.6007s	💔 13.91%
Summary	1.0000s	1.1270s	💔 12.70%

Benchmark	Before	After
	Time	Time	%
🟠 clap:debug	2.3400s	2.4645s	💔 5.32%
🟠 hyper:debug	0.3807s	0.3919s	💔 2.96%
🟠 regex:debug	1.7557s	1.7719s	0.92%
🟠 syn:debug	1.7401s	1.8426s	💔 5.89%
🟠 syntex_syntax:debug	6.6743s	6.9144s	💔 3.60%
Total	12.8908s	13.3854s	💔 3.84%
Summary	1.0000s	1.0374s	💔 3.74%

[SparrowLii]

The regression looks regrettable, and it will probably prevent us from landing parallel rustc.

I would like us to decide whether to work-stealing by querying dependencies instead of prohibiting it in inner loops.

I hope we can left this change and try other implementations first.

[Zoxc]

By increasing the thread count we can increase the work stealing done and thus reduce the regression:

Benchmark	Before PR (6 threads ,-j6)	After PR (12 threads ,-j6)
	Time	Time	%
🟣 clap:check	0.6556s	0.6647s	💔 1.38%
🟣 hyper:check	0.1485s	0.1519s	💔 2.32%
🟣 regex:check	0.4050s	0.4126s	💔 1.89%
🟣 syn:check	0.6795s	0.6895s	💔 1.47%
🟣 syntex_syntax:check	2.1858s	2.2038s	0.82%
Total	4.0743s	4.1224s	💔 1.18%
Summary	1.0000s	1.0158s	💔 1.58%

[Zoxc]

@cuviper Would you be up for reviewing this?

[zetanumbers]

Hello! Do you have any tips on currently triggering those bugs?

[cuviper]

I haven't had time to review this in depth -- but seeing how many deep areas it needs to change, along with existing patches diverging from rayon, I'm starting to think that we would be better off with a bespoke rustc_threadpool that doesn't try to be a modified rayon. Most of rayon's API is not used at all in rustc #[cfg(parallel_compiler)], and I think a lot of rustc-rayon's customization would be easier under a more limited API.

[zetanumbers]

I haven't had time to review this in depth -- but seeing how many deep areas it needs to change, along with existing patches diverging from rayon, I'm starting to think that we would be better off with a bespoke rustc_threadpool that doesn't try to be a modified rayon. Most of rayon's API is not used at all in rustc #[cfg(parallel_compiler)], and I think a lot of rustc-rayon's customization would be easier under a more limited API.

I am currently working on this and other parallelization issues.

[zetanumbers]

By increasing the thread count we can increase the work stealing done and thus reduce the regression

@Zoxc wouldn't that mess up with the jobserver? It raises the limit of number cores used in parallel.

[Zoxc]

I'm starting to think that we would be better off with a bespoke rustc_threadpool that doesn't try to be a modified rayon.

Yeah, this does seem to imply a move away from tracking Rayon. This approach doesn't match well with Rayon iterators and we may want to introduce fibers too. I'd like to see some more extensive testing to verify this solves the deadlock issues before moving further in this direction though.

@Zoxc wouldn't that mess up with the jobserver? It raises the limit of number cores used in parallel.

Not sure what you mean here? The jobserver would still limit active threads across cores as before.

[zetanumbers]

I should have done it much earlier, but I endorse this PR. Merging it would allow to make progress on parallelizing rustc code and stabilizing parallel front-end. Doubling number of threads seem like a worth enough compromise.

More details: https://rust-lang.zulipchat.com/#narrow/channel/187679-t-compiler.2Fwg-parallel-rustc/topic/Deadlocks.20and.20Rayon/near/490996600

[oli-obk]

There's a lot of tests that are now ignored. Either change them to the new behaviour or leave a comment on them why they are ignored and why their behaviour is now nonsensical. Or I guess remove them if the tests only ever tested work stealing from all threads.

[oli-obk]

use expose_provenance instead to clearly signal intent

[oli-obk]

or use an AtomicPtr instead of an AtomicUsize for the latch. That should work, too, right?

[oli-obk]

preexisting, but it is very confusing that this entire function body is unsafe. Let's move to a new edition once the crate is in tree

[oli-obk]

This avoids situations where we steal work we can't complete due to waiting on queries. This should fix bugs like rust-lang/rust#111521 and rust-lang/rust#115223, but it does have a negative performance impact.

kind of a big hammer for this problem. But I can't think of anything robust otherwise