Skip to content

take into account the num of processes by ulimit #143614

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

klim-iv
Copy link

@klim-iv klim-iv commented Jul 7, 2025

No description provided.

@rustbot
Copy link
Collaborator

rustbot commented Jul 7, 2025

r? @tgross35

rustbot has assigned @tgross35.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added O-unix Operating system: Unix-like S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Jul 7, 2025
@klim-iv
Copy link
Author

klim-iv commented Jul 7, 2025

due to ulimit restrictions is not used in std::thread
apears crashes like this one:
thread 'main' panicked at .../.cargo/registry/.../rayon-core-1.12.1/src/registry.rs:168:10:
The global thread pool has not been initialized.: ThreadPoolBuildError
 { kind: IOError(Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }) }

@@ -440,13 +440,24 @@ pub fn available_parallelism() -> io::Result<NonZero<usize>> {
}
}
}

let mut ulimit = u64::MAX;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The type is rlim_t, this won't work on x86-32

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment on lines 445 to 446
#[cfg(any(target_os = "android", target_os = "linux"))]
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The above block has the same cfg, this can be combined

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@tgross35
Copy link
Contributor

tgross35 commented Jul 7, 2025

Could you please put a more detailed description of how the problem is reproduced in the PR description? Or open a separate issue for it.

The change here also needs comments, and the docs at https://doc.rust-lang.org/beta/std/thread/fn.available_parallelism.html should be updated if this winds up being accepted.

Since this is a user-visible change this will need libs-api signoff, but first cc @cuviper since it looks like the issue shows up in Rayon (possibly rayon-rs/rayon#694?)

@rust-log-analyzer

This comment has been minimized.

@cuviper
Copy link
Member

cuviper commented Jul 7, 2025

I doubt that this will be useful -- e.g. starting a number of threads equal to the current RLIMIT_NPROC is already doomed by the existence of the current thread, nevermind any other threads or even other processes.

let mut r: libc::rlimit = unsafe { mem::zeroed() };
unsafe {
if libc::getrlimit(libc::RLIMIT_NPROC, &mut r) == 0 {
ulimit = r.rlim_max
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note, the "soft" rlim_cur is the value that is currently enforced, while the "hard" rlim_max is a limit on how high an unprivileged process can adjust its own rlim_cur.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@klim-iv
Copy link
Author

klim-iv commented Jul 8, 2025

I doubt that this will be useful -- e.g. starting a number of threads equal to the current RLIMIT_NPROC is already doomed by the existence of the current thread, nevermind any other threads or even other processes.

I see, patchset updated according this comment (will count working threads)

due to ulimit restrictions is not used in std::thread
upears crashed like this one:
thread 'main' panicked at .../.cargo/registry/.../rayon-core-1.12.1/src/registry.rs:168:10:
The global thread pool has not been initialized.: ThreadPoolBuildError
 { kind: IOError(Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }) }

resolve #143635
@klim-iv
Copy link
Author

klim-iv commented Jul 8, 2025

Could you please put a more detailed description of how the problem is reproduced in the PR description? Or open a separate issue for it.

The change here also needs comments, and the docs at https://doc.rust-lang.org/beta/std/thread/fn.available_parallelism.html should be updated if this winds up being accepted.

Since this is a user-visible change this will need libs-api signoff, but first cc @cuviper since it looks like the issue shows up in Rayon (possibly rayon-rs/rayon#694?)

bug here: rust-lang/rustup#4403

@rust-log-analyzer
Copy link
Collaborator

The job pr-check-1 failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)
[RUSTC-TIMING] object test:false 9.953
error: variable does not need to be mutable
   --> library/std/src/sys/pal/unix/thread.rs:458:17
    |
458 |             let mut ulimit = libc::rlim_t::MAX;
    |                 ----^^^^^^
    |                 |
    |                 help: remove this `mut`
    |
    = note: `-D unused-mut` implied by `-D warnings`
    = help: to override `-D warnings` add `#[allow(unused_mut)]`

error: function `count_user_threads` is never used
   --> library/std/src/sys/pal/unix/thread.rs:408:4
    |
408 | fn count_user_threads() -> Result<usize, io::Error> {
    |    ^^^^^^^^^^^^^^^^^^
    |
    = note: `-D dead-code` implied by `-D warnings`
    = help: to override `-D warnings` add `#[allow(dead_code)]`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
O-unix Operating system: Unix-like S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants