Skip to content

Improve SLURM support to address individual tasks #227

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
satra opened this issue Apr 2, 2020 · 1 comment
Open

Improve SLURM support to address individual tasks #227

satra opened this issue Apr 2, 2020 · 1 comment
Labels
enhancement New feature or request

Comments

@satra
Copy link
Contributor

satra commented Apr 2, 2020

SLURM support for options is rather limited. so switching partitions or requesting hardware resources (e.g., GPUs) at a task specific basis is not implemented.

this would relate to #34 in terms of requirements. a task should be able to say things about its resource requirements: storage, network, gpu, cpu, memory (using the CWL specs). and then the worker should have a way of mapping those requirements to potential slurm configurations of a given cluster.

@satra satra added the enhancement New feature or request label Apr 2, 2020
@satra satra added this to the 0.5 May Day milestone Apr 2, 2020
@djarecka djarecka modified the milestones: 0.5 May Day, 0.7 - julius May 4, 2020
@satra satra modified the milestones: 0.7 - julius , 0.8 - agosto Jul 8, 2020
@djarecka djarecka modified the milestones: 0.8 - agosto, outubro Sep 10, 2020
@tclose tclose added the to consider suggesting changes that require more discussion label Apr 1, 2025
@tclose tclose moved this to Proposed in Pydra Roadmap Apr 1, 2025
@tclose tclose removed the to consider suggesting changes that require more discussion label Apr 10, 2025
@tclose tclose moved this from Proposed to temp in Pydra Roadmap Apr 29, 2025
@tclose tclose moved this from temp to To do (v1.0) in Pydra Roadmap Apr 29, 2025
@satra
Copy link
Contributor Author

satra commented May 6, 2025

@tclose - this nipype issue (nipy/nipype#3726) reminded me of this. it would be nice to have some tutorials touch on this issue of different requirements for different tasks, and how that interacts with the schedulers and environments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Triage
Development

No branches or pull requests

3 participants