Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cluster-api: node template in scale-from-0-nodes scenario with DRA #7724

Open
ttsuuubasa opened this issue Jan 20, 2025 · 0 comments
Open

cluster-api: node template in scale-from-0-nodes scenario with DRA #7724

ttsuuubasa opened this issue Jan 20, 2025 · 0 comments
Labels
area/cluster-autoscaler kind/feature Categorizes issue or PR as related to a new feature.

Comments

@ttsuuubasa
Copy link

Which component are you using?:

/area cluster-autoscaler

Is your feature request designed to solve a problem? If so describe the problem this feature should solve.:

Cluster Autoscaler was extended significantly for DRA and there are parts that cluster-api also needs to address as Cloud Provider.

Cluster Autoscaler has a way to scale-up that called "scale-from-0-nodes" scenario where there are no existing Nodes in NodeGroup and then a new Node spawns from there.
In this case, each Cloud Provider is responsible for providing a template node (NodeInfo) by TemplateNodeInfo() and it has the resource information of the node like CPU, memory and GPU.

At the age of Device Plugin, cluster-api provided the node template by using Annotation added to NodeGroup such as MachineSet and MachineDeployment.[1]
However, for DRA, cluster-api has not yet implemented the logic to create the template of ResourceSlice from this information at this point.

For example, when users want to spawn the node with GPU as DRA resources in NodeGroup where there is no existing node. Although it's for a pending pod that requires GPU with ResourceClaim, they could have no option to execute it and it is likely not to work.

Therefore, cluster-api needs the feature for users to specify devices in ResourceSlice of the node to be spawned in scale-from-0-nodes scenario.

[1] https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/clusterapi/README.md#scale-from-zero-support

Describe the solution you'd like.:

The simplest idea is adding the more annotations into the NodeGroup to be the basis of ResourceSlice like the following.

capacity.cluster-autoscaler.kubernetes.io/dra-driver: gpu.nvidia.com
capacity.cluster-autoscaler.kubernetes.io/dra-pool  : <pool-name>
@ttsuuubasa ttsuuubasa added the kind/feature Categorizes issue or PR as related to a new feature. label Jan 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/cluster-autoscaler kind/feature Categorizes issue or PR as related to a new feature.
Projects
None yet
Development

No branches or pull requests

2 participants