Skip to content

Allow running without worker nodes #17544

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Allow running without worker nodes #17544

wants to merge 2 commits into from
+139 −17

Conversation

hakman
Copy link
Member

@hakman hakman commented Aug 7, 2025

/cc @justinsb

@k8s-ci-robot k8s-ci-robot requested a review from justinsb August 7, 2025 20:32
@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. area/addons area/api labels Aug 7, 2025
@hakman hakman force-pushed the zero-nodes branch 2 times, most recently from 6bf9c24 to d349606 Compare August 8, 2025 04:08
@hakman
Copy link
Member Author

hakman commented Aug 8, 2025

/retest

@hakman
Copy link
Member Author

hakman commented Aug 8, 2025

Addons test is failing due to bug in upstream: kubernetes/kubernetes#133405
/override pull-kops-e2e-aws-upgrade-k132-ko132-to-klatest-kolatest-many-addons

@k8s-ci-robot
Copy link
Contributor

@hakman: Overrode contexts on behalf of hakman: pull-kops-e2e-aws-upgrade-k132-ko132-to-klatest-kolatest-many-addons

In response to this:

Addons test is failing due to bug in upstream: kubernetes/kubernetes#133405
/override pull-kops-e2e-aws-upgrade-k132-ko132-to-klatest-kolatest-many-addons

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@hakman hakman changed the title [WIP] Allow running with 0 worker nodes Allow running with 0 worker nodes Aug 8, 2025
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 8, 2025
@hakman
Copy link
Member Author

hakman commented Aug 8, 2025

/hold for feedback

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 8, 2025
@hakman
Copy link
Member Author

hakman commented Aug 8, 2025

/retest

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 8, 2025
@hakman hakman changed the title Allow running with 0 worker nodes Allow running without worker nodes Aug 8, 2025
@hakman
Copy link
Member Author

hakman commented Aug 9, 2025

/retest

2 similar comments
@hakman
Copy link
Member Author

hakman commented Aug 9, 2025

/retest

@hakman
Copy link
Member Author

hakman commented Aug 10, 2025

/retest

@hakman
Copy link
Member Author

hakman commented Aug 12, 2025

/assign @justinsb

justinsb
justinsb reviewed Aug 14, 2025
View reviewed changes
pkg/apis/kops/validation/legacy.go
@@ -274,10 +274,6 @@ func DeepValidate(c *kops.Cluster, groups []*kops.InstanceGroup, strict bool, vf
return fmt.Errorf("must configure at least one ControlPlane InstanceGroup")
}

if nodeGroupCount == 0 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to feature flag this? I'm OK with not doing so...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure, I think should be pretty safe without too, as one has to explicitly specify --node-count=0.

upup/models/cloudup/resources/addons/coredns.addons.k8s.io/k8s-1.12.yaml.template
@@ -138,7 +138,7 @@ spec:
- key: "CriticalAddonsOnly"
operator: "Exists"
{{- end }}
{{- if KarpenterEnabled }}
{{- if or IsControlPlaneOnly KarpenterEnabled }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want a helper like CoreDNSOnControlPlane , but not a big deal

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was hoping to reuse this, it has nothing to do with CoreDNS specifically.

upup/pkg/fi/cloudup/new_cluster.go Outdated
nodes, err = setupNodes(opt, cluster, zoneToSubnetsMap)
if err != nil {
return nil, err
if opt.NodeCount > 0 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry - why do we want to guard behind opt.NodeCount > 0 ? Can we add a comment?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved a little further in the code and added comment.
If the number of nodes is 0, there's nothing to do. No instance group should be created for worker nodes.

upup/pkg/fi/cloudup/new_cluster.go Outdated
@@ -176,6 +176,7 @@ func (o *NewClusterOptions) InitDefaults() {
o.AdminAccess = []string{"0.0.0.0/0", "::/0"}
o.EtcdClusters = []string{"main", "events"}
o.Networking = "cilium"
o.NodeCount = 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should instead look at whether the flag is specified. It's weird to have a default of one, but then if you specify three zones it actually defaults to 3

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, was not sure it's possible.
Done.

@hakman hakman requested a review from justinsb August 15, 2025 21:24
@k8s-ci-robot
Copy link
Contributor

@hakman: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kops-e2e-aws-upgrade-k132-ko132-to-klatest-kolatest-many-addons 84de39f link false /test pull-kops-e2e-aws-upgrade-k132-ko132-to-klatest-kolatest-many-addons

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@hakman
Copy link
Member Author

hakman commented Aug 16, 2025

/unhold

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 16, 2025
@hakman
Copy link
Member Author

hakman commented Aug 17, 2025

/hold

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 17, 2025
@justinsb
Copy link
Member

/approve
/lgtm

Awesome stuff!

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 27, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: justinsb

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 27, 2025
@hakman
Copy link
Member Author

hakman commented Aug 27, 2025

@justinsb let's discuss this on Thursday, I think #17567 may be closer to what we want.

@hakman hakman mentioned this pull request Aug 29, 2025
Merged
@hakman hakman removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 6, 2025
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 6, 2025
@k8s-ci-robot
Copy link
Contributor

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@justinsb justinsb Awaiting requested review from justinsb

Assignees

@justinsb justinsb

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/addons area/api area/documentation cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@hakman @k8s-ci-robot @justinsb