Skip to content

Commit a12493e

Browse files
tssuryatssurya
committed
NPEP: Iron out Egress Support API Design
Signed-off-by: Surya Seetharaman <[email protected]>
1 parent 95ae2f8 commit a12493e

File tree

1 file changed

+175
-4
lines changed

1 file changed

+175
-4
lines changed

‎npeps/npep-126-egress-traffic-control.md

+175-4
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# NPEP-126: Add northbound traffic support in (B)ANP API
22

33
* Issue: [#126](https://github.com/kubernetes-sigs/network-policy-api/issues/126)
4-
* Status: Provisional
4+
* Status: Implementable
55

66
## TLDR
77

@@ -76,13 +76,184 @@ selected cluster workloads to k8s-apiservers for securing the server.
7676

7777
## API
7878

79-
(... details, can point to PR with changes)
79+
Proof of Concept for the API design details can be found here:
8080

81+
* https://github.com/kubernetes-sigs/network-policy-api/pull/143
82+
* https://github.com/kubernetes-sigs/network-policy-api/pull/185
83+
84+
### Implementing egress traffic control towards cluster nodes
85+
86+
This NPEP proposes to add a new type of `AdminNetworkPolicyEgressPeer` called `Nodes`
87+
to be able to explicitly select nodes (based on the node's labels) in the cluster.
88+
This ensures that if the list of IPs on a node OR list of nodes change, the users
89+
don't need to manually intervene to include those new IPs. The label selectors will
90+
take care of this automatically. Note that the nodeIPs that this type of peer matches
91+
on are the IPs present in `Node.Status.Addresses` field of the node.
92+
93+
```
94+
// AdminNetworkPolicyEgressPeer defines a peer to allow traffic to.
95+
// Exactly one of the selector pointers must be set for a given peer. If a
96+
// consumer observes none of its fields are set, they must assume an unknown
97+
// option has been specified and fail closed.
98+
// +kubebuilder:validation:MaxProperties=1
99+
// +kubebuilder:validation:MinProperties=1
100+
type AdminNetworkPolicyEgressPeer struct {
101+
<snipped>
102+
// Nodes defines a way to select a set of nodes in
103+
// in the cluster. This field follows standard label selector
104+
// semantics; if present but empty, it selects all Nodes.
105+
//
106+
// Support: Core
107+
//
108+
// +optional
109+
Nodes *metav1.LabelSelector `json:"nodes,omitempty"`
110+
}
111+
```
112+
113+
Note that `AdminNetworkPolicyPeer` will be changed to
114+
`AdminNetworkPolicyEgressPeer` and `AdminNetworkPolicyIngressPeer` since ingress and
115+
egress peers have started to diverge at this point and it is easy to
116+
maintain it with two sets of peer definitions.
117+
This ensures nodes can be referred to only as "egress peers".
118+
119+
Example: Admin wants to deny egress traffic from tenants who don't have
120+
`restricted`, `confidential` or `internal` level security clearance
121+
to control-plane nodes at 443 and 6443 ports in the cluster
122+
123+
```
124+
apiVersion: policy.networking.k8s.io/v1alpha1
125+
kind: AdminNetworkPolicy
126+
metadata:
127+
name: node-as-egress-peers
128+
spec:
129+
priority: 55
130+
subject:
131+
namespaces:
132+
matchExpressions:
133+
- {key: security, operator: notIn, values: [restricted, confidential, internal]}
134+
egress:
135+
- name: "deny-all-egress-to-kapi-server"
136+
action: "Deny"
137+
to:
138+
- nodes:
139+
matchLabels:
140+
node-role.kubernetes.io/control-plane:
141+
ports:
142+
- portNumber:
143+
protocol: TCP
144+
port: 443
145+
- portNumber:
146+
protocol: TCP
147+
port: 6443
148+
```
149+
150+
### Implementing egress traffic control towards external destinations
151+
152+
This NPEP proposes to add a new type of `AdminNetworkPolicyEgressPeer` called `Networks`
153+
to be able to explicitly select external destination CIDRs outside the cluster.
154+
This peer type will not be supported in `AdminNetworkPolicyIngressPeer`.
155+
156+
```
157+
// AdminNetworkPolicyEgressPeer defines a peer to allow traffic to.
158+
// Exactly one of the selector pointers must be set for a given peer. If a
159+
// consumer observes none of its fields are set, they must assume an unknown
160+
// option has been specified and fail closed.
161+
// +kubebuilder:validation:MaxProperties=1
162+
// +kubebuilder:validation:MinProperties=1
163+
type AdminNetworkPolicyEgressPeer struct {
164+
<snipped>
165+
// Networks defines a way to select IPs and CIDR blocks that represent
166+
// entities that live outside the cluster as a peer.
167+
// It is the list of NetworkCIDR (both v4 & v6) that can be used to define
168+
// external destinations.
169+
// You could use this to select internal cluster networks like podCIDR blocks, but it is
170+
// recommended to use namespaces and pods peers instead in those cases.
171+
//
172+
// Support: Core
173+
//
174+
// +optional
175+
// +kubebuilder:validation:MinItems=1
176+
// +kubebuilder:validation:MaxItems=100
177+
Networks []string `json:"networks,omitempty" validate:"omitempty,dive,cidr"`
178+
}
179+
```
180+
181+
Note that the API expects `networks` to be a set of "external" destinations.
182+
However if user puts a podCIDR, nodeCIDR, serviceCIDR or other intra-cluster
183+
networks, it will work but it is recommended to use namespaces, pods, nodes
184+
peers to express such entities. Not all implementations can correctly define
185+
the boundary between "internal" and "external" destinations with respect to a
186+
Kubernetes cluster.
187+
188+
Let's define ANP and BANP that refer to these external networks:
189+
```
190+
apiVersion: policy.networking.k8s.io/v1alpha1
191+
kind: AdminNetworkPolicy
192+
metadata:
193+
name: node-as-egress-peers
194+
spec:
195+
priority: 70
196+
subject:
197+
namespaces: {}
198+
egress:
199+
- name: "allow-all-egress-to-intranet"
200+
action: "Allow"
201+
to:
202+
- networks:
203+
- 192.0.2.0/24
204+
- 203.0.113.0/24
205+
- 198.51.100.0/24
206+
- name: "allow-egress-to-external-dns"
207+
action: "Allow"
208+
to:
209+
- networks:
210+
- 194.0.2.0/24
211+
- 205.0.113.15/32
212+
- 199.51.100.10/32
213+
ports:
214+
- portNumber:
215+
protocol: UDP
216+
port: 53
217+
- name: "pass-egress-to-internet"
218+
action: "Pass"
219+
to:
220+
- networks:
221+
- 0.0.0.0/0
222+
---
223+
apiVersion: policy.networking.k8s.io/v1alpha1
224+
kind: BaselineAdminNetworkPolicy
225+
metadata:
226+
name: default
227+
spec:
228+
subject:
229+
namespaces: {}
230+
egress:
231+
- name: "deny-egress-to-internet"
232+
action: "Deny"
233+
to:
234+
- networks:
235+
- 0.0.0.0/0
236+
```
237+
This allows admins to specify all cluster workloads can talk to
238+
the company's internet CIDR's and the external DNS network. It
239+
also allows them to put up default guardrails of not allowing
240+
any other egress traffic towards internet in a BANP.
81241

82242
## Alternatives
83243

84-
(List other design alternatives and why we did not go in that
85-
direction)
244+
* Instead of adding CIDR peer directly into the main object, we can
245+
define a new object called `ExternalNetworkSet` and use selectors or
246+
name of that object to be referred to from AdminNetworkPolicy and
247+
BaselineAdminNetworkPolicy objects. This is particularly useful
248+
if CIDR ranges are prone to changes versus the current model is
249+
is better if the set of CIDRs are mostly a constant and are only referred
250+
to from one or two egress rules. It increases readability. However the
251+
drawback is if the CIDRs do change, then one has to ensure to update all
252+
the relevant ANPs and BANP accordingly. In order to see whether we need
253+
a new object to be able to define CIDRs in addition to the in-line peer,
254+
we have another NPEP where that is being discussed
255+
https://github.com/kubernetes-sigs/network-policy-api/pull/183. The scope
256+
of this NPEP is limited to inline CIDR peers.
86257

87258
## References
88259

0 commit comments

Comments
 (0)