Skip to content

Commit 778497b

Browse files
apelissek8s-ci-robot
authored andcommitted
Create apiserver-dry-run-and-kubectl-diff (kubernetes#12135)
* Create apiserver-dry-run-and-kubectl-diff * Update apiserver-dry-run-and-kubectl-diff * Update apiserver-dry-run-and-kubectl-diff * Rename apiserver-dry-run-and-kubectl-diff to apiserver-dry-run-and-kubectl-diff.md * Update apiserver-dry-run-and-kubectl-diff.md
1 parent ab2918b commit 778497b

File tree

1 file changed

+99
-0
lines changed

1 file changed

+99
-0
lines changed
Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
---
2+
layout: blog
3+
title: 'APIServer dry-run and kubectl diff'
4+
date: 2019-01-14
5+
---
6+
7+
**Author**: Antoine Pelisse (Google Cloud, @apelisse)
8+
9+
Declarative configuration management, also known as configuration-as-code, is
10+
one of the key strengths of Kubernetes. It allows users to commit the desired state of
11+
the cluster, and to keep track of the different versions, improve auditing and
12+
automation through CI/CD pipelines. The [Apply working-group](https://groups.google.com/forum/#!forum/kubernetes-wg-apply)
13+
is working on fixing some of the gaps, and is happy to announce that Kubernetes
14+
1.13 promoted server-side dry-run and `kubectl diff` to beta. These
15+
two features are big improvements for the Kubernetes declarative model.
16+
17+
## Challenges
18+
19+
A few pieces are still missing in order to have a seamless declarative
20+
experience with Kubernetes, and we tried to address some of these:
21+
22+
- While compilers and linters do a good job to detect errors in pull-requests
23+
for code, a good validation is missing for Kubernetes configuration files.
24+
The existing solution is to run `kubectl apply --dry-run`, but this runs a
25+
*local* dry-run that doesn't talk to the server: it doesn't have server
26+
validation and doesn't go through validating admission controllers. As an
27+
example, Custom resource names are only validated on the server so a local
28+
dry-run won't help.
29+
- It can be difficult to know how your object is going to be applied by the
30+
server for multiple reasons:
31+
- Defaulting will set some fields to potentially unexpected values,
32+
- Mutating webhooks might set fields or clobber/change some values.
33+
- Patch and merges can have surprising effects and result in unexpected
34+
objects. For example, it can be hard to know how lists are going to be
35+
ordered once merged.
36+
37+
The working group has tried to address these problems.
38+
39+
## APIServer dry-run
40+
41+
[APIServer dry-run](https://kubernetes.io/docs/reference/using-api/api-concepts/#dry-run) was implemented to address these two problems:
42+
43+
- it allows individual requests to the apiserver to be marked as "dry-run",
44+
- the apiserver guarantees that dry-run requests won't be persisted to storage,
45+
- the request is still processed as typical request: the fields are
46+
defaulted, the object is validated, it goes through the validation admission
47+
chain, and through the mutating admission chain, and then the final object is
48+
returned to the user as it normally would, without being persisted.
49+
50+
While dynamic admission controllers are not supposed to have side-effects on
51+
each request, dry-run requests are only processed if all admission controllers
52+
explicitly announce that they don't have any dry-run side-effects.
53+
54+
### How to enable it
55+
56+
Server-side dry-run is enabled through a feature-gate. Now that the feature is
57+
Beta in 1.13, it should be enabled by default, but still can be enabled/disabled
58+
using `kube-apiserver --feature-gates DryRun=true`.
59+
60+
If you have dynamic admission controllers, you might have to fix them to:
61+
62+
- Remove any side-effects when the dry-run parameter is specified on the webhook request,
63+
- Specify in the [`sideEffects`](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/#webhook-v1beta1-admissionregistration)
64+
field of the `admissionregistration.k8s.io/v1beta1.Webhook` object to indicate that the object doesn't
65+
have side-effects on dry-run (or at all).
66+
67+
### How to use it
68+
69+
You can trigger the feature from kubectl by using `kubectl apply
70+
--server-dry-run`, which will decorate the request with the dryRun flag
71+
and return the object as it would have been applied, or an error if it would
72+
have failed.
73+
74+
## Kubectl diff
75+
76+
APIServer dry-run is convenient because it lets you see how the object would be
77+
processed, but it can be hard to identify exactly what changed if the object is
78+
big. `kubectl diff` does exactly what you want by showing the differences between
79+
the current "live" object and the new "dry-run" object. It makes it very
80+
convenient to focus on only the changes that are made to the object, how the
81+
server has merged these and how the mutating webhooks affects the output.
82+
83+
### How to use it
84+
85+
`kubectl diff` is meant to be as similar as possible to `kubectl apply`:
86+
`kubectl diff -f some-resources.yaml` will show a diff for the resources in the yaml file. One can even use the diff program of their choice by using the KUBECTL_EXTERNAL_DIFF environment variable, for example:
87+
```
88+
KUBECTL_EXTERNAL_DIFF=meld kubectl diff -f some-resources.yaml
89+
```
90+
91+
## What's next
92+
93+
The working group is still busy trying to improve some of these things:
94+
95+
- Server-side apply is trying to improve the apply scenario, by adding owner
96+
semantics to fields! It's also going to improve support for CRDs and unions!
97+
- Some kubectl apply features are missing from diff and could be useful, like the ability
98+
to filter by label, or to display pruned resources.
99+
- Eventually, kubectl diff will use server-side apply!

0 commit comments

Comments
 (0)