|
| 1 | +--- |
| 2 | +layout: blog |
| 3 | +title: 'APIServer dry-run and kubectl diff' |
| 4 | +date: 2019-01-14 |
| 5 | +--- |
| 6 | + |
| 7 | +**Author**: Antoine Pelisse (Google Cloud, @apelisse) |
| 8 | + |
| 9 | +Declarative configuration management, also known as configuration-as-code, is |
| 10 | +one of the key strengths of Kubernetes. It allows users to commit the desired state of |
| 11 | +the cluster, and to keep track of the different versions, improve auditing and |
| 12 | +automation through CI/CD pipelines. The [Apply working-group](https://groups.google.com/forum/#!forum/kubernetes-wg-apply) |
| 13 | +is working on fixing some of the gaps, and is happy to announce that Kubernetes |
| 14 | +1.13 promoted server-side dry-run and `kubectl diff` to beta. These |
| 15 | +two features are big improvements for the Kubernetes declarative model. |
| 16 | + |
| 17 | +## Challenges |
| 18 | + |
| 19 | +A few pieces are still missing in order to have a seamless declarative |
| 20 | +experience with Kubernetes, and we tried to address some of these: |
| 21 | + |
| 22 | +- While compilers and linters do a good job to detect errors in pull-requests |
| 23 | + for code, a good validation is missing for Kubernetes configuration files. |
| 24 | + The existing solution is to run `kubectl apply --dry-run`, but this runs a |
| 25 | + *local* dry-run that doesn't talk to the server: it doesn't have server |
| 26 | + validation and doesn't go through validating admission controllers. As an |
| 27 | + example, Custom resource names are only validated on the server so a local |
| 28 | + dry-run won't help. |
| 29 | +- It can be difficult to know how your object is going to be applied by the |
| 30 | + server for multiple reasons: |
| 31 | + - Defaulting will set some fields to potentially unexpected values, |
| 32 | + - Mutating webhooks might set fields or clobber/change some values. |
| 33 | + - Patch and merges can have surprising effects and result in unexpected |
| 34 | + objects. For example, it can be hard to know how lists are going to be |
| 35 | + ordered once merged. |
| 36 | + |
| 37 | +The working group has tried to address these problems. |
| 38 | + |
| 39 | +## APIServer dry-run |
| 40 | + |
| 41 | +[APIServer dry-run](https://kubernetes.io/docs/reference/using-api/api-concepts/#dry-run) was implemented to address these two problems: |
| 42 | + |
| 43 | +- it allows individual requests to the apiserver to be marked as "dry-run", |
| 44 | +- the apiserver guarantees that dry-run requests won't be persisted to storage, |
| 45 | +- the request is still processed as typical request: the fields are |
| 46 | + defaulted, the object is validated, it goes through the validation admission |
| 47 | + chain, and through the mutating admission chain, and then the final object is |
| 48 | + returned to the user as it normally would, without being persisted. |
| 49 | + |
| 50 | +While dynamic admission controllers are not supposed to have side-effects on |
| 51 | +each request, dry-run requests are only processed if all admission controllers |
| 52 | +explicitly announce that they don't have any dry-run side-effects. |
| 53 | + |
| 54 | +### How to enable it |
| 55 | + |
| 56 | +Server-side dry-run is enabled through a feature-gate. Now that the feature is |
| 57 | +Beta in 1.13, it should be enabled by default, but still can be enabled/disabled |
| 58 | +using `kube-apiserver --feature-gates DryRun=true`. |
| 59 | + |
| 60 | +If you have dynamic admission controllers, you might have to fix them to: |
| 61 | + |
| 62 | +- Remove any side-effects when the dry-run parameter is specified on the webhook request, |
| 63 | +- Specify in the [`sideEffects`](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/#webhook-v1beta1-admissionregistration) |
| 64 | +field of the `admissionregistration.k8s.io/v1beta1.Webhook` object to indicate that the object doesn't |
| 65 | +have side-effects on dry-run (or at all). |
| 66 | + |
| 67 | +### How to use it |
| 68 | + |
| 69 | +You can trigger the feature from kubectl by using `kubectl apply |
| 70 | +--server-dry-run`, which will decorate the request with the dryRun flag |
| 71 | +and return the object as it would have been applied, or an error if it would |
| 72 | +have failed. |
| 73 | + |
| 74 | +## Kubectl diff |
| 75 | + |
| 76 | +APIServer dry-run is convenient because it lets you see how the object would be |
| 77 | +processed, but it can be hard to identify exactly what changed if the object is |
| 78 | +big. `kubectl diff` does exactly what you want by showing the differences between |
| 79 | +the current "live" object and the new "dry-run" object. It makes it very |
| 80 | +convenient to focus on only the changes that are made to the object, how the |
| 81 | +server has merged these and how the mutating webhooks affects the output. |
| 82 | + |
| 83 | +### How to use it |
| 84 | + |
| 85 | +`kubectl diff` is meant to be as similar as possible to `kubectl apply`: |
| 86 | +`kubectl diff -f some-resources.yaml` will show a diff for the resources in the yaml file. One can even use the diff program of their choice by using the KUBECTL_EXTERNAL_DIFF environment variable, for example: |
| 87 | +``` |
| 88 | +KUBECTL_EXTERNAL_DIFF=meld kubectl diff -f some-resources.yaml |
| 89 | +``` |
| 90 | + |
| 91 | +## What's next |
| 92 | + |
| 93 | +The working group is still busy trying to improve some of these things: |
| 94 | + |
| 95 | +- Server-side apply is trying to improve the apply scenario, by adding owner |
| 96 | +semantics to fields! It's also going to improve support for CRDs and unions! |
| 97 | +- Some kubectl apply features are missing from diff and could be useful, like the ability |
| 98 | +to filter by label, or to display pruned resources. |
| 99 | +- Eventually, kubectl diff will use server-side apply! |
0 commit comments