|
| 1 | +--- |
| 2 | +layout: post |
| 3 | +title: Cert Manager with ACME |
| 4 | +subtitle: Issuing certificates in OpenShift with Let's Encrypt |
| 5 | +tags: [cert-manager, openshift, k8s, acme, ssl, certificate, wildcard, letsencrypt, dns-server, tls-certificate, acme-challenge, acme-dns] |
| 6 | +author: cmeissner |
| 7 | +--- |
| 8 | + |
| 9 | +If you install an OCP cluster certificates for API and the wildcard domain will be signed by a self-signed CA. |
| 10 | +This is not a security but a convenience issue as you need to accept or ignore warnings regarding self-signed certificates. |
| 11 | + |
| 12 | +Replacing the certificates by ones signed by a known certificate authority is [well documented](https://access.redhat.com/documentation/en-us/openshift_container_platform/4.14/html/security_and_compliance/configuring-certificates){:target="_blank"} and works well. |
| 13 | +If you want to use Let's Encrypt as the CA of your choice, you can request certificates there and provide the authorization information on your own, but since the availability of [cert-manager Operator for Red Hat OpenShift](https://access.redhat.com/documentation/en-us/openshift_container_platform/4.14/html/security_and_compliance/cert-manager-operator-for-red-hat-openshift){:target="_blank"} tools for managing certificates automatically available. |
| 14 | + |
| 15 | +## Preparing for dns-01 challenge |
| 16 | + |
| 17 | +If you have a working [acme-dns](/2023/12/01/wildcard_certs_with_acme/), it can be used to issue both host and wildcard certificates with cert-manager and Let's Encrypt. |
| 18 | + |
| 19 | +1. Register an account with your acme-dns service and save the account data to a file |
| 20 | + |
| 21 | + ```shell |
| 22 | + $ curl -XPOST https://auth.example.com/register | jq > acmedns.json |
| 23 | + $ curl -s -XPOST https://auth.example.com/update \ |
| 24 | + -H "X-Api-User: <username>" \ |
| 25 | + -H "X-Api-Key: <password>" \ |
| 26 | + --data '{"subdomain": "<subdomain>", "txt": "___validation_token_received_from_the_ca___"}' | jq |
| 27 | + { |
| 28 | + "txt": "___validation_token_received_from_the_ca___" |
| 29 | + } |
| 30 | + ``` |
| 31 | + |
| 32 | + Replace the data between square brackets with the data from your registration. |
| 33 | +2. Configure your DNS zone to forward dns-01 challenge requests to your acme-dns service |
| 34 | + |
| 35 | + ```shell |
| 36 | + $ dig +noall +answer -t CNAME _acme-challenge.apps.ocp4.example.com @9.9.9.9 |
| 37 | + _acme-challenge.apps.ocp4.example.com. 50 IN CNAME <subdomain>.auth.example.com. |
| 38 | + |
| 39 | + ``` |
| 40 | + |
| 41 | + Test whether a TXT record will be returned. |
| 42 | + |
| 43 | + ```shell |
| 44 | + $ dig +noall +answer -t TXT _acme-challenge.apps.ocp4.example.com @9.9.9.9 |
| 45 | + _acme-challenge.apps.ocp4.example.com. 50 IN CNAME <subdomain>.auth.example.com. |
| 46 | + <subdomain>.auth.example.com. 1 IN TXT "___validation_token_received_from_the_ca___" |
| 47 | + ``` |
| 48 | + |
| 49 | +With these steps, the DNS setup is prepared for handling the dns-01 challenges. |
| 50 | + |
| 51 | +## cert-manager Operator |
| 52 | + |
| 53 | +As the DNS side of the setup has finished, it is now time to install the `cert-manager Operator for Red Hat OpenShift`. |
| 54 | + |
| 55 | +1. Open the OpenShift web console. You need to log in as a cluster admin. |
| 56 | +2. Navigate to the OperatorHub. Operator → OperatorHub |
| 57 | +3. Search for `cert-manager`. |
| 58 | +4. Select the `cert-manager Operator for Red Hat OpenShift` and click Install. In the upcoming wizard, leave all values on its defaults and click Install. |
| 59 | + |
| 60 | +### Configure cert-manager |
| 61 | + |
| 62 | +After installing the cert-manager Operator successfully, it is time to prepare it for issuing Let's Encrypt certificates. To do so, we need |
| 63 | + |
| 64 | +1. Save the credentials json snippet from the registration process to a file with a key for all your domain that should be handled by the `ClusterIssuer`. |
| 65 | + |
| 66 | + As we want later replace the API and wildcard certificate with newly cert-manager created ones, the file should look like this. |
| 67 | + |
| 68 | + ```json |
| 69 | + { |
| 70 | + "api.ocp4.example.com": { |
| 71 | + "username": "<username>", |
| 72 | + "password": "<password>", |
| 73 | + "fulldomain": "<subdomain>.acme-dns.adsfg.xyz", |
| 74 | + "subdomain": "<subdomain>", |
| 75 | + "allowfrom": [] |
| 76 | + }, |
| 77 | + "apps.ocp4.example.com": { |
| 78 | + "username": "<username>", |
| 79 | + "password": "<password>", |
| 80 | + "fulldomain": "<subdomain>.acme-dns.adsfg.xyz", |
| 81 | + "subdomain": "<subdomain>", |
| 82 | + "allowfrom": [] |
| 83 | + } |
| 84 | + } |
| 85 | + ``` |
| 86 | + |
| 87 | + Save this data in a file (e.g. `acmedns.json`) and create a secret in the `cert-manager` project |
| 88 | + |
| 89 | + If you want to use `Issuer` in favor of a `ClusterIssuer` the secret needs to be created in the same project as the Issue will be created. |
| 90 | + |
| 91 | + ```shell |
| 92 | + $ oc -n cert-manager create secret generic acme-dns-staging --from-file acmedns.json=acmedns.json |
| 93 | + secret/acme-dns-staging created |
| 94 | + ``` |
| 95 | + |
| 96 | + This secret and the key (`acmedns.json`) needs to be placed in the `ClusterIssuer` manifest. |
| 97 | + |
| 98 | +2. Create a `ClusterIssuer` to handle the certificate issuing process |
| 99 | + |
| 100 | + ```shell |
| 101 | + $ cat <<EOF | oc create -f |
| 102 | + apiVersion: cert-manager.io/v1 |
| 103 | + kind: ClusterIssuer |
| 104 | + metadata: |
| 105 | + name: letsencrypt-staging |
| 106 | + spec: |
| 107 | + acme: |
| 108 | + |
| 109 | + preferredChain: "" |
| 110 | + privateKeySecretRef: |
| 111 | + name: letsencrypt-staging-private-key |
| 112 | + server: https://acme-staging-v02.api.letsencrypt.org/directory |
| 113 | + solvers: |
| 114 | + - dns01: |
| 115 | + acmeDNS: |
| 116 | + accountSecretRef: |
| 117 | + key: acmedns.json |
| 118 | + name: acme-dns-staging |
| 119 | + host: https://auth.example.com |
| 120 | + EOF |
| 121 | + ``` |
| 122 | +
|
| 123 | + {: .box-note} |
| 124 | + **Note:** For this article, we use the staging instance of Let's Encrypt. You should do it also this way to check if the setup is working properly. Only after issuing a certificate successfully, you should switch to the production ACME endpoint. Otherwise, you risk running into rate limiting and being blocked from the API if anything does not work as expected. |
| 125 | +
|
| 126 | +## OCP certificates |
| 127 | +
|
| 128 | +To issue a certificate, it is needed to create a `Certificate` CR in the project where you need it. If you create a certificate, some corresponding custom resources will be created. |
| 129 | +
|
| 130 | +- CertificateRequest, is used to request a signed certificate |
| 131 | +- Order, represents an order with an ACME server |
| 132 | +- Challenge, represents a challenge request with an ACME server |
| 133 | +
|
| 134 | +{:.mx-auto.d-block :} |
| 135 | +
|
| 136 | +### default ingress certificate |
| 137 | +
|
| 138 | +Unfortunately, it is not that easy to replace the ingress certificate as with upstream cert-manager and Kubernetes `Ingress` resources. In that case, only annotating the resource is needed to let the magic happen. See the original [documentation](https://cert-manager.io/docs/usage/ingress/){:target="_blank"} for details. |
| 139 | +
|
| 140 | +Replacing the default ingress certificate for the *.apps subdomain is a common day-2 task. Combining it with the use of cert-manager, it is really comfortable to have this automated for future updates of the related certificate. |
| 141 | +
|
| 142 | +1. Starting with creating a `Certificate` CR which uses the former configured `ClusterIssuer` to interact with the Let's Encrypt API. |
| 143 | +
|
| 144 | + ```yaml |
| 145 | + $ cat <<EOF | oc create -f |
| 146 | + apiVersion: cert-manager.io/v1 |
| 147 | + kind: Certificate |
| 148 | + metadata: |
| 149 | + name: letsencrypt-staging-wildcard |
| 150 | + namespace: openshift-ingress |
| 151 | + spec: |
| 152 | + secretName: letsencrypt-staging-wildcard |
| 153 | + secretTemplate: |
| 154 | + labels: |
| 155 | + stage: staging |
| 156 | + duration: 2160h # 90d |
| 157 | + renewBefore: 360h # 15d |
| 158 | + isCA: false |
| 159 | + privateKey: |
| 160 | + algorithm: RSA |
| 161 | + encoding: PKCS1 |
| 162 | + size: 2048 |
| 163 | + usages: |
| 164 | + - server auth |
| 165 | + - client auth |
| 166 | + dnsNames: |
| 167 | + - apps.ocp4.example.com |
| 168 | + - '*.apps.ocp4.example.com' |
| 169 | + issuerRef: |
| 170 | + name: letsencrypt-staging |
| 171 | + kind: ClusterIssuer |
| 172 | + group: cert-manager.io |
| 173 | + EOF |
| 174 | + ``` |
| 175 | +
|
| 176 | +2. After applying the manifest, the former discussed resources would be created, and you can monitor the status of the certificate by watching on it. As soon as the dns-01 challenge was solved the certificate will reach the ready state, and it can be used. |
| 177 | +
|
| 178 | + ```shell |
| 179 | + $ oc -n openshift-ingress get certificates |
| 180 | +
|
| 181 | + NAME READY SECRET AGE |
| 182 | + certificate.cert-manager.io/letsencrypt-staging-wildcard False letsencrypt-staging-wildcard 9s |
| 183 | + ``` |
| 184 | +
|
| 185 | +3. Patching the ingress controller after the certificate was issued is done by the following command. |
| 186 | +
|
| 187 | + ```shell |
| 188 | + $ oc -n openshift-ingress-operator patch --type=merge ingresscontrollers/default --patch '{"spec":{"defaultCertificate":{"name":"letsencrypt-staging-wildcard"}}}' |
| 189 | + ingresscontroller.operator.openshift.io/default patched |
| 190 | + ``` |
| 191 | +
|
| 192 | + After patching the resource, the router pods will be restarted and the certificate will be used. |
| 193 | +
|
| 194 | +### API server certificate |
| 195 | +
|
| 196 | +To replace the default API server certificate, you need to run similar steps as with the default ingress certificate. |
| 197 | +
|
| 198 | +1. A `Certificate` resource need to be created as followed. |
| 199 | +
|
| 200 | + ```shell |
| 201 | + $ cat <<EOF | oc create -f |
| 202 | + apiVersion: cert-manager.io/v1 |
| 203 | + kind: Certificate |
| 204 | + metadata: |
| 205 | + name: letsencrypt-staging-api |
| 206 | + namespace: openshift-config |
| 207 | + spec: |
| 208 | + dnsNames: |
| 209 | + - api.ocp4.example.com |
| 210 | + duration: 2160h0m0s |
| 211 | + issuerRef: |
| 212 | + group: cert-manager.io |
| 213 | + kind: ClusterIssuer |
| 214 | + name: letsencrypt-staging |
| 215 | + privateKey: |
| 216 | + algorithm: RSA |
| 217 | + encoding: PKCS1 |
| 218 | + size: 2048 |
| 219 | + renewBefore: 360h0m0s |
| 220 | + secretName: letsencrypt-staging-api |
| 221 | + secretTemplate: |
| 222 | + labels: |
| 223 | + stage: staging |
| 224 | + usages: |
| 225 | + - server auth |
| 226 | + - client auth |
| 227 | + EOF |
| 228 | + ``` |
| 229 | +
|
| 230 | +2. The status of the requested certificate can be monitored by watching the former created `Certificate` resource. |
| 231 | +
|
| 232 | + ```shell |
| 233 | + $ oc -n openshift-config get certificates |
| 234 | + NAME READY SECRET AGE |
| 235 | + letsencrypt-staging-api True letsencrypt-staging-api 122s |
| 236 | + ``` |
| 237 | +
|
| 238 | +3. To replace the API server certificate with the newly created one, the following command is enough. It will patch the `Apiserver` resource and a restart of the apiserver pods will be initiated. |
| 239 | +
|
| 240 | + ```shell |
| 241 | + $ oc patch apiserver/cluster --type=merge -p '{"spec":{"servingCerts": {"namedCertificates": [{"names": ["api.ocp4.example.com"], "servingCertificate": {"name": "letsencrypt-staging-api"}}]}}}' |
| 242 | + apiserver.config.openshift.io/cluster patched |
| 243 | + ``` |
0 commit comments