Readiness probe

[guicassolato]

Implements health and readiness probe endpoints for the controllers. ATM, the endpoints are trivial readiness and health check endpoints the by default always return "ok".

The aggregated status of all AuthConfigs watched by the controller is an opt-in check.

New endpoints introduced:

• /healthy: Health probe (ping)
• /readyz: Aggregated readiness probe (by default, none)
• /readyz/authconfigs: Aggregated status of all AuthConfigs watched by the controller

In general, all endpoints return either 200 ("ok") or 500 (when 1+ probes fail).

The default binding network address is :8081. It can be changed using a newly introduced flag (command-line arg) --health-probe-addr.

The following query string parameters are supported:

• verbose=any: more verbose response messages;
• include=authconfigs: to include the aggregated status of AuthConfigs in the response;
• exclude=(check name): to exclude a particular probe - has no effect when passed on requests to /readyz/authconfigs.

Closes #355

Verification steps

$ make local-setup FF=1
$ kubectl port-forward deployment/authorino 8081:8081 &

$ curl http://localhost:8081/healthz
ok

$ curl http://localhost:8081/readyz
ok

$ kubectl apply -f -<<EOF
apiVersion: authorino.kuadrant.io/v1beta1
kind: AuthConfig
metadata:
  name: talker-api-protection
spec:
  hosts:
  - talker-api-authorino.127.0.0.1.nip.io
EOF
authconfig.authorino.kuadrant.io/talker-api-protection created

$ curl http://localhost:8081/readyz
ok

$ curl "http://localhost:8081/readyz?verbose=true"
[+]authconfigs ok
healthz check passed

$ kubectl apply -f -<<EOF
apiVersion: authorino.kuadrant.io/v1beta1
kind: AuthConfig
metadata:
  name: talker-api-protection-2
spec:
  hosts:
  - talker-api-authorino.127.0.0.1.nip.io
EOF
authconfig.authorino.kuadrant.io/talker-api-protection-2 created

$ curl http://localhost:8081/readyz
ok

$ curl "http://localhost:8081/readyz?include=authconfigs"
[-]authconfigs failed: reason withheld
healthz check failed

$ curl http://localhost:8081/readyz/authconfigs
internal server error: authconfig is not ready: default/talker-api-protection-2 (reason: HostsNotLinked)

[guicassolato]

It turns out this approach can be improved to solve the issue for the readiness probe.

An AuthConfig being not ready (due to a host collision, for example) has nothing to do with the AuthConfig Reconciler (or even less with the Authorino instance!) not being ready. Exposing this endpoint through controller-runtime's readiness check mixes those two things together and introduces a risk of occasionally shutting the entire Authorino instance off because of a unexceptional situation of having one or more AuthConfigs not ready.

This is because one could think that the /readyz endpoint is suitable for configuring the readinessProbe of the Authorino Deployment, when in fact, as-is, it is not. Currently, this is not done by the Authorino Operator, but when it starts doing so, this endpoint cannot be used indiscriminately. To use this endpoint, the authconfigs check would have to be excluded by supplying in the request the exclude=authconfigs parameter.

[guicassolato]

This is because one could think that the /readyz endpoint is suitable for configuring the readinessProbe of the Authorino Deployment, when in fact, as-is, it is not.

No longer true.

I've changed so the authconfigs check is now opt-in. Therefore people can use the upper level (aggregated) /readyz endpoint to set readiness probe for the Authorino Deployment without risking the status of AuthConfig CRs to affect the state of the service itself.

One who wants to include the readiness state of the AuthConfigs in the overall aggregated readiness probe check can call /readyz?include=authconfigs (not suitable for setting readiness check for the Authorino Deployment).

Alternatively, only the aggregated status of the AuthConfig CRs watched by that particular Authorino instance can be checked by calling /readyz/authconfigs.

[eguzki]

now I have to learn Golang generics 🥳

[didierofrivia]

Looks good to me! The only issue I see is the one reported by Eguz regarding returning the map of statuses