This example shows how to configure alert rules in Kyma.
- Define a new alerting rule in Prometheus.
- Configure Alertmanager to handle alerts.
- Review alert-rules/README.md in Kyma. It is mandatory in order to complete the example.
- Review alertmanager/README.md in Kyma. It is mandatory in order to complete the example.
- Kyma as the target deployment environment.
- VictorOps Routing Keys and a Service API Key Prometheus Integration Guide – VictorOps
-
Edit unhealthy-pods-rules.yaml by removing the line comment symbol
# -
Edit alertmanager/values.yaml to enable the victorOps (and/or) slack integration. You will have to replace some data such a tokens and urls with your configuration.
-
Run Kyma followed by the kubectl command to access prometheus dashboard.
$ kubectl port-forward pod/prometheus-core-0 -n kyma-system 9090:9090In http://localhost:9090/rules, you will find the rule, pod-not-running-rule.
-
After Kyma in fully running we should start to receive alerts in VictorOps or Slack.
-
VictorOps
-
Slack
Review you configuration if the alerts are not received after the expected time. You also should Validate alertmanager/values.yaml files.
- Create the required pod sample-metrics to solve the issue.
$ cd /examples/monitoring-alert-rules/k8s/
$ kubectl apply -f lambda-function.yaml- VictorOps
- Slack
Run the following command to completely remove the example and all its resources from the cluster:
# notice that you do not need to indicate the Namespace, as the system created the resources using 'default namespace'
kubectl delete all -l example=monitoring-alert-rules