Add support for cri-resource-manager project

This commit adds support for installing and configuring  the
[cri-resource-manager](https://github.com/intel/cri-resource-manager) project as an systemd-based service.

It also adds the automation to configure `kubelet` service to consume
it as a remote container runtime. Finally, it's providing the automation
for cleaning up the `kubelet` service configuration to its original state
without `cri-resource-manager`.

Binary installation will be temporally consumed from a personal fork that
is currently hosting `cri-resource-manager` binaries in the meantime that
`cri-resource-manager` generates its packaging strategy.

Signed-off-by: Obed N Munoz <[email protected]>

Author: Obed N Munoz

clr-k8s-examples/10-cri-resource-manager/README.md

@@ -0,0 +1,70 @@
+# CRI Resource Manager
+CRI Resource Manager serves as a relay/proxy between kubelet and the container runtime, relaying requests and responses back and forth between these two, potentially altering requests as they fly by.
+
+This document explains a very simple use case for the `cri-resource-manager`, for more details and tweaks
+on CRI Resource Manager service, you can go to https://github.com/intel/cri-resource-manager.
+
+## Install
+
+[`install.sh`](install.sh) script will download the binary and install it as an `systemd` service unit. This script will be executed in all nodes where `cri-resmgr` is required.
+
+Below you can see the available variables you can use to customize the usage of your CRI Resource Manager service.
+
+| Variable                    | Description                               | Default Value                                    |
+|-----------------------------|-------------------------------------------|--------------------------------------------------|
+| `RUNNER`                    | Default Container Runtime                 | `containerd`                                     |
+| `CRI_RESMGR_POLICY`         | CRI Resource Manager Policy type          | `null`                                           |
+| `CRI_RESMGR_POLICY_OPTIONS` | CRI Resource Manager extra policy options | `-dump='reset,full:.*' -dump-file=/tmp/cri.dump` |
+| `CRI_RESMGR_DEBUG_OPTIONS`  | CRI Resource Manager debugging options    | `<none>`                                         |
+
+**Example:**
+```bash
+$ RUNNER=containerd ./install.sh
+```
+
+Verify that the cri-resource-manager service is actually running.
+
+```bash
+$ systemctl status cri-resource-manager
+```
+
+Verify that the `cri-resmgr` socket is created, it will indicate that `cri-resource-manager` is ready to receive requests.
+```bash
+$ sudo ls -la /var/run/cri-resmgr/cri-resmgr.sock
+```
+
+## Setup as a container runtime in `kubelet`
+
+The [`setup.sh`](setup.sh) script will configure the `kubelet` service to use the `cri-resource-manager` relay as its remote container runtime. This script will be executed in all nodes where `cri-resmgr` is being configured.
+
+**Example:**
+```bash
+$ ./setup.sh
+```
+
+Kubelet service should be restarted and now using `cri-resource-manager` as its container runtime
+
+```bash
+$ ps aux | grep kubelet | grep container-runtime
+root       28703  1.7  2.0 1246348 83088 ?       Ssl  20:03   0:06 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime remote --container-runtime-endpoint unix:///var/run/cri-resmgr/cri-resmgr.sock
+```
+
+`cri-resource-manager` service's logs will be located at `/tmp/cri.dump`
+
+```bash
+$ tail /tmp/cri.dump
+```
+
+## Cleanup
+
+The [`clean.sh`](clean.sh) will first clean the `kubelet` service as it was before the `cri-resource-manager` and restarts `kubelet` service. This script will be executed in all nodes where `cri-resmgr` is being uninstalled.
+Then. it will proceed to stop the `cri-resource-manager` service.
+
+**Example:**
+```bash
+$ ./clean.sh
+```
+
+## More kubernetes native approach (experimental)
+
+In case that you're interested in a more Kubernetes native way of deploying the CRI Resource manager, take a look on: https://github.com/intel/cri-resource-manager/pull/55

clr-k8s-examples/10-cri-resource-manager/clean.sh

@@ -0,0 +1,23 @@
+#!/bin/bash
+# Copyright (c) 2019 Intel Corporation
+#
+# SPDX-License-Identifier: Apache-2.0
+#
+# Uninstall and stop the CRI Resource Manager service
+
+set -o errexit
+set -o nounset
+
+# Kubelet
+KUBEADM_FLAGS="/var/lib/kubelet/kubeadm-flags.env"
+sudo rm -f /etc/systemd/system/kubelet.service.d/99-cri-resource-manager.conf
+sudo systemctl daemon-reload
+sudo systemctl restart kubelet
+
+if sudo test -f "$KUBEADM_FLAGS.bkp" ; then
+    sudo mv $KUBEADM_FLAGS.bkp $KUBEADM_FLAGS
+fi
+
+# CRI Resource Manager
+sudo systemctl stop cri-resource-manager
+sudo systemctl disable cri-resource-manager

clr-k8s-examples/10-cri-resource-manager/install.sh

@@ -0,0 +1,36 @@
+#!/bin/bash
+# Copyright (c) 2019 Intel Corporation
+#
+# SPDX-License-Identifier: Apache-2.0
+#
+# Install and start the CRI Resource Manager service
+
+set -o errexit
+set -o nounset
+
+RUNNER=${RUNNER:-"containerd"}
+CRI_RESMGR_POLICY=${CRI_RESMGR_POLICY:-"null"}
+CRI_RESMGR_POLICY_OPTIONS=${CRI_RESMGR_POLICY_OPTIONS:-"-dump='reset,full:.*' -dump-file=/tmp/cri.dump"}
+CRI_RESMGR_DEBUG_OPTIONS=${CRI_RESMGR_DEBUG_OPTIONS:-""}
+
+curl https://raw.githubusercontent.com/obedmr/cri-resource-manager/master/godownloader.sh | bash
+sudo cp ./bin/* /usr/bin/
+
+runtime_socket=$(sudo find /run/ -iname $RUNNER.sock | head -1)
+CRI_RESMGR_POLICY_OPTIONS+=" -runtime-socket=$runtime_socket -image-socket=$runtime_socket"
+
+sudo mkdir -p /etc/sysconfig/
+cat <<EOF | sudo tee /etc/sysconfig/cri-resource-manager
+POLICY=$CRI_RESMGR_POLICY
+POLICY_OPTIONS=$CRI_RESMGR_POLICY_OPTIONS
+DEBUG_OPTIONS=$CRI_RESMGR_DEBUG_OPTIONS
+EOF
+
+sudo mkdir -p /etc/systemd/system/
+curl https://raw.githubusercontent.com/obedmr/cri-resource-manager/master/cmd/cri-resmgr/cri-resource-manager.service | sudo tee /etc/systemd/system/cri-resource-manager.service
+
+sudo sed -i '/Requires=/d' /etc/systemd/system/cri-resource-manager.service
+sudo systemctl daemon-reload
+sudo systemctl restart cri-resource-manager.service
+sudo systemctl enable cri-resource-manager.service
+

clr-k8s-examples/10-cri-resource-manager/setup.sh

@@ -0,0 +1,29 @@
+#!/bin/bash
+# Copyright (c) 2019 Intel Corporation
+#
+# SPDX-License-Identifier: Apache-2.0
+#
+# Configure CRI Resource Manager as container runtime endpoint for kubelet
+
+set -o errexit
+set -o nounset
+
+CRI_RESMGR_SOCKET="/var/run/cri-resmgr/cri-resmgr.sock"
+KUBEADM_FLAGS="/var/lib/kubelet/kubeadm-flags.env"
+
+if sudo test -S "$CRI_RESMGR_SOCKET" ; then
+	sudo mkdir -p /etc/systemd/system/kubelet.service.d/
+	cat <<EOF | sudo tee /etc/systemd/system/kubelet.service.d/99-cri-resource-manager.conf
+[Service]
+Environment=KUBELET_EXTRA_ARGS=
+Environment=KUBELET_EXTRA_ARGS="--container-runtime remote --container-runtime-endpoint unix://${CRI_RESMGR_SOCKET}"
+EOF
+
+	if sudo test -f "$KUBEADM_FLAGS" ; then
+	    sudo mv $KUBEADM_FLAGS $KUBEADM_FLAGS.bkp
+	fi
+
+	sudo systemctl daemon-reload
+	sudo systemctl restart cri-resource-manager
+	sudo systemctl restart kubelet
+fi

clr-k8s-examples/README.md

@@ -156,6 +156,9 @@ kubectl -n monitoring port-forward svc/grafana 3000
 Grafana is available at this URL http://localhost:3000 . Default credentials are
 `admin/admin`. Upon entering you will be asked to chose a new password.
 
+### CRI Resource Manager
+Go to [`10-cri-resource-manager`](./10-cri-resource-manager).
+
 ## Cleaning up the cluster (Hard reset to a clean state)
 
 Run `reset_stack.sh` on all the nodes