Kubernete: Support for tls/x509 redis session connections

README.md

@@ -30,7 +30,7 @@ See our [product comparison](https://www.open-emr.org/wiki/index.php/AWS_Cloud_P
 * [OpenEMR Monitor](utilities/openemr-monitor): OpenEMR Monitor is based on Prometheus, cAdvisor, Grafana, and alertmanger which helps administrator to monitor the status of containers
 * [Portainer](utilities/portainer): Portainer is a lightweight management UI which allows you to easily manage your different Docker environments (Docker hosts)
 * [OpenEMR Environment Migrator](utilities/openemr-env-migrator): OpenEMR Environment Migrator is used to migrate your container environment to the new storage directory or the remote host easily
-* [OpenEMR Kubernetes Orchestrations](kubernetes):  OpenEMR Kubernetes orchestration on Minikube. Creates 2 instances of OpenEMR with 1 instance of MariaDB, Redis, and phpMyAdmin.
+* [OpenEMR Kubernetes Orchestrations](kubernetes):  OpenEMR Kubernetes orchestration.
 
 ### Community Contributions
 

kubernetes/README.md

@@ -2,7 +2,7 @@
 OpenEMR Kubernetes orchestration. Orchestration included OpenEMR, MariaDB, Redis, and phpMyAdmin.
   - OpenEMR - 3 deployment replications of OpenEMR are created. Replications can be increased/decreased. Ports for both http and https.
   - MariaDB - 2 statefulset replications of MariaDB (1 primary/master with 1 replica/slave) are created. Replications can be increased/decreased which will increase/decrease number of replica/slaves. Connections are encrypted over the wire (ssl is enforced by default; X509 can be enforced by following pertinent comments in following scripts: 2 places in mysql/configmap.yaml, 2 places in openemr/deployment.yaml, 1 place in phpmyadmin/configmap.yaml, 1 place in phpmyadmin/deployment.yaml).
-  - Redis - Configured to support failover. There is 1 master and 2 slaves (no read access on slaves) for a statefulset, 3 sentinels for another statefulset, and then 2 proxies deployment. The proxies ensure that redis traffic is always directed towards master. The proxy replications can be increased/decreased. However the primary/slaves and sentinels would require script changes if wish to increase/decrease replicates for these since these are hard-coded several place in the scripts. There are 3 users/passwords (`default` (defaultpassword), `replication` (replicationpassword), `admin` (adminpassword)) used in this redis scheme, and the passwords should be set to something else if use this scheme in production. The main place the passwords are set is in kubernetes/redis/configmap-acl.yaml script. Other places where passwords are used include the following: `replication` in kubernetes/redis/configmap-main.yaml, `admin` in kubernetes/redis/configmap-pipy.yaml, `admin` in kubernetes/redis/statefulset-sentinel.yaml. The `default` is the typical worker/app/client user.
+  - Redis - Configured to support failover. There is 1 master and 2 slaves (no read access on slaves) for a statefulset, 3 sentinels for another statefulset, and then 2 proxies deployment. The proxies ensure that redis traffic is always directed towards master. The proxy replications can be increased/decreased. However the primary/slaves and sentinels would require script changes if wish to increase/decrease replicates for these since these are hard-coded several place in the scripts. There are 3 users/passwords (`default` (defaultpassword), `replication` (replicationpassword), `admin` (adminpassword)) used in this redis scheme, and the passwords should be set to something else if use this scheme in production. The main place the passwords are set is in redis/configmap-acl.yaml script. Other places where passwords are used include the following: `replication` in redis/configmap-main.yaml, `admin` in redis/configmap-pipy.yaml, `admin` in redis/statefulset-sentinel.yaml, `admin` in redis/healthcheck-haproxy.yaml. The `default` is the typical worker/app/client user. Connections are encrypted over the wire (ssl is enforced by default; X509 can be enforced by following pertinent comments in following scripts: 2 places in openemr/deployment.yaml, 1 place in redis/configmap-main.yaml, 1 place in redis/healthcheck-haproxy.yaml, 1 place in redis/statefulset-redis.yaml, 2 places in redis/statefulset-sentinel.yaml).
   - phpMyAdmin - There is 1 deployment instance of phpMyAdmin. Ports for both http and https.
 
 Would not consider this production quality, but will be a good working, starting point, and hopefully open the door to a myriad of other kubernetes based solutions. Note this is supported by 7.0.0 and higher dockers. If wish to use the most recent development codebase, then can change from openemr/openemr:7.0.1 to openemr/openemr:dev (in the openemr/deployment.yaml script), which is built nightly from the development codebase. If you wish to build dynamically from a branch/tag from a github repo or other git repo, then can change from openemr/openemr:7.0.1 to openemr/openemr:flex (in the openemr/deployment.yaml script) (note this will take much longer to start up (probably at least 10 minutes and up to 90 minutes) and is more cpu intensive since each instance of OpenEMR will download codebase and build separately).

kubernetes/certs/redis-openemr-client.yaml

@@ -0,0 +1,25 @@
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  name: redis-openemr-client
+spec:
+  secretName: redis-openemr-client-certs
+  duration: 87660h # 10y
+  renewBefore: 360h # 15d
+  isCA: false
+  privateKey:
+    size: 2048
+    algorithm: RSA
+    encoding: PKCS1
+  usages:
+    - digital signature
+    - key encipherment
+    - client auth
+  subject:
+    organizations:
+      - openemr
+  commonName: openemr
+  issuerRef:
+    name: ca-issuer
+    kind: Issuer
+    group: cert-manager.io

kubernetes/certs/redis.yaml

@@ -0,0 +1,31 @@
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  name: redis
+spec:
+  secretName: redis-certs
+  duration: 87660h # 10y
+  renewBefore: 360h # 15d
+  isCA: false
+  privateKey:
+    size: 2048
+    algorithm: RSA
+    encoding: PKCS1
+  usages:
+    - digital signature
+    - key encipherment
+    - server auth
+    - client auth
+  subject:
+    organizations:
+      - redis
+  commonName: redis
+  dnsNames:
+    - redisproxy
+    - redis-0.redis
+    - redis-1.redis
+    - redis-2.redis
+  issuerRef:
+    name: ca-issuer
+    kind: Issuer
+    group: cert-manager.io

kubernetes/certs/redisproxy.yaml

@@ -0,0 +1,26 @@
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  name: redisproxy
+spec:
+  secretName: redisproxy-certs
+  duration: 87660h # 10y
+  renewBefore: 360h # 15d
+  isCA: false
+  privateKey:
+    size: 2048
+    algorithm: RSA
+    encoding: PKCS1
+  usages:
+    - digital signature
+    - key encipherment
+    - server auth
+    - client auth
+  subject:
+    organizations:
+      - redisproxy
+  commonName: redisproxy
+  issuerRef:
+    name: ca-issuer
+    kind: Issuer
+    group: cert-manager.io

kubernetes/certs/sentinel.yaml

@@ -0,0 +1,26 @@
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  name: sentinel
+spec:
+  secretName: sentinel-certs
+  duration: 87660h # 10y
+  renewBefore: 360h # 15d
+  isCA: false
+  privateKey:
+    size: 2048
+    algorithm: RSA
+    encoding: PKCS1
+  usages:
+    - digital signature
+    - key encipherment
+    - server auth
+    - client auth
+  subject:
+    organizations:
+      - sentinel
+  commonName: sentinel
+  issuerRef:
+    name: ca-issuer
+    kind: Issuer
+    group: cert-manager.io

kubernetes/custom-dockers/haproxy-28-redis-tools/Dockerfile

@@ -0,0 +1,4 @@
+FROM haproxy:2.8
+USER root
+RUN apt-get update && apt-get install -y --no-install-recommends redis-tools
+USER haproxy

kubernetes/kub-down

@@ -8,7 +8,11 @@ kubectl delete \
     -f certs/mysql-replication.yaml \
     -f certs/mysql-openemr-client.yaml \
     -f certs/phpmyadmin.yaml \
-    -f certs/mysql-phpmyadmin-client.yaml
+    -f certs/mysql-phpmyadmin-client.yaml \
+    -f certs/redis.yaml \
+    -f certs/redis-openemr-client.yaml \
+    -f certs/sentinel.yaml \
+    -f certs/redisproxy.yaml
 
 kubectl delete -f https://github.com/cert-manager/cert-manager/releases/download/v1.11.0/cert-manager.yaml
 
@@ -19,7 +23,8 @@ kubectl delete \
     -f mysql/statefulset.yaml \
     -f redis/configmap-main.yaml \
     -f redis/configmap-acl.yaml \
-    -f redis/configmap-pipy.yaml \
+    -f redis/configmap-haproxy.yaml \
+    -f redis/healthcheck-haproxy.yaml \
     -f redis/statefulset-redis.yaml \
     -f redis/statefulset-sentinel.yaml \
     -f redis/deployment-redisproxy.yaml \
@@ -34,4 +39,4 @@ kubectl delete \
     -f volumes/website.yaml \
     -f openemr/secret.yaml \
     -f openemr/deployment.yaml \
-    -f openemr/service.yaml 
+    -f openemr/service.yaml 

kubernetes/kub-down.bat

@@ -8,7 +8,11 @@ kubectl delete ^
     -f certs/mysql-replication.yaml ^
     -f certs/mysql-openemr-client.yaml ^
     -f certs/phpmyadmin.yaml ^
-    -f certs/mysql-phpmyadmin-client.yaml
+    -f certs/mysql-phpmyadmin-client.yaml ^
+    -f certs/redis.yaml ^
+    -f certs/redis-openemr-client.yaml ^
+    -f certs/sentinel.yaml ^
+    -f certs/redisproxy.yaml
 
 kubectl delete -f https://github.com/cert-manager/cert-manager/releases/download/v1.11.0/cert-manager.yaml
 
@@ -19,7 +23,8 @@ kubectl delete ^
     -f mysql/statefulset.yaml ^
     -f redis/configmap-main.yaml ^
     -f redis/configmap-acl.yaml ^
-    -f redis/configmap-pipy.yaml ^
+    -f redis/configmap-haproxy.yaml ^
+    -f redis/healthcheck-haproxy.yaml ^
     -f redis/statefulset-redis.yaml ^
     -f redis/statefulset-sentinel.yaml ^
     -f redis/deployment-redisproxy.yaml ^

kubernetes/kub-up

@@ -13,7 +13,11 @@ kubectl apply \
     -f certs/mysql-replication.yaml \
     -f certs/mysql-openemr-client.yaml \
     -f certs/phpmyadmin.yaml \
-    -f certs/mysql-phpmyadmin-client.yaml
+    -f certs/mysql-phpmyadmin-client.yaml \
+    -f certs/redis.yaml \
+    -f certs/redis-openemr-client.yaml \
+    -f certs/sentinel.yaml \
+    -f certs/redisproxy.yaml
 echo "...waiting 15 seconds to ensure certs are created..."
 sleep 15
 
@@ -24,7 +28,8 @@ kubectl apply \
     -f mysql/statefulset.yaml \
     -f redis/configmap-main.yaml \
     -f redis/configmap-acl.yaml \
-    -f redis/configmap-pipy.yaml \
+    -f redis/configmap-haproxy.yaml \
+    -f redis/healthcheck-haproxy.yaml \
     -f redis/statefulset-redis.yaml \
     -f redis/statefulset-sentinel.yaml \
     -f redis/deployment-redisproxy.yaml \
@@ -39,4 +44,4 @@ kubectl apply \
     -f volumes/website.yaml \
     -f openemr/secret.yaml \
     -f openemr/deployment.yaml \
-    -f openemr/service.yaml
+    -f openemr/service.yaml

kubernetes/kub-up.bat

@@ -11,7 +11,11 @@ kubectl apply ^
     -f certs/mysql-replication.yaml ^
     -f certs/mysql-openemr-client.yaml ^
     -f certs/phpmyadmin.yaml ^
-    -f certs/mysql-phpmyadmin-client.yaml
+    -f certs/mysql-phpmyadmin-client.yaml ^
+    -f certs/redis.yaml ^
+    -f certs/redis-openemr-client.yaml ^
+    -f certs/sentinel.yaml ^
+    -f certs/redisproxy.yaml
 timeout 15
 
 kubectl apply ^
@@ -21,7 +25,8 @@ kubectl apply ^
     -f mysql/statefulset.yaml ^
     -f redis/configmap-main.yaml ^
     -f redis/configmap-acl.yaml ^
-    -f redis/configmap-pipy.yaml ^
+    -f redis/configmap-haproxy.yaml ^
+    -f redis/healthcheck-haproxy.yaml ^
     -f redis/statefulset-redis.yaml ^
     -f redis/statefulset-sentinel.yaml ^
     -f redis/deployment-redisproxy.yaml ^

kubernetes/openemr/deployment.yaml

@@ -46,17 +46,24 @@ spec:
               key: admin-pass
         - name: OE_USER
           value: "admin"
+        - name: PHPREDIS_BUILD
+          value: "e571a81f8d3009aab38cbb88dde865edeb0607ac"
         - name: REDIS_SERVER
           value: "redisproxy"
         - name: REDIS_PASSWORD
           value: "defaultpassword"
+        - name: REDIS_TLS
+          value: "yes"
+        # uncomment below if using redis x509
+        #- name: REDIS_X509
+        #  value: "yes"
         - name: SWARM_MODE
           value: "yes"
         - name: FORCE_DATABASE_SSL_CONNECT
         # uncomment below line (and comment above line) if forcing mysql x509
         #- name: FORCE_DATABASE_X509_CONNECT
           value: "1"
-        image: openemr/openemr:7.0.1
+        image: openemr/openemr:7.0.2
         name: openemr
         ports:
         - containerPort: 80
@@ -68,6 +75,8 @@ spec:
         volumeMounts:
         - mountPath: /root/certs/mysql/server
           name: mysql-openemr-client-certs
+        - mountPath: /root/certs/redis
+          name: redis-openemr-client-certs
         - mountPath: /var/www/localhost/htdocs/openemr/sites
           name: websitevolume
         - mountPath: /etc/ssl
@@ -87,6 +96,17 @@ spec:
           #  path: mysql-cert
           #- key: tls.key
           #  path: mysql-key
+      - name: redis-openemr-client-certs
+        secret:
+          secretName: redis-openemr-client-certs
+          items:
+          - key: ca.crt
+            path: redis-ca
+          # uncomment below if using redis x509
+          #- key: tls.crt
+          #  path: redis-cert
+          #- key: tls.key
+          #  path: redis-key
       - name: websitevolume
         persistentVolumeClaim:
           claimName: websitevolume

kubernetes/redis/configmap-haproxy.yaml

@@ -0,0 +1,28 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: haproxy-config
+data:
+  haproxy.cfg: |
+    defaults
+     mode tcp
+     timeout connect 4s
+     timeout server 30s
+     timeout client 30s
+    global
+     insecure-fork-wanted
+     external-check
+    resolvers mydns
+     parse-resolv-conf
+     accepted_payload_size 4096
+     hold valid 3s
+    frontend ft_redis
+     bind *:6379
+     default_backend bk_redis
+    backend bk_redis
+     option external-check
+     external-check command /var/lib/haproxy/healthcheck.sh
+     default-server check inter 3s fall 1 rise 1 resolvers mydns
+     server R1 redis-0.redis.default.svc.cluster.local:6379
+     server R2 redis-1.redis.default.svc.cluster.local:6379
+     server R3 redis-2.redis.default.svc.cluster.local:6379

kubernetes/redis/configmap-main.yaml

@@ -27,7 +27,17 @@ data:
     # this is the second ConfigMap will be mounted to. it has the list of users needed.
     aclfile /conf/acl/users.acl
 
-    # port, each redis nodes will be used
-    port 6379
+    # tls certs and setting
+    tls-cert-file /certs/tls.crt
+    tls-key-file /certs/tls.key
+    tls-ca-cert-file /certs/ca.crt
+    tls-auth-clients no
+    # uncomment below (and comment line above) if using redis x509
+    # tls-auth-clients yes
+    tls-replication yes
+
+    # port, each redis nodes will be used (only use tls)
+    port 0
+    tls-port 6379
     # More configurations are optional, if not provided, redis will consider default values ------
     # ------ More details on configuration : https://redis.io/docs/manual/config/ ------