Skip to content

Commit a39a2f6

Browse files
mbentleymbentley
authored
Merge pull request #503 from achetronic/docs/how-to-kubernetes
docs: How to deploy in Kubernetes
2 parents 1424dda + 8af0050 commit a39a2f6

File tree

16 files changed

+595
-0
lines changed

16 files changed

+595
-0
lines changed

‎external_mongodb/README.md

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -196,3 +196,62 @@ While I have this WIP for migrating from all in one, it would be much simplier t
196196
docker volume rm omada-data omada-logs ;\
197197
docker network rm omada
198198
```
199+
200+
## Kubernetes Deployment Guide
201+
202+
In advanced setups, deploying applications in container orchestrators like Kubernetes is common,
203+
even in homelabs to enhance reliability.
204+
205+
Below, we’ll use both Helm and Kustomize to deploy **MongoDB** and **Omada Controller**.
206+
207+
### Deploy MongoDB
208+
209+
210+
We use a custom Helm chart that wraps Bitnami's MongoDB chart.
211+
212+
The customization includes templates for managing secrets securely, avoiding the bad practice of
213+
hardcoding credentials in `values.yaml`.
214+
215+
Instead, it supports tools like **External Secrets** to fetch credentials from secure vaults and
216+
generate Kubernetes secrets automatically.
217+
218+
The custom Helm chart has credentials hardcoded as an example, but provides commented examples
219+
about how to get secrets safely from mentioned vaults.
220+
221+
Let's deploy our chart:
222+
223+
1. Update dependencies
224+
225+
```console
226+
helm dependency update kubernetes/mongodb
227+
```
228+
229+
2. Deploy MongoDB in the cluster currently pointed by your Kubeconfig
230+
231+
> [!IMPORTANT]
232+
> Customize chart's parameters to meet your needs
233+
234+
```console
235+
helm upgrade --install mongodb kubernetes/mongodb \
236+
-f kubernetes/mongodb/values-customized.yaml \
237+
-n omada-controller --create-namespace
238+
```
239+
240+
### Deploy Omada Controller
241+
242+
We'll use Kustomize since there's no official Helm chart provided by Omada Controller's maintainers
243+
or a trusted external provider like Bitnami.
244+
245+
Ensure your cluster has Ingress-Nginx as the ingress controller and Cert-Manager for certificate management.
246+
247+
If you're in an on-premises environment, you might need tools like MetalLB or Kube-VIP to provision
248+
load balancers for services of type LoadBalancer.
249+
250+
To deploy Omada Controller, just execute the following command
251+
252+
```console
253+
kubectl apply -k kubernetes/production
254+
```
255+
256+
> [!IMPORTANT]
257+
> Always adjust parameters to align with your infrastructure and requirements.

‎external_mongodb/kubernetes/mongodb/Chart.yaml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
apiVersion: v2
2+
name: mongodb
3+
version: 0.1.0
4+
dependencies:
5+
- name: mongodb
6+
version: 14.13.0
7+
repository: https://charts.bitnami.com/bitnami

‎external_mongodb/kubernetes/mongodb/templates/_helpers.tpl

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
{{/*
2+
Expand the name of the chart.
3+
*/}}
4+
{{- define "meta-mongodb.name" -}}
5+
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
6+
{{- end }}
7+
8+
{{/*
9+
Create a default fully qualified app name.
10+
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
11+
If release name contains chart name it will be used as a full name.
12+
*/}}
13+
{{- define "meta-mongodb.fullname" -}}
14+
{{- if .Values.fullnameOverride }}
15+
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
16+
{{- else }}
17+
{{- $name := default .Chart.Name .Values.nameOverride }}
18+
{{- if contains $name .Release.Name }}
19+
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
20+
{{- else }}
21+
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
22+
{{- end }}
23+
{{- end }}
24+
{{- end }}
25+
26+
{{/*
27+
Create chart name and version as used by the chart label.
28+
*/}}
29+
{{- define "meta-mongodb.chart" -}}
30+
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
31+
{{- end }}
32+
33+
{{/*
34+
Common labels
35+
*/}}
36+
{{- define "meta-mongodb.labels" -}}
37+
helm.sh/chart: {{ include "meta-mongodb.chart" . }}
38+
{{ include "meta-mongodb.selectorLabels" . }}
39+
{{- if .Chart.AppVersion }}
40+
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
41+
{{- end }}
42+
app.kubernetes.io/managed-by: {{ .Release.Service }}
43+
{{- end }}
44+
45+
{{/*
46+
Selector labels
47+
*/}}
48+
{{- define "meta-mongodb.selectorLabels" -}}
49+
app.kubernetes.io/name: {{ include "meta-mongodb.name" . }}
50+
app.kubernetes.io/instance: {{ .Release.Name }}
51+
{{- end }}

‎external_mongodb/kubernetes/mongodb/templates/external-secrets.yaml

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{{- range $key, $value := .Values.customComponents.externalSecrets }}
2+
apiVersion: external-secrets.io/v1beta1
3+
kind: ExternalSecret
4+
metadata:
5+
name: {{ include "meta-mongodb.fullname" $ }}-{{ $key }}
6+
labels:
7+
{{- include "meta-mongodb.labels" $ | nindent 4 }}
8+
{{- with $value.annotations }}
9+
annotations:
10+
{{- toYaml . | nindent 4 }}
11+
{{- end }}
12+
spec:
13+
{{ toYaml $value.spec | nindent 2 }}
14+
---
15+
{{- end }}

‎external_mongodb/kubernetes/mongodb/values-customized.yaml

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
customComponents: {}
2+
3+
# externalSecrets:
4+
# mongodb-users-credentials:
5+
# annotations: {}
6+
# spec:
7+
# secretStoreRef:
8+
# kind: ClusterSecretStore
9+
# name: gitlab-secret-store
10+
# target:
11+
# name: mongodb-users-credentials
12+
# data:
13+
# - secretKey: mongodb-root-password
14+
# remoteRef:
15+
# key: MONGODB_USERS_CREDENTIALS_PASSWORD_ROOT
16+
# - secretKey: mongodb-passwords
17+
# remoteRef:
18+
# key: MONGODB_USERS_CREDENTIALS_PASSWORD_OMADA
19+
20+
# Ref: https://github.com/bitnami/charts/blob/main/bitnami/mongodb/values.yaml
21+
mongodb:
22+
# MongoDB Authentication parameters
23+
auth:
24+
enabled: true
25+
26+
# MongoDB root user
27+
#rootUser: root
28+
29+
# MongoDB root password
30+
# Ref: https://github.com/bitnami/containers/tree/main/bitnami/mongodb#setting-the-root-user-and-password-on-first-run
31+
rootPassword: "provision-me-externally"
32+
33+
# MongoDB custom users and databases
34+
# Ref: https://github.com/bitnami/containers/tree/main/bitnami/mongodb#creating-a-user-and-database-on-first-run
35+
# List of custom users to be created during the initialization
36+
# List of passwords for the custom users set at `auth.usernames`
37+
# List of custom databases to be created during the initialization
38+
# Ref: https://github.com/bitnami/charts/issues/16975#issuecomment-1803017023
39+
# Relationship between following lists is [0]->[0]->[0]; [1]->[1]->[1]; ...
40+
databases: ["omada"]
41+
usernames: ["omada"]
42+
passwords: ["provision-me-externally"]
43+
44+
# Existing secret with MongoDB credentials (keys: `mongodb-passwords`, `mongodb-root-password`, `mongodb-metrics-password`, `mongodb-replica-set-key`)
45+
# When it's set the passwords defined in previous parameters are ignored.
46+
#existingSecret: "mongodb-users-credentials"
47+
48+
# Dictionary of initdb scripts
49+
# Specify dictionary of scripts to be run at first boot
50+
initdbScripts:
51+
52+
# Ref: https://github.com/mbentley/docker-omada-controller/blob/master/external_mongodb/omada.js
53+
init-mongo.sh: |
54+
#!/bin/bash
55+
56+
mongosh --authenticationDatabase admin -u root -p $MONGODB_ROOT_PASSWORD <<EOF
57+
use omada
58+
db.grantRolesToUser("omada", [
59+
{ role: "readWrite", db: "omada" },
60+
{ role: "dbOwner", db: "omada" },
61+
{ role: "readWrite", db: "omada_data" },
62+
{ role: "dbOwner", db: "omada_data" }
63+
]);
64+
EOF
65+
66+
# Optionally specify an extra list of additional volumeMounts for the MongoDB container(s)
67+
# extraVolumeMounts:
68+
# - name: extras
69+
# mountPath: /usr/share/extras
70+
# readOnly: true
71+
extraVolumeMounts: []
72+
73+
# Optionally specify an extra list of additional volumes to the MongoDB statefulset
74+
# extraVolumes:
75+
# - name: extras
76+
# emptyDir: {}
77+
extraVolumes: []
78+
79+
# Cluster IP is fixed as Omada Controller is listening on the host network to look for the APs
80+
# and then, cannot resolve internal DNS request properly for Service FQDN
81+
#service:
82+
# clusterIP: '10.96.89.12'
83+
84+
persistence:
85+
enabled: true
86+
87+
# Name of the PVC and mounted volume
88+
name: "mongodb-data"
89+
90+
# Provide an existing `PersistentVolumeClaim` (only when `architecture=standalone`)
91+
# If defined, PVC must be created manually before volume will be bound
92+
existingClaim: ""
93+
94+
# Setting it to "keep" to avoid removing PVCs during a helm delete operation.
95+
# Leaving it empty will delete PVCs after the chart is deleted
96+
resourcePolicy: "keep"
97+
98+
# PVC Storage Class for MongoDB data volume
99+
# If undefined (the default) or set to null, no storageClassName spec is
100+
# set, choosing the default provisioner.
101+
#storageClass: "standard-nfs"
102+
103+
# PV Access Mode
104+
accessModes:
105+
- ReadWriteOnce
106+
107+
# PVC Storage Request for MongoDB data volume
108+
size: 8Gi
109+
110+
# ATTENTION: Following parameters are only needed when deployed as StatefulSet.
111+
# I have not yet decided what fits better with my homelab in this regard
112+
113+
# Persistent Volume Claim Retention Policy
114+
# Ref: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#persistentvolumeclaim-retention
115+
persistentVolumeClaimRetentionPolicy:
116+
# Enable Persistent volume retention policy for MongoDB Statefulset
117+
enabled: false
118+
119+
# Volume retention behavior when the replica count of the StatefulSet is reduced
120+
whenScaled: Retain
121+
122+
# Volume retention behavior that applies when the StatefulSet is deleted
123+
whenDeleted: Retain

‎external_mongodb/kubernetes/mongodb/values.yaml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
nameOverride: ""
2+
fullnameOverride: ""
3+
4+
customComponents:
5+
6+
externalSecrets: {}
7+
# mongodb-auth-secret:
8+
# annotations: {}
9+
# spec: {}
10+
11+
# Ref: https://github.com/bitnami/charts/blob/main/bitnami/mongodb/values.yaml
12+
mongodb: {}

‎external_mongodb/kubernetes/omada-controller/base/configmap-env.yaml

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Ref: https://github.com/mbentley/docker-omada-controller/blob/master/docker-compose.yml
2+
3+
apiVersion: v1
4+
kind: ConfigMap
5+
metadata:
6+
name: omada-controller-environment
7+
data:
8+
#
9+
PUID: "508"
10+
PGID: "508"
11+
12+
#
13+
SHOW_SERVER_LOGS: "true"
14+
SHOW_MONGODB_LOGS: "false"
15+
SSL_CERT_NAME: "tls.crt"
16+
SSL_KEY_NAME: "tls.key"
17+
TZ: "Etc/UTC"
18+
19+
# Ref: https://github.com/mbentley/docker-omada-controller/tree/master/external_mongodb#common-steps
20+
NO_MONGODB: "true"
21+
MONGO_EXTERNAL: "true"
22+
MONGODB_HOST: "mongodb.mongodb.svc:27017"
23+
EAP_MONGOD_URI: "mongodb://omada:${MONGODB_PASSWORD}@${MONGODB_HOST}/omada"
24+
25+
#
26+
MANAGE_HTTP_PORT: "8088"
27+
MANAGE_HTTPS_PORT: "8043"
28+
PORTAL_HTTP_PORT: "8088"
29+
PORTAL_HTTPS_PORT: "8843"
30+
31+
#
32+
PORT_APP_DISCOVERY: "27001"
33+
PORT_DISCOVERY: "29810"
34+
35+
# V1
36+
PORT_MANAGER_V1: "29811"
37+
PORT_ADOPT_V1: "29812"
38+
PORT_UPGRADE_V1: "29813"
39+
40+
# V2
41+
PORT_MANAGER_V2: "29814"
42+
PORT_TRANSFER_V2: "29815"
43+
44+
PORT_RTTY: "29816"

0 commit comments

Comments
 (0)