Skip to content

Commit 71826e8

Browse files
author
Kubernetes Submit Queue
authored
Merge pull request #41399 from NitishT/master
Automatic merge from submit-queue Added examples for Minio in standalone and distributed Modes **What this PR does / why we need it**: Added examples for Minio standalone and distributed modes **Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes # **Special notes for your reviewer**: **Release note**: ``` Adds Minio server deployment example. ```
2 parents 3a6a09c + a1fcf13 commit 71826e8

7 files changed

+492
-0
lines changed

‎storage/minio/README.md

Lines changed: 345 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,345 @@
1+
# Cloud Native Deployment of Minio using Kubernetes
2+
3+
## Table of Contents
4+
5+
- [Introduction](#introduction)
6+
- [Prerequisites](#prerequisites)
7+
- [Minio Standalone Server Deployment](#minio-standalone-server-deployment)
8+
- [Standalone Quickstart](#standalone-quickstart)
9+
- [Step 1: Create Persistent Volume Claim](#step-1-create-persistent-volume-claim)
10+
- [Step 2: Create Deployment](#step-2-create-minio-deployment)
11+
- [Step 3: Create LoadBalancer Service](#step-3-create-minio-service)
12+
- [Step 4: Resource cleanup](#step-4-resource-cleanup)
13+
- [Minio Distributed Server Deployment](#minio-distributed-server-deployment)
14+
- [Distributed Quickstart](#distributed-quickstart)
15+
- [Step 1: Create Minio Headless Service](#step-1-create-minio-headless-service)
16+
- [Step 2: Create Minio Statefulset](#step-2-create-minio-statefulset)
17+
- [Step 3: Create LoadBalancer Service](#step-3-create-minio-service)
18+
- [Step 4: Resource cleanup](#step-4-resource-cleanup)
19+
20+
## Introduction
21+
Minio is an AWS S3 compatible, object storage server built for cloud applications and devops. Minio is _cloud native_, meaning Minio understands that it is running within a cluster manager, and uses the cluster management infrastructure for allocation of compute and storage resources.
22+
23+
## Prerequisites
24+
25+
This example assumes that you have a Kubernetes version >=1.4 cluster installed and running, and that you have installed the [`kubectl`](https://kubernetes.io/docs/user-guide/prereqs/) command line tool in your path. Please see the
26+
[getting started guides](https://kubernetes.io/docs/getting-started-guides/) for installation instructions for your platform.
27+
28+
## Minio Standalone Server Deployment
29+
30+
The following section describes the process to deploy standalone [Minio](https://minio.io/) server on Kubernetes. The deployment uses the [official Minio Docker image](https://hub.docker.com/r/minio/minio/~/dockerfile/) from Docker Hub.
31+
32+
This section uses following core components of Kubernetes:
33+
34+
- [_Pods_](https://kubernetes.io/docs/user-guide/pods/)
35+
- [_Services_](https://kubernetes.io/docs/user-guide/services/)
36+
- [_Deployments_](https://kubernetes.io/docs/user-guide/deployments/)
37+
- [_Persistent Volume Claims_](https://kubernetes.io/docs/user-guide/persistent-volumes/#persistentvolumeclaims)
38+
39+
### Standalone Quickstart
40+
41+
Run the below commands to get started quickly
42+
43+
```sh
44+
kubectl create -f https://github.com/kubernetes/kubernetes/blob/master/examples/storage/minio/minio-standalone-pvc.yaml?raw=true
45+
kubectl create -f https://github.com/kubernetes/kubernetes/blob/master/examples/storage/minio/minio-standalone-deployment.yaml?raw=true
46+
kubectl create -f https://github.com/kubernetes/kubernetes/blob/master/examples/storage/minio/minio-standalone-service.yaml?raw=true
47+
```
48+
49+
### Step 1: Create Persistent Volume Claim
50+
51+
Minio needs persistent storage to store objects. If there is no
52+
persistent storage, the data stored in Minio instance will be stored in the container file system and will be wiped off as soon as the container restarts.
53+
54+
Create a persistent volume claim (PVC) to request storage for the Minio instance. Kubernetes looks out for PVs matching the PVC request in the cluster and binds it to the PVC automatically.
55+
56+
This is the PVC description.
57+
58+
```sh
59+
apiVersion: v1
60+
kind: PersistentVolumeClaim
61+
metadata:
62+
# This name uniquely identifies the PVC. Will be used in deployment below.
63+
name: minio-pv-claim
64+
annotations:
65+
volume.alpha.kubernetes.io/storage-class: anything
66+
labels:
67+
app: minio-storage-claim
68+
spec:
69+
# Read more about access modes here: http://kubernetes.io/docs/user-guide/persistent-volumes/#access-modes
70+
accessModes:
71+
- ReadWriteOnce
72+
resources:
73+
# This is the request for storage. Should be available in the cluster.
74+
requests:
75+
storage: 10Gi
76+
```
77+
78+
Create the PersistentVolumeClaim
79+
80+
```sh
81+
kubectl create -f https://github.com/kubernetes/kubernetes/blob/master/examples/storage/minio/minio-standalone-pvc.yaml?raw=true
82+
persistentvolumeclaim "minio-pv-claim" created
83+
```
84+
85+
### Step 2: Create Minio Deployment
86+
87+
A deployment encapsulates replica sets and pods — so, if a pod goes down, replication controller makes sure another pod comes up automatically. This way you won’t need to bother about pod failures and will have a stable Minio service available.
88+
89+
This is the deployment description.
90+
91+
```sh
92+
apiVersion: extensions/v1beta1
93+
kind: Deployment
94+
metadata:
95+
# This name uniquely identifies the Deployment
96+
name: minio-deployment
97+
spec:
98+
strategy:
99+
type: Recreate
100+
template:
101+
metadata:
102+
labels:
103+
# Label is used as selector in the service.
104+
app: minio
105+
spec:
106+
# Refer to the PVC created earlier
107+
volumes:
108+
- name: storage
109+
persistentVolumeClaim:
110+
# Name of the PVC created earlier
111+
claimName: minio-pv-claim
112+
containers:
113+
- name: minio
114+
# Pulls the default Minio image from Docker Hub
115+
image: minio/minio
116+
command:
117+
- minio
118+
args:
119+
- server
120+
- /storage
121+
env:
122+
# Minio access key and secret key
123+
- name: MINIO_ACCESS_KEY
124+
value: "minio"
125+
- name: MINIO_SECRET_KEY
126+
value: "minio123"
127+
ports:
128+
- containerPort: 9000
129+
hostPort: 9000
130+
# Mount the volume into the pod
131+
volumeMounts:
132+
- name: storage # must match the volume name, above
133+
mountPath: "/storage"
134+
```
135+
136+
Create the Deployment
137+
138+
```sh
139+
kubectl create -f https://github.com/kubernetes/kubernetes/blob/master/examples/storage/minio/minio-standalone-deployment.yaml?raw=true
140+
deployment "minio-deployment" created
141+
```
142+
143+
### Step 3: Create Minio Service
144+
145+
Now that you have a Minio deployment running, you may either want to access it internally (within the cluster) or expose it as a Service onto an external (outside of your cluster, maybe public internet) IP address, depending on your use case. You can achieve this using Services. There are 3 major service types — default type is ClusterIP, which exposes a service to connection from inside the cluster. NodePort and LoadBalancer are two types that expose services to external traffic.
146+
147+
In this example, we expose the Minio Deployment by creating a LoadBalancer service. This is the service description.
148+
149+
```sh
150+
apiVersion: v1
151+
kind: Service
152+
metadata:
153+
name: minio-service
154+
spec:
155+
type: LoadBalancer
156+
ports:
157+
- port: 9000
158+
targetPort: 9000
159+
protocol: TCP
160+
selector:
161+
app: minio
162+
```
163+
Create the Minio service
164+
165+
```sh
166+
kubectl create -f https://github.com/kubernetes/kubernetes/blob/master/examples/storage/minio/minio-standalone-service.yaml?raw=true
167+
service "minio-service" created
168+
```
169+
170+
The `LoadBalancer` service takes couple of minutes to launch. To check if the service was created successfully, run the command
171+
172+
```sh
173+
kubectl get svc minio-service
174+
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
175+
minio-service 10.55.248.23 104.199.249.165 9000:31852/TCP 1m
176+
```
177+
178+
### Step 4: Resource cleanup
179+
180+
Once you are done, cleanup the cluster using
181+
```sh
182+
kubectl delete deployment minio-deployment \
183+
&& kubectl delete pvc minio-pv-claim \
184+
&& kubectl delete svc minio-service
185+
```
186+
187+
## Minio Distributed Server Deployment
188+
189+
The following document describes the process to deploy [distributed Minio](https://docs.minio.io/docs/distributed-minio-quickstart-guide) server on Kubernetes. This example uses the [official Minio Docker image](https://hub.docker.com/r/minio/minio/~/dockerfile/) from Docker Hub.
190+
191+
This example uses following core components of Kubernetes:
192+
193+
- [_Pods_](https://kubernetes.io/docs/user-guide/pods/)
194+
- [_Services_](https://kubernetes.io/docs/user-guide/services/)
195+
- [_Statefulsets_](https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/)
196+
197+
### Distributed Quickstart
198+
199+
Run the below commands to get started quickly
200+
201+
```sh
202+
kubectl create -f https://github.com/kubernetes/kubernetes/blob/master/examples/storage/minio/minio-distributed-headless-service.yaml?raw=true
203+
kubectl create -f https://github.com/kubernetes/kubernetes/blob/master/examples/storage/minio/minio-distributed-statefulset.yaml?raw=true
204+
kubectl create -f https://github.com/kubernetes/kubernetes/blob/master/examples/storage/minio/minio-distributed-service.yaml?raw=true
205+
```
206+
207+
### Step 1: Create Minio Headless Service
208+
209+
Headless Service controls the domain within which StatefulSets are created. The domain managed by this Service takes the form: `$(service name).$(namespace).svc.cluster.local` (where “cluster.local” is the cluster domain), and the pods in this domain take the form: `$(pod-name-{i}).$(service name).$(namespace).svc.cluster.local`. This is required to get a DNS resolvable URL for each of the pods created within the Statefulset.
210+
211+
This is the Headless service description.
212+
213+
```sh
214+
apiVersion: v1
215+
kind: Service
216+
metadata:
217+
name: minio
218+
labels:
219+
app: minio
220+
spec:
221+
clusterIP: None
222+
ports:
223+
- port: 9000
224+
name: minio
225+
selector:
226+
app: minio
227+
```
228+
229+
Create the Headless Service
230+
231+
```sh
232+
$ kubectl create -f https://github.com/kubernetes/kubernetes/blob/master/examples/storage/minio/minio-distributed-headless-service.yaml?raw=true
233+
service "minio" created
234+
```
235+
236+
### Step 2: Create Minio Statefulset
237+
238+
A StatefulSet provides a deterministic name and a unique identity to each pod, making it easy to deploy stateful distributed applications. To launch distributed Minio you need to pass drive locations as parameters to the minio server command. Then, you’ll need to run the same command on all the participating pods. StatefulSets offer a perfect way to handle this requirement.
239+
240+
This is the Statefulset description.
241+
242+
```sh
243+
apiVersion: apps/v1beta1
244+
kind: StatefulSet
245+
metadata:
246+
name: minio
247+
spec:
248+
serviceName: minio
249+
replicas: 4
250+
template:
251+
metadata:
252+
annotations:
253+
pod.alpha.kubernetes.io/initialized: "true"
254+
labels:
255+
app: minio
256+
spec:
257+
containers:
258+
- name: minio
259+
env:
260+
- name: MINIO_ACCESS_KEY
261+
value: "minio"
262+
- name: MINIO_SECRET_KEY
263+
value: "minio123"
264+
image: minio/minio
265+
command:
266+
- minio
267+
args:
268+
- server
269+
- http://minio-0.minio.default.svc.cluster.local/data
270+
- http://minio-1.minio.default.svc.cluster.local/data
271+
- http://minio-2.minio.default.svc.cluster.local/data
272+
- http://minio-3.minio.default.svc.cluster.local/data
273+
ports:
274+
- containerPort: 9000
275+
hostPort: 9000
276+
# These volume mounts are persistent. Each pod in the PetSet
277+
# gets a volume mounted based on this field.
278+
volumeMounts:
279+
- name: data
280+
mountPath: /data
281+
# These are converted to volume claims by the controller
282+
# and mounted at the paths mentioned above.
283+
volumeClaimTemplates:
284+
- metadata:
285+
name: data
286+
annotations:
287+
volume.alpha.kubernetes.io/storage-class: anything
288+
spec:
289+
accessModes:
290+
- ReadWriteOnce
291+
resources:
292+
requests:
293+
storage: 10Gi
294+
```
295+
296+
Create the Statefulset
297+
298+
```sh
299+
$ kubectl create -f https://github.com/kubernetes/kubernetes/blob/master/examples/storage/minio/minio-distributed-statefulset.yaml?raw=true
300+
statefulset "minio" created
301+
```
302+
303+
### Step 3: Create Minio Service
304+
305+
Now that you have a Minio statefulset running, you may either want to access it internally (within the cluster) or expose it as a Service onto an external (outside of your cluster, maybe public internet) IP address, depending on your use case. You can achieve this using Services. There are 3 major service types — default type is ClusterIP, which exposes a service to connection from inside the cluster. NodePort and LoadBalancer are two types that expose services to external traffic.
306+
307+
In this example, we expose the Minio Deployment by creating a LoadBalancer service. This is the service description.
308+
309+
```sh
310+
apiVersion: v1
311+
kind: Service
312+
metadata:
313+
name: minio-service
314+
spec:
315+
type: LoadBalancer
316+
ports:
317+
- port: 9000
318+
targetPort: 9000
319+
protocol: TCP
320+
selector:
321+
app: minio
322+
```
323+
Create the Minio service
324+
325+
```sh
326+
$ kubectl create -f https://github.com/kubernetes/kubernetes/blob/master/examples/storage/minio/minio-distributed-service.yaml?raw=true
327+
service "minio-service" created
328+
```
329+
330+
The `LoadBalancer` service takes couple of minutes to launch. To check if the service was created successfully, run the command
331+
332+
```sh
333+
$ kubectl get svc minio-service
334+
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
335+
minio-service 10.55.248.23 104.199.249.165 9000:31852/TCP 1m
336+
```
337+
338+
### Step 4: Resource cleanup
339+
340+
You can cleanup the cluster using
341+
```sh
342+
kubectl delete statefulset minio \
343+
&& kubectl delete svc minio \
344+
&& kubectl delete svc minio-service
345+
```

‎storage/minio/minio-distributed-headless-service.yaml

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
apiVersion: v1
2+
kind: Service
3+
metadata:
4+
name: minio
5+
labels:
6+
app: minio
7+
spec:
8+
clusterIP: None
9+
ports:
10+
- port: 9000
11+
name: minio
12+
selector:
13+
app: minio

‎storage/minio/minio-distributed-service.yaml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
apiVersion: v1
2+
kind: Service
3+
metadata:
4+
name: minio-service
5+
spec:
6+
type: LoadBalancer
7+
ports:
8+
- port: 9000
9+
targetPort: 9000
10+
protocol: TCP
11+
selector:
12+
app: minio

0 commit comments

Comments
 (0)