Skip to content
This repository was archived by the owner on Jan 16, 2018. It is now read-only.

Commit 0f31e98

Browse files
author
Christian Barra
committed
first commit
0 parents  commit 0f31e98

File tree

7 files changed

+773
-0
lines changed

7 files changed

+773
-0
lines changed

‎.gitignore

+62
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Byte-compiled / optimized / DLL files
2+
__pycache__/
3+
*.py[cod]
4+
5+
# C extensions
6+
*.so
7+
8+
# Distribution / packaging
9+
.Python
10+
env/
11+
build/
12+
develop-eggs/
13+
dist/
14+
downloads/
15+
eggs/
16+
lib/
17+
lib64/
18+
parts/
19+
sdist/
20+
var/
21+
*.egg-info/
22+
.installed.cfg
23+
*.egg
24+
MANIFEST
25+
26+
# PyInstaller
27+
# Usually these files are written by a python script from a template
28+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
29+
*.manifest
30+
*.spec
31+
32+
# Installer logs
33+
pip-log.txt
34+
pip-delete-this-directory.txt
35+
36+
# Unit test / coverage reports
37+
htmlcov/
38+
.tox/
39+
.coverage
40+
.cache
41+
nosetests.xml
42+
coverage.xml
43+
44+
# Translations
45+
*.mo
46+
*.pot
47+
48+
# Django stuff:
49+
*.log
50+
51+
# Sphinx documentation
52+
docs/_build/
53+
54+
# PyBuilder
55+
target/
56+
57+
userlist
58+
ssl.cert
59+
ssl.key
60+
env
61+
jupyterhub.sqlite
62+
src

‎LICENSE

+27
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
Copyright (c) 2017, Cassiny.io OÜ
2+
All rights reserved.
3+
4+
Redistribution and use in source and binary forms, with or without
5+
modification, are permitted provided that the following conditions are met:
6+
7+
* Redistributions of source code must retain the above copyright notice, this
8+
list of conditions and the following disclaimer.
9+
10+
* Redistributions in binary form must reproduce the above copyright notice,
11+
this list of conditions and the following disclaimer in the documentation
12+
and/or other materials provided with the distribution.
13+
14+
* Neither the name of the Cassiny.io OÜ nor the names of its
15+
contributors may be used to endorse or promote products derived from
16+
this software without specific prior written permission.
17+
18+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
19+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
20+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
21+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
22+
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
23+
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
24+
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
25+
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
26+
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
27+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

‎README.md

+213
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,213 @@
1+
# SwarmSpawner
2+
3+
**SwarmSpawner** enables [JupyterHub](https://github.com/jupyterhub/jupyterhub)
4+
to spawn single user notebook servers in Docker Services.
5+
6+
More info about Docker Services [here](https://docs.docker.com/engine/reference/commandline/service_create/).
7+
8+
## Prerequisites
9+
10+
Python version 3.3 and above is required.
11+
12+
Clone the repo:
13+
14+
```bash
15+
git clone https://github.com/cassinyio/SwarmSpawner
16+
cd SwarmSpawner
17+
```
18+
19+
Install dependencies:
20+
21+
```bash
22+
pip install -r requirements.txt
23+
```
24+
25+
## Installation
26+
27+
Install SwarmSpawner to the system:
28+
```bash
29+
python setup.py install
30+
```
31+
32+
## Configuration
33+
34+
### SwarmSpawner
35+
36+
Docker Engine in Swarm mode and the related services work in a different way compared to Docker containers.
37+
38+
39+
Tell JupyterHub to use SwarmSpawner by adding the following line to
40+
your `jupyterhub_config.py`:
41+
42+
```python
43+
c.JupyterHub.spawner_class = 'cassinyspawner.SwarmSpawner'
44+
45+
46+
# This should be the name of the jupyterhub service
47+
c.JupyterHub.hub_ip = '0.0.0.0'
48+
c.SwarmSpawner.jupyterhub_service_name = 'NameOfTheService'
49+
```
50+
There is the possibility to set parameters using `user_options`
51+
52+
53+
```python
54+
# To use user_options in service creation
55+
c.SwarmSpawner.use_user_options = False
56+
```
57+
58+
To control the creation of the services you have 2 ways, using _jupyterhub_config.py_ or _user_options_.
59+
60+
Remember that at the end you are just using the [Docker Engine API](https://docs.docker.com/engine/api/).
61+
62+
(https://docs.docker.com/engine/reference/commandline/service_create/#/specify-service-constraints---constraint)
63+
64+
### Use a configuration inside jupyterhub_config.py
65+
You can define *container_spec*, *resource_spec* and _networks_ within jupyterhub_config.py.
66+
67+
#### [Container_spec](https://github.com/docker/docker-py/blob/master/docs/user_guides/swarm_services.md)
68+
'command' depends from the image that you are using.
69+
If you use one of the images from the Jupyter docker stack you need to specify a command: /usr/local/bin/start-singleuser.sh
70+
71+
If you are using a specific image, well it's up to you to specify the right command.
72+
73+
```python
74+
c.SwarmSpawner.container_spec = {
75+
# The command to be run inside the service
76+
'command' : '/usr/local/bin/start-singleuser.sh', #(string or list)
77+
'Image' : 'YourImage',
78+
'mounts' : mounts
79+
}
80+
```
81+
82+
83+
##### Bind a Host dir
84+
With mounts your are going to mount a local directory of the host inside the container.
85+
86+
<u>Remember that source should exist in the node where you are creating the service.</u>
87+
88+
```python
89+
notebook_dir = os.environ.get('DOCKER_NOTEBOOK_DIR') or '/home/jovyan/work'
90+
c.SwarmSpawner.notebook_dir = notebook_dir
91+
```
92+
93+
```python
94+
mounts = [{'type' : 'bind',
95+
'source' : 'MountPointOnTheHost',
96+
'target' : 'MountPointInsideTheContainer',}]
97+
```
98+
99+
##### Mount a named volume
100+
With mounts your are going to mount a Docker Volume inside the container.
101+
If the volume doesn't exist it will be created.
102+
103+
```python
104+
mounts = [{'type' : 'volume',
105+
'source' : 'NameOfTheVolume',
106+
'target' : 'MountPointInsideTheContainer',}]
107+
```
108+
109+
For this type of volume you can also specify something like this:
110+
111+
```python
112+
mounts = [{'type' : 'volume',
113+
'source' : 'jupyterhub-user-{username}',
114+
'target' : 'MountPointInsideTheContainer',}]
115+
```
116+
117+
username will be the hashed version of the username.
118+
119+
120+
##### Mount an anonymous volume
121+
__This kind of volume will be removed with the service__
122+
```python
123+
mounts = [{'type' : 'volume',
124+
'target' : 'MountPointInsideTheContainer',}]
125+
```
126+
127+
#### Resource_spec
128+
129+
You can also specify some resource for each service
130+
131+
```python
132+
c.SwarmSpawner.resource_spec = {
133+
'cpu_limit' : 1, # (int) – CPU limit in units of 10^9 CPU shares.
134+
'mem_limit' : 512 * 1e6, # (int) – Memory limit in Bytes.
135+
'cpu_reservation' : 1000, # (int) – CPU reservation in units of 10^9 CPU shares.
136+
'mem_reservation' : 512 * 1e6, # (int) – Memory reservation in Bytes
137+
}
138+
```
139+
140+
#### Networks
141+
You can also specify a network, remember to create the network before creating the service.
142+
SwarmSpawner use the service name instead of the service ip as a consequence JupyterHub and servers should share the same overlay network (network across nodes).
143+
144+
```python
145+
c.SwarmSpawner.networks = ["mynetwork"] #list of networks
146+
```
147+
148+
### Using user_options
149+
150+
**user_options, if used, will overwrite jupyter_config.py for services.**
151+
152+
If you set 'c.SwarmSpawner.use_user_option = True' the spawner will use the dict passed through the form or as json body when using the Hub Api.
153+
154+
The spawner expect a dict with these keys:
155+
156+
```python
157+
user_options = {
158+
'container_spec' : {
159+
'command' : '/usr/local/bin/start-singleuser.sh', #(string or list) command to be run in the image.
160+
'Image' : '', # name of the image
161+
'mounts' : mounts, # Same as jupyterhub_config
162+
'resource_spec' : {
163+
'cpu_limit' : 1, # (int) – CPU limit in units of 10^9 CPU shares.
164+
'mem_limit' : 512 * 1e6,# (int) – Memory limit in Bytes.
165+
'cpu_reservation' : 1000, # (int) – CPU reservation in units of 10^9 CPU shares.
166+
'mem_reservation' : 512 * 1e6, # (int) – Memory reservation in Bytes
167+
},
168+
'placement' : [], #list of constrains
169+
'network' : [], #list of networks
170+
'name' : '' # Name of service
171+
```
172+
173+
### Services Prefix
174+
175+
Services using this format `{service_prefix}-{service_owner}-{service_suffix}`
176+
177+
You can change the service_prefix in this way:
178+
179+
Prefix of the service in Docker
180+
```python
181+
c.SwarmSpawner.service_prefix = "jupyterhub"
182+
```
183+
184+
`service_owner` is the hexdigest() of the hashed `user.name`.
185+
186+
In case of multi sigle-server per user `service_suffix` is the name of the server, otherwise is always 1.
187+
188+
### Downloading images
189+
Docker Engine in Swarm mode downloads images automatically from the repository.
190+
Either the image is available on the remote repository or locally, if not you will get an error.
191+
192+
You can use all the docker images built for [JupyterHub](https://github.com/jupyter/docker-stacks).
193+
194+
## Contributing
195+
196+
If you would like to contribute to the project, please read our
197+
[contributor documentation](http://jupyter.readthedocs.io/en/latest/contributor/content-contributor.html)
198+
and the [`CONTRIBUTING.md`](CONTRIBUTING.md).
199+
200+
For a **development install**, clone the [repository](https://github.com/jupyterhub/dockerspawner)
201+
and then install from source:
202+
203+
```bash
204+
git clone https://github.com/cassiny/SwarmSpawner
205+
cd SwarmSpawner
206+
pip3 install -r dev-requirements.txt -e .
207+
```
208+
209+
## License
210+
211+
All code is licensed under the terms of the revised BSD license.
212+
213+

‎cassinyspawner/__init__.py

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
from .swarmspawner import SwarmSpawner
2+
3+
__all__ = ['SwarmSpawner']

0 commit comments

Comments
 (0)