Skip to content
This repository was archived by the owner on Nov 26, 2018. It is now read-only.

Commit e20ac17

Browse files
author
Brian Luft
committed
Merge pull request #76 from BotBotMe/fix-installation
Clean up installation steps and docs
2 parents 0634636 + 07c87e8 commit e20ac17

File tree

9 files changed

+202
-166
lines changed

9 files changed

+202
-166
lines changed

‎.env.example

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Required
2+
SECRET_KEY=supersecretkeyhere
3+
WEB_PORT=8000
4+
EMAIL_BACKEND='django.core.mail.backends.console.EmailBackend'
5+
GOPATH=$VIRTUAL_ENV
6+
WEB_SECRET_KEY=somerandomstring
7+
STORAGE_URL=postgres://user:pass@localhost:5432/botbot
8+
REDIS_PLUGIN_STORAGE_URL=redis://localhost:6379/0
9+
REDIS_PLUGIN_QUEUE_URL=redis://localhost:6379/1
10+
REDIS_SSEQUEUE_URL=redis://localhost:6379/2
11+
SSE_ENDPOINT_URL=http://localhost:3000/
12+
SSE_HOST=localhost:3000
13+
14+
# Optional
15+
# MEMCACHE_URL=127.0.0.1:11211
16+
# STATIC_ROOT=/var/www/botbot/static
17+
# MEDIA_ROOT=/var/www/botbot/uploads
18+
# DEBUG=True
19+
# SMTP_URL=smtp://user:pass@host:port
20+
# SMTP_TLS=True
21+
# ALLOWED_HOSTS=host1,host2
22+
# INCLUDE_DJANGO_ADMIN=False
23+
# EXCLUDE_NICKS=nick1,nick2

‎.travis.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,6 @@ before_script:
1818
- "psql -c 'create database botbot;' -U postgres"
1919
- "psql -c 'create extension hstore;' -U postgres botbot"
2020
- "psql -c 'create extension hstore;' -U postgres template1"
21-
- "echo 'DATABASE_URL=postgres://postgres@localhost:5432/botbot' >> .env"
21+
- "echo 'STORAGE_URL=postgres://postgres@localhost:5432/botbot' >> .env"
2222
- "manage.py collectstatic --noinput"
2323
script: "manage.py test"

‎README.md

Lines changed: 0 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -13,62 +13,3 @@ Botbot is collection of tools for running IRC bots. It has primarily been
1313
used with Freenode channels but works with other IRC networks or servers.
1414

1515
[Documentation](http://botbot.readthedocs.org/en/latest/)
16-
17-
18-
Quickstart Guide:
19-
=================
20-
21-
```
22-
mkvirtualenv botbotenv
23-
cdvirtualenv
24-
25-
# Set necessary environment variables e.g. in:
26-
vi bin/postactivate
27-
28-
export GOPATH=$VIRTUAL_ENV
29-
export WEB_SECRET_KEY=somerandomstring
30-
export STORAGE_URL=postgres://user:pass@localhost:5432/botbot
31-
export REDIS_PLUGIN_STORAGE_URL=redis://localhost:6379/0
32-
export REDIS_PLUGIN_QUEUE_URL=redis://localhost:6379/1
33-
export REDIS_SSEQUEUE_URL=redis://localhost:6379/2
34-
export SSE_ENDPOINT_URL=http://localhost:3000/
35-
export SSE_HOST=localhost:3000
36-
37-
source bin/postactivate
38-
39-
# Clone and install the Django part
40-
mkdir src
41-
cd src
42-
git clone [email protected]:BotBotMe/botbot-web.git
43-
pip install -e botbot-web
44-
pip install -r botbot-web/requirements.txt
45-
46-
# Install the Go related part
47-
cd ..
48-
go get github.com/BotBotMe/botbot-bot
49-
go get github.com/BotBotMe/botbot-eventsource
50-
51-
# Setup the database
52-
manage.py dbshell
53-
CREATE EXTENSION hstore;
54-
CREATE EXTENSION unaccent;
55-
ALTER FUNCTION unaccent(text) IMMUTABLE;
56-
57-
manage.py syncdb
58-
manage.py migrate
59-
manage.py collectstatic
60-
manage.py createsuperuser
61-
62-
# Run all of the services at once with Foreman
63-
ln -s src/botbot-web/Procfile
64-
foreman start
65-
66-
# Or each one manually
67-
botbot-bot
68-
botbot-eventsource
69-
manage.py runserver
70-
manage.py run_plugins
71-
```
72-
73-
Go to http://127.0.0.1:8000/admin/ and setup a bot and a channel.
74-
Finally open http://127.0.0.1:8000/.

‎botbot/apps/bots/admin.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ class MembershipInline(admin.TabularInline):
3030

3131

3232
class ChatBotAdmin(admin.ModelAdmin):
33+
exclude = ('connection', 'server_identifier')
3334
list_display = ('__unicode__', 'is_active')
3435
list_editable = ('is_active',)
3536
list_filter = ('is_active',)

‎docs/developers.rst

Lines changed: 91 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,75 @@
1-
Developing with Botbot
2-
========================
1+
**********************
2+
Developing with BotBot
3+
**********************
4+
5+
Architecture
6+
============
7+
8+
Several loosely coupled pieces make up the whole of BotBot:
9+
10+
1. **botbot-bot:** An IRC client capable of connecting to multiple IRC networks, and connecting multiple channels and nicks per network. (Go)
11+
2. **botbot-plugins:** A plugin framework - plugins receive messages from IRC channels and can respond within the channel. (Python)
12+
3. **botbot-web:** A web site for managing bots/channels as well as a beautiful public interface for channel logs. (Python/Django)
13+
4. **botbot-eventsource:** A basic SSE provider that the web site can connect to for real-time logs. (Go)
14+
15+
.. image:: /images/botbot-architecture.png
16+
17+
Configuration / Environment Variables
18+
-------------------------------------
19+
20+
At Lincoln Loop we've found the 12-factor style to be helpful and use those guidelines. As recommended, we rely on environment variables whenever possible for determining site-specific configuration values.
21+
22+
The .env file
23+
^^^^^^^^^^^^^
24+
25+
We've adopted a convention used by Heroku and others of creating a ``.env`` file in the project root that can be used as a base for managing environment variables. Tools like Honcho and Foreman will use the ``.env`` file to bootstrap the environment for each process it starts up. If you prefer to use these tools, all you have to worry about is editing the ``.env`` file as needed. The ``.env`` file can also be used as a template for other methods of populating environment variables.
26+
27+
We don't want make assumptions about how developers prefer to configure their environments. There are some handy open source libs out there that we could use to easily make each service utilize the ``.env`` file, but this won't work well for everyone or every environment. If you need to run a service individually, you'll need to make sure the proper environment variables are configured. The next section describes some easy ways to handle this.
28+
29+
30+
Running Services
31+
----------------
32+
33+
Run All Services
34+
^^^^^^^^^^^^^^^^
35+
36+
Honcho is great for getting everything started quickly. Running this command will load all services defined in the ``Procfile``::
37+
38+
honcho start
39+
40+
Run Services Individually
41+
^^^^^^^^^^^^^^^^^^^^^^^^^
42+
43+
When working on code changes or debugging, it is often desirable to manage one or more of the services (bot, plugin runner, runserver, eventsource) independently.
44+
45+
If you'd like to make use of the ``.env`` file, you can still start services individually using Honcho::
46+
47+
honcho start web # Starts Django site
48+
honcho start bot # Starts IRC client
49+
honcho start plugins # Starts plugin runner
50+
honcho start realtime # Starts event source (SSE)
51+
52+
If you would prefer not to use Honcho, you'll need to manage the environment variables. Many developers use the virtualenv ``postactivate`` feature to set required environment variables whenever a virtualenv is activated. An alternative approach could be to attempt to set variables directly from the ``.env`` file. As an example, you could put the following into a ``set_env.sh`` file::
53+
54+
export $(cat .env | grep -v ^# | xargs)
55+
56+
Then you could invoke commands and individual services like::
57+
58+
source set_env.sh && manage.py runserver
59+
source set_env.sh && botbot-bot
60+
source set_env.sh && botbot-eventsource
61+
source set_env.sh && manage.py run_plugins
62+
63+
If you've explicitly set the environment through your own methods, services can be invoked like usual::
64+
65+
manage.py runserver
66+
botbot-bot
67+
botbot-eventsource
68+
manage.py run_plugins
69+
370

471
Go IRC Client (bot)
5-
-------------------
72+
~~~~~~~~~~~~~~~~~~~
673

774
Execution starts in main.go, in function "main". That starts the chatbots (via NetworkManager), the goroutine which listens for commands from Redis, and the mainLoop goroutine, then waits for a Ctrl-C or kill to quit.
875

@@ -29,10 +96,27 @@ And now, in ASCII art::
2996

3097

3198
Django Site
32-
------------
99+
~~~~~~~~~~~~
100+
101+
You can run commands within the Honcho environment using the ``run`` command::
102+
103+
honcho run manage.py dbshell
104+
honcho run manage.py syncdb
105+
106+
If you're using the ``set_env`` method::
107+
108+
source set_env.sh && manage.py dbshell
109+
source set_env.sh && manage.py syncdb
110+
111+
If you've explicitly set the environment variables, run commands like usual::
112+
113+
manage.py dbshell
114+
manage.py syncdb
115+
116+
33117

34118
Working with LESS
35-
~~~~~~~~~~~~~~~~~~
119+
~~~~~~~~~~~~~~~~~
36120

37121
LESS requires Node.js. There are shortcuts in the Makefile for installing everything necessary:
38122

@@ -49,8 +133,10 @@ From this point forward, if you need to compile LESS run:
49133
To automatically compile whenever you save a change:
50134

51135
.. code-block:: bash
136+
52137
make less-watch
53138
139+
54140
Plugins
55141
--------
56142

‎docs/getting_started.rst

Lines changed: 24 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -42,21 +42,30 @@ Now that your bot is connected to a network or server, you can start having it j
4242

4343
1. Go to ``http://localhost:8000/admin/bots/channel/add/``
4444
2. Select your bot from the dropdown
45-
3. **Channel**: ``#botbot-test``
46-
4. Select **'Is public'**
47-
5. In the **Active Plugins** section click *Add another Active Plugin* and select **Ping**.
48-
6. Save. In the ``honcho`` console output you should see messages similar to::
49-
50-
12:14:42 bot.1 | 2013/09/19 12:14:42 Command: REFRESH
51-
12:14:42 bot.1 | 2013/09/19 12:14:42 Reloading configuration from database
52-
12:14:42 bot.1 | 2013/09/19 12:14:42 config.Channel: [#botbot-test ]
53-
12:14:42 bot.1 | 2013/09/19 12:14:42 [RAW1] -->JOIN #botbot-test
54-
55-
7. In your IRC client, join `#botbot-test <irc://irc.freenode.net:6667/botbot-test>`_. Try issuing a `ping` command (using your bot's nick in place of "mybot"). The bot should respond with a friendly message.
56-
8. Go back to the home page ``http://localhost:8000``, you should see the channel listed as a public channel.
57-
9. **Add another Active Plugin** and this time select **Logger**.
58-
10. **Save**. Your ``honcho`` console should once again show a refresh
59-
11. In your IRC client, go to `#botbot-test <irc:irc.freenode.net:6667/botbot-test>`_ and post a message. You should now have a log available at ``http://localhost:8000/freenode/botbot-test``. Each message you post in the channel shows up in the ``honcho`` console.
45+
3. **Channel**: ``#botbot-warmup``. This is a channel where we test channel bots.
46+
4. If you'd like the channel to be listed on the site home page, Select **'Is public'**
47+
5. Several useful plugins will already be configured. At a minimum, ``ping`` and ``logger`` will be helpful for testing the bot.
48+
6. Save your channel. You should be directed back to the Channel list view.
49+
7. On the Channel list view, select your channel. From the ``Actions`` dropdown select "Reload botbot-bot configuration" and press "Go". You should see something similar in the ``honcho`` console output (edited for brevity)::
50+
51+
14:15:07 bot.1 | I0711 14:15:07.557470 61493 botbot.go:67] Command: REFRESH
52+
14:15:07 bot.1 | I0711 14:15:07.557546 61493 botbot.go:124] HandleCommand: REFRESH
53+
14:15:07 bot.1 | I0711 14:15:07.557557 61493 botbot.go:153] Reloading configuration from database
54+
14:15:07 bot.1 | I0711 14:15:07.557564 61493 network.go:49] Entering in NetworkManager.RefreshChatbots
55+
14:15:07 bot.1 | I0711 14:15:07.558013 61493 storage.go:121] config.Id: 1
56+
14:15:07 bot.1 | I0711 14:15:07.558753 61493 storage.go:145] config.Channel: [#botbot-warmup]
57+
...
58+
14:15:07 bot.1 | I0711 14:15:07.559072 61493 irc.go:460] [Info] The channels the bot is connected to need to be updated
59+
14:15:07 bot.1 | I0711 14:15:07.559083 61493 irc.go:473] [Info] Joining new channel: #botbot-warmup
60+
14:15:07 bot.1 | I0711 14:15:07.559096 61493 network.go:98] Exiting NetworkManager.RefreshChatbots
61+
14:15:07 bot.1 | I0711 14:15:07.559111 61493 irc.go:228] [RAW thahslkd334558 on chat.freenode.net:6667 (0xc208028750) ] --> JOIN #botbot-warmup
62+
63+
64+
8. In your IRC client, join `#botbot-warmup <irc://irc.freenode.net:6667/botbot-warmup>`_. Try issuing a `ping` command (using your bot's nick in place of "mybot"). The bot should respond with a friendly message.
65+
9. Go back to the home page ``http://localhost:8000``, you should see the channel listed as a public channel.
66+
10. **Add another Active Plugin** and this time select **Logger**.
67+
11. **Save** and "Reload botbot-bot configuration" as before. Your ``honcho`` console should once again show a refresh
68+
12. In your IRC client, go to `#botbot-warmup <irc:irc.freenode.net:6667/botbot-warmup>`_ and post a message. You should now have a log available at ``http://localhost:8000/freenode/botbot-warmup``. Each message you post in the channel shows up in the ``honcho`` console.
6069

6170

6271
.. warning:

‎docs/images/botbot-architecture.png

87.6 KB
Loading

‎docs/install.rst

Lines changed: 4 additions & 86 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ Postgresql with hStore extension
2525
Go
2626
~~
2727

28-
`botbot-bot` requires a go version greater than 1.2
28+
Version 1.2 or higher required
2929

3030
* **OS X**: ``brew install go``
3131
* **Ubuntu**: ``apt-get install golang-go``
@@ -51,7 +51,7 @@ Run in a terminal:
5151
make dependencies
5252
5353
# Adjust ``.env`` file if necessary. Defaults are chosen for local debug environments.
54-
# If your Postgres server requires a password, you'll need to override DATABASE_URL
54+
# If your Postgres server requires a password, you'll need to override STORAGE_URL
5555
# The default database name is 'botbot'
5656
$EDITOR .env
5757
@@ -80,91 +80,9 @@ You should now be able to access the site at ``http://localhost:8000``. Log in w
8080

8181
See :doc:`getting_started` for instructions on configuring a bot.
8282

83-
Environment Variables to override
84-
---------------------------------::
83+
If you plan make code changes, please read through the :doc:`developers` doc.
8584

86-
# Django (required)
87-
WEB_SECRET_KEY=supersecretkeyhere
88-
89-
# DB Storage where channel/bot information is stored
90-
STORAGE_URL=postgres://user:pass@localhost:5432/botbot
91-
92-
# Pipes
93-
REDIS_PLUGIN_STORAGE_URL=redis://localhost:6379/1
94-
REDIS_PLUGIN_QUEUE_URL=redis://localhost:6379/2
95-
REDIS_SSEQUEUE_URL=redis://localhost:6379/3
96-
97-
# The host and port eventsource is delivering and the browser is listening
98-
# for live changes.
99-
SSE_ENDPOINT_URL=http://localhost:3000/
100-
101-
# Specific Django settings to override
102-
# MEMCACHE_URL=127.0.0.1:11211
103-
# STATIC_ROOT=/var/www/botbot/static
104-
# MEDIA_ROOT=/var/www/botbot/uploads
105-
# DEBUG=True
106-
# SMTP_URL=smtp://user:pass@host:port
107-
# SMTP_TLS=True
108-
# ALLOWED_HOSTS=host1,host2
109-
# INCLUDE_DJANGO_ADMIN=False
110-
# EXCLUDE_NICKS=nick1,nick2
111-
112-
Serving In Production
113-
---------------------
114-
115-
When you deploy botbot to production, we recommend that you do not use the Procfile. Instead, serve three pieces individually:
116-
117-
* **botbot-web**: should be served as a wsgi application, from the ``wsgi.py`` file located at ``src/botbot/botbot/wsgi.py`` from `uwsgi <https://uwsgi-docs.readthedocs.org/en/latest/>`_, `gunicorn <http://gunicorn.org/>`_, `mod_wsgi <https://code.google.com/p/modwsgi/>`_, or any other wsgi server.
118-
* **botbot-plugins**: should be run as an application from botbot's manage.py file. Use `upstart <http://upstart.ubuntu.com/>`_, `systemd <http://freedesktop.org/wiki/Software/systemd/>`_, `init <http://www.sensi.org/~alec/unix/redhat/sysvinit.html>`_, or whatever your system uses for managing long-running tasks. An example upstart script is provided below.
119-
* **botbot-bot**: should also be run as an application from your system's task management system. An example upstart script is provided below.
120-
121-
Example upstart scripts
122-
-----------------------
123-
124-
``botbot-plugins.conf``:
125-
126-
.. code-block:: bash
127-
128-
# BotBot Plugins
129-
# logs to /var/log/upstart/botbot_plugins.log
130-
131-
description "BotBot Plugins"
132-
start on startup
133-
stop on shutdown
134-
135-
respawn
136-
env LANG=en_US.UTF-8
137-
exec /srv/botbot/bin/manage.py run_plugins
138-
setuid www-data
139-
140-
``botbot-bot.conf``:
141-
142-
.. code-block:: bash
143-
144-
# BotBot-bot
145-
# logs to /var/log/upstart/botbot.log
146-
147-
description "BotBot"
148-
start on startup
149-
stop on shutdown
150-
151-
respawn
152-
env LANG=en_US.UTF-8
153-
env DATABASE_URL=postgres://yourdburl
154-
env REDIS_PLUGIN_QUEUE_URL=redis://localhost:6379/0
155-
156-
exec /srv/botbot/bin/botbot-bot
157-
setuid www-data
158-
159-
Running In A Subdirectory
160-
-------------------------
161-
162-
If you intend to run botbot in a subdirectory of your website, for example at ``http://example.com/botbot`` you'll need to add two options to your ``settings.py``:
163-
164-
.. code-block:: python
165-
166-
FORCE_SCRIPT_NAME = '/botbot'
167-
USE_X_FORWARDED_HOST = True
85+
If you plan to run BotBot in a production environment please read the :doc:`production` doc.
16886

16987

17088
Running Tests

0 commit comments

Comments
 (0)