Devops, add ganache, travis and tests support (#4)

* ops

* rm volume mount

* gitignore fix

* rename file

* bug fix

* apply ganache patch + fix integration test

* create abis folder if not exists

* travis.yml

* fix travis

* cache npm & docker builds

* rm `cat` cmd in travis

* fix travis

* update README

* Updated package.json

Author: tsuberim

Diff for: .dockerignore

@@ -0,0 +1,5 @@
+node_modules
+package-lock.json
+*.log
+.env
+subgraph.yaml

Diff for: .env

@@ -0,0 +1,2 @@
+# daostack_mainnet__postgresPassword=<use a strong passowrd>
+# daostack_mainnet__ethereumProvider=https://mainnet.infura.io/v3/<replace with your api key>

Diff for: .gitignore

@@ -1,5 +1,8 @@
 node_modules
 dist
-data
-.env
-.api_key
+*.log
+subgraph.yaml
+docker-compose.yml
+config.json
+abis
+src/types

Diff for: .travis.yml

@@ -0,0 +1,38 @@
+language: node_js
+node_js:
+  - "8"
+
+sudo: required
+
+services:
+  - docker
+cache:
+  directories:
+    - node_modules
+    - $HOME/docker
+
+addons:
+  apt:
+    packages:
+      - docker-ce
+
+before_cache:
+  - mkdir -p $HOME/docker && docker images -a --filter='dangling=false' --format '{{.Repository}}:{{.Tag}} {{.ID}}' | xargs -n 2 -t sh -c 'test -e $HOME/docker/$1.tar.gz || docker save $0 | gzip -2 > $HOME/docker/$1.tar.gz'
+
+before_install:
+  - if [[ -d $HOME/docker ]]; then ls $HOME/docker/*.tar.gz | xargs -I {file} sh -c "zcat {file} | docker load"; fi
+
+install:
+  - docker --version
+  - docker-compose --version
+  - sudo service postgresql stop
+  - while sudo lsof -Pi :5432 -sTCP:LISTEN -t; do sleep 1; done
+  - npm install
+  - npm run configure:development
+  - npm run codegen
+
+script:
+  - sudo docker-compose run subgraph test
+
+after_script:
+  - docker-compose down

Diff for: Dockerfile

@@ -0,0 +1,18 @@
+FROM node:8
+
+# install the 'host' command used to get ip of ipfs container
+RUN apt-get update -y && apt-get install dnsutils -y
+
+## Add the wait script to the image
+ADD https://github.com/ufoscout/docker-compose-wait/releases/download/2.4.0/wait /wait
+
+ADD ops/entry /entry
+RUN chmod +x /wait /entry
+
+WORKDIR /usr/app
+COPY . .
+RUN npm install
+RUN npm run codegen
+
+ENTRYPOINT [ "/entry" ]
+CMD [ "deploy" ]

Diff for: README.md

@@ -1,22 +1,60 @@
-# daostack-subgraph
-A DAOstack subgraph for graph-node
+# DAOstack subgraph
 
-# Get started
+DAOstack subgraph for [TheGraph](https://thegraph.com/) project.
 
-Develop locally:
+## Getting started
 
-1. `npm i` - install dependencies.
-2. `npm run codegen` - generate automatic typings.
-3. create a `.env` file with the following environment variables (**should be kept secret**):
-  - `POSTGRES_PASSWORD` - a secret string used as the PostgreSQL password for the created DB.
-  - `ETHEREUM` - url to an rpc ethereum node prefixed with the network name (e.g. `mainnet:http://localhost:8545`)
-4. (in a seperate terminal) `docker-compose up` ([install](https://docs.docker.com/compose/install/)) - run a local graph-node with the above config.
-6. `npm run deploy`/`npm run deploy:watch` - deploy the subgraph.
+1 `git clone https://github.com/daostack/subgraph.git && cd subgraph`
 
-Note: All persistent data for the node will be stored in the `./data` directory, remove this directory in order to reset the node.
+2. `npm install`
+3. `npm run configure:<development|mainnet>` - configure the project to use ganache or mainnet via infura.
 
-Exposed endpoints:
-- `http://localhost:8000/daostack` - GraphQL API.
-- `http://localhost:8020` - graph-node RPC server.
-- `postgresql://postgres:${POSTGRES_PASSWORD}@postgres:5432/postgres` - postgres DB connection.
-- `http://localhost:5001` - ipfs node api.
+All npm scripts can be called within a container using `docker-compose` with all dependencies and services set up:
+
+`docker-compose run subgraph -v $(pwd):/usr/app -v /usr/app/node_modules subgraph <command>`
+
+## Commands
+
+1. `configure:mainnet` - configure the project to run against mainnet.
+2. `configure:development` - configure the project to run against ganache.
+3. `migrate:development` - migrate contracts to ganache and update project configuration.
+4. `codegen` - automatically generate abi & type definitions for required contracts.
+5. `test` - run integration tests.
+6. `deploy` - deploy subgraph.
+7. `deploy:watch` - redeploy on file change.
+
+Example: `docker-compose run subgraph -v $(pwd):/usr/app -v /usr/app/node_modules subgraph test` (run intergation tests)
+
+To stop all services: `docker-compose down`
+
+## Exposed endpoints
+
+After running a command with docker-compose, the following endpoints will be exposed on your local machine:
+
+- `http://localhost:8000/daostack` - GraphiQL graphical user interface.
+- `http://localhost:8000/daostack/graphql` - GraphQL api endpoint.
+- `http://localhost:8001/daostack` - graph-node's websockets endpoint
+- `http://localhost:8020` - graph-node's RPC endpoint
+- `http://localhost:5001` - ipfs endpoint.
+- (if using development) `http://localhost:8545` - ganache RPC endpoint.
+- `http://localhost:5432` - postgresql connection endpoint.
+
+## Configuration
+
+This project automatically generates `.yaml` files used by `docker-compose` & `graph-node` based on configuration.
+Project configuration lives under: `ops/config.yaml` (public configration), `.env` (secret configuration).
+
+The following `.env` variables can be configured:
+
+- `daostack_mainnet__postgresPassword` - postgres password when running on mainnet (e.g `123`).
+- `daostack_mainnet__ethereumProvider` - mainnet web3 provider (e.g `https://mainnet.infura.io/v3/<api key>`)
+
+## Add a new contract tracker
+
+In order to add support for a new contract follow these steps:
+
+1. Create a mapping file at `src/<contract name>.ts`.
+2. Create a test file at `test/integration/<contract name>.spec.ts`.
+3. Configure the contract's mainnet address at `ops/config.yaml` under `addresses.<contract name>`.
+4. Add the contract to the migration script at the `migrate` function in `ops/index.js`.
+5. Add an additional datasource for the new contract at `subgraph.handlebars.yaml`, use `{{addresses.<contract name>}}` in place of the contract address.