End 2 end tests (jsperf#134)

* move unit test to unit direcotry

* fix failing health check responding 200 instead of 500

* add end to end test structure

Author: maxbeatty

Diff for: .dockerignore

@@ -3,11 +3,10 @@ node_modules
 .dockerignore
 .editorconfig
 .env
-.eslintignore
-.eslintrc
 .gitattributes
 .gitignore
 .nvmrc
 *.md
 docker-compose.yml
+docker-compose.e2e.yml
 Dockerfile

Diff for: .eslintignore

@@ -1,2 +1,3 @@
 node_modules
 public/_js
+tmp*

Diff for: Dockerfile

@@ -1,10 +1,12 @@
 FROM node:5.11.1
 
-ADD ./package.json /code/package.json
 WORKDIR /code
+
+COPY ./package.json /code/package.json
+
 RUN npm install
 
-ADD . /code
+COPY . /code
 
 EXPOSE 3000
 

Diff for: Dockerfile-e2e-runner

@@ -0,0 +1,13 @@
+FROM mhart/alpine-node:5.11.1
+
+WORKDIR /code
+
+COPY ./package.json /code/package.json
+
+RUN npm install --only=dev
+
+COPY .eslint* /code/
+COPY ./test/e2e /code/test/e2e
+
+# CMD ["npm", "run", "test-e2e"]
+CMD ["/bin/true"]

Diff for: README.md

@@ -13,7 +13,7 @@
 2. Install dependencies: `npm install`
 3. Get a [Browserscope.org](http://www.browserscope.org/) API key by signing in and going to [the settings page](http://www.browserscope.org/user/settings). (You'll need this in the last step)
 4. Register a new OAuth GitHub development application by going to [your settings page in github](https://github.com/settings/applications/new). Take note to copy the "Client ID" and "Client Secret". The callback URL is simply the root url of the application, e.g., `http://localhost:3000`
-5. Setup environment configuration: `npm run setup`
+5. Setup environment configuration: `node setup`
 
 ### Running the server
 
@@ -23,9 +23,13 @@ jsPerf is available at `localhost` and changes to the codebase can be applied by
 
 ##### One-time Setup
 
+_`$MYSQL_PASSWORD` is whatever you chose or generated during `node setup`_
+
 1. Install [Docker Toolbox](https://docs.docker.com/engine/installation/) so you have `docker` and `docker-compose`
 2. Create a Data Volume Container to persist data: `docker create -v /var/lib/mysql --name data-jsperf-mysql mysql /bin/true`
-3. After completing the "Compose" steps below, setup database tables with: `docker-compose run web node /code/setup/tables`
+3. Start Docker Compose in detached mode: `MYSQL_PASSWORD=$MYSQL_PASSWORD docker-compose up`
+4. Setup database tables with: `MYSQL_PASSWORD=$MYSQL_PASSWORD docker-compose run web node /code/setup/tables`
+5. Verify everything is working: `open http://$(docker-machine ip)`
 
 ##### Compose
 
@@ -89,19 +93,34 @@ _If you'd just like to lint and save a little time, you can run `npm run lint` w
 
 _If you're missing code coverage, open `coverage.html` in the root of the project for a detailed visual report._
 
+### End to End
+
+End to end (e2e) testing is done with Selenium. There is a separate Docker Compose file to define the Selenium Grid Hub, Selenium Nodes, and the test runner. Running the e2e test suite is a three step process:
+
+1. Build containers: `docker-compose -f docker-compose.yml -f docker-compose.e2e.yml build`
+2. Start the app along with Selenium: `MYSQL_PASSWORD=$MYSQL_PASSWORD docker-compose -f docker-compose.yml -f docker-compose.e2e.yml up -d`
+  - To scale up available Selenium Nodes to make testing faster, run: `docker-compose -f docker-compose.yml -f docker-compose.e2e.yml scale firefox=5` (_5 is the number of VMs Sauce Labs gives open source projects_)
+3. Run the test suite: `docker-compose -f docker-compose.yml -f docker-compose.e2e.yml run --rm runner npm run test-e2e`
+
+When you're done, you can safely stop and remove all of the containers by running:
+
+```
+docker-compose -f docker-compose.yml -f docker-compose.e2e.yml down
+```
+
 ## Gotchas
 
 - ES6 Template Strings are not supported by esprima which means you can't generate coverage reports which means `npm test` won't pass.
 
 ### Adding new dependencies
 
 1. Install using `npm` and either `--save` or `--save-dev`. Do not edit `package.json` manually.
-2. Run `npm run shrinkwrap` to update `npm-shrinkwrap.json`
+2. Run `npm shrinkwrap --dev` to update `npm-shrinkwrap.json`
 
-If you get an error while shrinkwrapping, try removing what you have installed currently, reinstalling based on `package.json` instead of `npm-shrinkwrap.json`, and then shrinkwrap again.
+If you get an error while shrinkwrapping, try pruning your `node_modules` directory by running `npm prune`. If that doesn't work, try removing what you have installed currently, reinstalling based on `package.json` instead of `npm-shrinkwrap.json`, and then shrinkwrap again.
 
 ```
-rm -r node_modules/ && npm install --ignore-shrinkwrap && npm run shrinkwrap
+rm -r node_modules/ && npm install --no-shrinkwrap && npm shrinkwrap --dev
 ```
 
 ## Debugging

Diff for: docker-compose.e2e.yml

@@ -0,0 +1,20 @@
+hub:
+  image: selenium/hub:2.46.0
+  ports:
+    - '4444:4444'
+
+firefox:
+  image: selenium/node-firefox:2.46.0
+  restart: on-failure
+  links:
+    - hub
+    - lb # from main docker-compose.yml
+
+runner:
+  build: .
+  dockerfile: Dockerfile-e2e-runner
+  environment:
+    SELENIUM_REMOTE_URL: http://hub:4444/wd/hub
+    JSPERF_REMOTE_URL: http://lb
+  links:
+    - hub

Diff for: docker-compose.yml

@@ -12,12 +12,12 @@ web:
     SERVICE_3000_CHECK_HTTP: '/health'
     SERVICE_3000_CHECK_INTERVAL: '1s'
 db:
-  image: mysql
+  image: mysql:5.7
   volumes_from:
     - data-jsperf-mysql
   env_file: .env
   environment:
-    MYSQL_RANDOM_ROOT_PASSWORD: 'True'
+    MYSQL_RANDOM_ROOT_PASSWORD: 'yes'
     MYSQL_PASSWORD: $MYSQL_PASSWORD
     MYSQL_DATABASE: jsperf
     MYSQL_USER: jsperf