Github's github:pages feature supports sharing any content in
a gh-pages repository branch as static web content.
The gh-pages branch of the Lab repository
is used to store the static pages and client-side code built by the Makefile at the directory public.
concord-consortium.github.com/lab
In addition the content of the gh-pages branch is used to create the
downloadable archive distributions of Lab
The contents of the gh-pages branch are automatically made available in a standard web-page form (as opposed to the standard Github page for showing a repository) at this url:
concord-consortium.github.com/lab
when you push to the gh-pages branch.
If you maintain a fork of this project on Github, you get a Github Page for free, and the instructions below apply to you as well!
If you haven't done this yet, make the public folder track the contents of the gh-pages branch.
If you have a Guard process running make sure and stop it before continuing!
# public/ needs to be empty for git clone to be happy:
rm -rf public
# substitute the URL for whatever fork of the Lab repository you have write access to:
git clone [email protected]:concord-consortium/lab.git -b gh-pages public
Note that make clean now empties the public folder in-place, leaving the Git
public/.git and public/jnlp directories intact.
First, make sure your public folder tracks the gh-pages branch, as per the above.
Then run the following shell command in the script/ folder:
script/gh-pages
This script will first make sure there is nothing that isn't committed. If
there are unstaged or staged and uncommitted files the gh-pages script will halt.
Test and commit (or save the changes to a topic branch) and if your testing show
the bugs are fixed or the new features or examples are stable then push
these changes to the master branch and try running the gh-pages script again:
git push origin master
script/gh-pages
The Lab project has a suite of scripts for creating and deploying to EC2 servers running on Amazon Web services (AWS).
The scripts for creating, stopping and re-creating a server require an AWS account. These scrpts use the Ruby proge
The Capistrano scripts for updating a deployment to an existing server only require that you have a copy of the pem file associated
To deploy to AWS you will need an AWS account and the ability to create and modify EC2 instances.
If you are create a new server from scratch you will also need access to the AWS DNS service Route53.
Note: currently there are a few steps/resources that only work properly if you have a concord.org AWS account.
-
AWS account. For deployment to the lab servers managed by CC you will need an AWS account managed by concord.org.
-
Create your AWS Access Key ID and AWS Secret Access Key. To do this, go to the IAM Dashboard in AWS (Services -> IAM), click 'Users', click the checkbox by your username, and select User Actions -> Manage Access Keys
-
Copy your AWS Access Key ID and AWS Secret Access Key to the following yaml configuration file
.fogin your home directory::default: :aws_access_key_id: YOUR_AWS_ACCESS_KEY_ID :aws_secret_access_key: YOUR_AWS_SECRET_ACCESS_KEY -
Place a copy of a the appropriate AWS PEM file on your local files system. For deployment to the lab servers managed by CC use the
lab-devpem. This can be found in CC's Google Docs/Drive. This file should have its permissions set to 600, and the folder it is in (e.g. ~/.ec2) should be 700. -
Create or identify an appropriate AWS security group. For deployment to the lab servers managed by CC the
lab.devsecurity group is used. -
Edit the
:deploysection ofconfig/config.ymlusing values inconfig/config.sample.ymlas a guide.Here's an example from the
:deploysection of a workingconfig.yml::deploy: :targets: - :name: lab-dev :url: lab.dev.concord.org :branch: master :group_name: lab.dev :zone_domain: concord.org. :zone_records_name: dev.concord.org :pem: :name: lab-dev :path: ~/.ec2There is one deploy target named
lab-devassociated with a server running atlab.dev.concord.org. Deployments tolab-devuse the master branch of the repository. Thelab.devsecurity group is used when new servers are created or existing sever are re-created.When a whole new server is created the DNS entry is created in the
concord.org.zone domain and when searching for an existing DNS entry for adeploy-targetthe zone record namedev.concord.orgis used.Besides the AWS Access Key ID and AWS Secret Access Key security credentials copyied locally to to the file
~/.fogthelab-dev.pemfile saved in the directory:~/.ec2is also used when communicating with AWS. -
List the deploy targets described in
config/config.ymlwith the task:thor cloud:list_targetsto confirm the configuration is valid:$ thor cloud:list_targets Deploy Targets name url branch ------------------------------------------------------------------------------------------ lab-dev lab.dev.concord.org master -
Generate specific Capistrano deploy tasks and littlechef nodes using
deploy-targetsspecified inconfig/config.yml. Run thisthortask whenever you change the:deploysection inconfig/config.ymlto generate the Ruby Capastrano configuration files inconfig/deployment/<deploy-target>.rband the littlechef JSON configurations inconfig/littlechef/nodes/<deploy-target>.json$ thor cloud:setup -
List the running AWS server instances to confirm that your local AWS security credentials are setup correctly:
$ thor cloud:list target hostname state ipaddress ec2-id ec2-dns ------------------------------------------------------------------------------------------------------------------------------------------------------ lab-dev lab.dev.concord.org. running 107.22.184.173 i-f844ec81 ec2-107-22-184-173.compute-1.amazonaws.com -
If you are working with an existing host that has already been setup such as
lab.dev.concord.orggenerate the proper ssh configuration and add the remote host key to~/.ssh/known_hosts. This adds a localubuntuuser in~/ssh/configand connects to the remote host to add the key.Example of setting up SSH configuration with the existing remote AWS host:
lab..dev.concord.org:$ thor cloud:setup_ssh lab.dev.concord.org -
You will have to manually edit config/config.yml on your new host.
SSH into your host, and edit the configuration file config/config.yml. The setup process should notify you of this (from aws-lab-server.rb): "If the new server provisioning with littlechef was successful login to the server with ssh and update the server hostname settings in config/config.yml".TODO: Should we write a tool to easily view/edit configuration files?
After testing, committing, and pushing code to a public repository use the Capistrano tasks to update a remote server.
The capistrano commands take the form:
cap <deploy-target> task
The basic command to update a server:
cap <deploy-target> deploy:update
Here are the list of current Capistrano deploy commands:
$ cap -T deploy:
cap deploy:restart # restart rails app
cap deploy:clean_and_update # clean and update server
cap deploy:setup # setup server
cap deploy:status # display last commit on deployed server
cap deploy:update # update server
cap deploy:update_jnlps # update public/jnlp dir on server
Update the lab.dev.concord.org server with the latest code committed on the master branch on
concord-consortium/lab:
cap lab-dev deploy:update
When you have made changes in the repository like adding or updating a git submodule in
src/vendor then you will need instead run the deploy:clean_and_update task:
cap lab-dev deploy:clean_and_update
The Java resources require much less frequent updates since the main body of work for Lab is occuriring in the HTML5 development.
The capistrano task: deploy:update_jnlps erases the public/jnlp/
directory on the remote server and re-generates and deploy the packed signed
jars from source or from downloads:
$ cap <deploy-target> deploy:update_jnlps
The resulting directory on the server will look something like this:
$ tree /var/www/app/public/jnlp/
public/jnlp/
└── org
└── concord
├── energy2d
│ ├── energy2d__V0.1.0-20120531.005123-1.jar
│ └── energy2d__V0.1.0-20120531.005123-1.jar.pack.gz
└── modeler
├── mw__V2.1.0-20120531.005123-1.jar
└── mw__V2.1.0-20120531.005123-1.jar.pack.gz
There are a set of thor tasks for managing, creating, and re-creating AWS servers for Lab:
$ thor -T
cloud
-----
thor cloud:create hostname # create a new server instance using this hostname
thor cloud:delete hostname # delete an existing server instance running at this hostname
thor cloud:find_dns_record hostname # find dns record for hostname
thor cloud:list # list existing servers
thor cloud:list_targets # list existing deploy targets
thor cloud:recreate hostname # recreate a new server instance for this hostname by destroying and rebuilding an existing server
thor cloud:setup # setup capistrano deploy tasks and littlechef nodes using targets in config/config.yml
thor cloud:setup_ssh hostname # setup ssh configuration for communication to hostname
thor cloud:start ec2_id # start a stopped existing server instance using the ec2-id
thor cloud:stop reference # stop a running existing server instance at this hostname or ec2-id
thor cloud:update reference # update server <ec2_id|hostname> provisioning with littlechef 'lab-server' role
thor cloud:update_dns_record hostname ipaddress # updating IP address for DNS record hostname to ipaddress
Creating a new Lab server on AWS consists of three steps:
-
Creating a new hostname, server, and provisioning the server with thor:
$ thor cloud:create <hostname>This task will create a new hostname as a DNS A record if the hostname does not already exists.
If the hostname already exists as a CNAME first login to the AWS:Route53 service and delete the existing host name.
If the new DNS entry for hostname is not properly propogated when the hostname is created or changed you will get an error that looks something like this:
*** running local command: echo ' Host <hostname> User ubuntu IdentityFile ~/.ec2/lab-dev.pem ' >> ~/.ssh/config *** running local command: ssh-keygen -R <hostname> /Users/stephen/.ssh/known_hosts updated. Original contents retained as /Users/stephen/.ssh/known_hosts.old *** running local command: ssh ubuntu@<hostname> -o StrictHostKeyChecking=no exit ssh: Could not resolve hostname <hostname>: nodename nor servname provided, or not known *** updating littlechef node: <hostname>.json *** provisioning <hostname> with littlechef role: lab-server <ec2-id>, <ec2-hostname>, <new-ip-address> command: cd /Users/stephen/dev/concord/lab/config/littlechef && fix node:<hostname> role:lab-server == Applying role 'lab-server' to <hostname> == Fatal error: Name lookup failed for <hostname> Underlying exception: nodename nor servname provided, or not knownYou will need to resolve the issue with getting the correct DNS record for hostname before continuing. After this is resolved you can follow these steps to continue the initial setup and provisioning:
$ ssh-keygen -R <hostname> $ ssh ubuntu@<hostname> -o StrictHostKeyChecking=no exit $ thor cloud:update <hostname> -
Optionally configure the server to use a valid Java code-siging certificate.
If you wish to support the integration of the optional Java resources that are required to be signed to work:
- legacy Molecular Worbench and Energy2D Java Web Start applications
- Java-based Vernier GoIO browser-sensor applet integration
You should put copy of a valid Java siging certificate keystore on hostname and edit
config/config.ymlto reference this keystore before runningcap <deploy-target> deploy:setupThe one supplied with the repository is a sample self-signed certificate and end user will be warned that it is not valid.
Here is one way to acomplish this:
$ scp <path-to-keystore> deploy@<hostname>:/var/www/app/config/<new-keystore-name>.jksNow ssh to the new host and edit the java section of
/var/www/app/config/config.ymlto update the values for:password,:alias, and:keystore_path. -
Finishing the setup of the server with a capistrano task
$ cap <deploy-target> deploy:setupThis completes the initial deploy and builds of all the project resources to the server.