Initial Code Commit

Co-authored-by: IllyaYalovyy <[email protected]>
Co-authored-by: nbraid <[email protected]>
Co-authored-by: tneely <[email protected]>
Co-authored-by: wleepang <[email protected]>
Co-authored-by: markjschreiber <[email protected]>
Co-authored-by: AndreyDovydenko <[email protected]>
Co-authored-by: elliot-smith <[email protected]>
Co-authored-by: AhmadBassyiouni <[email protected]>
Co-authored-by: mlin <[email protected]>

Author: 10 people

.gitignore

@@ -1,2 +1,12 @@
+/.idea/
+/output/
+codebuild_build.sh
 .DS_Store
-
+venv
+.venv
+*.iml
+coverage.out
+/packages/common/bin/
+vendor
+/docs/
+/site/content/en/docs/Reference/agc*

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "site/themes/docsy"]
+	path = site/themes/docsy
+	url = https://github.com/google/docsy.git

.go-version

@@ -0,0 +1 @@
+1.16.0

.nvmrc

@@ -0,0 +1 @@
+14

CODE_OF_CONDUCT.md

@@ -1,4 +1,4 @@
 ## Code of Conduct
 This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
 For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
-[email protected] with any additional questions or comments.
+[email protected] with any additional questions or comments.

CONTRIBUTING.md

@@ -11,13 +11,27 @@ information to effectively respond to your bug report or contribution.
 
 We welcome you to use the GitHub issue tracker to report bugs or suggest features.
 
-When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already
+When filing an issue, please check [existing open](https://github.com/aws/amazon-genomics-cli/issues), 
+or [recently closed](https://github.com/aws/amazon-genomics-cli/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), 
+issues to make sure somebody else hasn't already
 reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
 
-* A reproducible test case or series of steps
-* The version of our code being used
-* Any modifications you've made relevant to the bug
-* Anything unusual about your environment or deployment
+* Environment
+  * AGC version
+  * OS version
+  * Node version
+  * Shell (bash/zsh) and shell version
+* Steps to reproduce the error
+  * Sequence of commands used
+  * Demonstration workflow or inputs if applicable
+* Expected result
+* Actual result
+* Is if the issue reproducible on the latest version of AGC?
+* A copy of the `agc-project.yaml`
+* Workflow, engine, adapter, and access logs
+* AWS region(s) where the issue was observed
+* Was a custom S3 bucket or custom VPC used? If so what differences are there from the standard deployed by the CDK code of AGC
+
 
 
 ## Contributing via Pull Requests
@@ -31,17 +45,22 @@ To send us a pull request, please:
 
 1. Fork the repository.
 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
-3. Ensure local tests pass.
+3. Run `make` to ensure local tests pass and to automate standard formatting and linting.
 4. Commit to your fork using clear commit messages.
 5. Send us a pull request, answering any default questions in the pull request interface.
 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
 
+If your contribution requires custom container images for components such as engines or WES adapters please ensure these
+are also made available via an appropriate mechanism.
+
 GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
 [creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
 
 
 ## Finding contributions to work on
-Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start.
+Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the 
+default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), 
+looking at any ['help wanted'](https://github.com/aws/amazon-genomics-cli/labels/help%20wanted) issues is a great place to start.
 
 
 ## Code of Conduct
@@ -51,9 +70,15 @@ [email protected] with any additional questions or comments.
 
 
 ## Security issue notifications
-If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
+If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via 
+our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). 
+Please do **not** create a public github issue.
 
 
 ## Licensing
 
-See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
+See the [LICENSE](https://github.com/https://github.com/aws/amazon-genomics-cli/blob/main/LICENSE) file for our 
+project's licensing. We will ask you to confirm the licensing of your contribution.
+
+We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) 
+for larger changes.

Makefile

@@ -0,0 +1,47 @@
+VERSION := $(shell git describe --always --tags)
+PACKAGES := $(shell ls -d ${PWD}/packages/*/ | grep -v -E "(vendor|api)")
+
+.PHONY: test build build-cli release release-cli release-cdk $(PACKAGES)
+
+all: format test build
+
+format:
+	target=format $(MAKE) $(PACKAGES)
+
+test:
+	target=test $(MAKE) $(PACKAGES)
+
+$(PACKAGES):
+	(cd $@; $(MAKE) $(target))
+
+build: build-cli
+
+build-cli:
+	(cd packages/cli; $(MAKE) build)
+
+release: release-cli release-cdk
+
+release-cli:
+	(cd packages/cli; $(MAKE) release)
+
+release-cdk:
+	(cd packages/cdk; $(MAKE) release)
+
+init:
+	go env -w GOPROXY=direct
+	goenv install --skip-existing
+	target=init $(MAKE) $(PACKAGES)
+
+docs: build-cli
+	packages/cli/bin/local/agc --docs site/content/en/docs/Reference/
+	git submodule update --init --recursive
+	cd site && npm install && hugo
+
+clean-docs:
+	rm -f site/content/en/docs/Reference/agc*.md
+	rm -rf docs
+
+start-docs: build-cli
+	packages/cli/bin/local/agc --docs site/content/en/docs/Reference/
+	git submodule update --init --recursive
+	cd site && npm install && hugo server -D

README.md

@@ -1,15 +1,184 @@
-## My Project
+# Amazon Genomics CLI
 
-TODO: Fill this README out!
+## Overview
 
-Be sure to:
+The AWS Genomics Command Line Interface (AGC) is a tool to simplify the processes of deploying the AWS infrastructure required to
+run genomics workflows in the cloud, to submit those workflows to run, and to monitor the logs and outputs of those workflows.
 
-* Change the title in this README
-* Edit your repository description on GitHub
+## Quick Start
+
+To get an introduction to AGC refer to the [Quick Start Guide](https://github.com/aws/amazon-genomics-cli/wiki/Quick-Start-Guide)
+in our wiki.
+
+## Further Reading
+
+For full documentation, please refer to our [wiki](https://github.com/aws/amazon-genomics-cli/wiki).
+
+## Releases
+
+All releases can be accessed on our releases [page](https://github.com/aws/amazon-genomics-cli/releases).
+
+## Development
+
+To build from source you will need to ensure the following prerequisites are met.
+
+### One-time setup
+
+There are a few prerequisites you'll need to install on your machine before you can start developing.
+
+Once you've installed all the dependencies listed here, run `make init` to install the rest.
+
+
+#### Go
+
+The Amazon Genomics CLI is written in Go.
+
+To manage and install Go versions, we use [goenv](https://github.com/syndbg/goenv). Follow the installation
+instructions [here](https://github.com/syndbg/goenv/blob/master/INSTALL.md).
+
+#### Node
+
+AGC makes use of the AWS CDK to deploy infrastructure into an AWS account. Our CDK code is written in TypeScript.
+You'll need Node to ensure the appropriate dependencies are installed at build time.
+
+To manage and install Node versions, we use [nvm](https://github.com/nvm-sh/nvm).
+
+```bash
+curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash
+echo 'export NVM_DIR="$([ -z "${XDG_CONFIG_HOME-}" ] && printf %s "${HOME}/.nvm" || printf %s "${XDG_CONFIG_HOME}/nvm")"' >> ~/.bashrc
+echo '[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm' >> ~/.bashrc
+source ~/.bashrc
+```
+
+Note: If you are using Zsh, replace `~/.bashrc` with `~/.zshrc`.
+
+#### Sed (OSX)
+
+OSX uses an outdated version of [sed](https://www.gnu.org/software/sed/manual/sed.html). If you are on a Mac, you will
+need to use a newer version of sed to ensure script compatibility.
+
+```bash
+brew install gnu-sed
+echo 'export PATH="/usr/local/opt/gnu-sed/libexec/gnubin:$PATH"' >> ~/.bashrc
+source ~/.bashrc
+```
+
+Note: If you are using Zsh, replace `~/.bashrc` with `~/.zshrc`.
+
+#### One time setup 
+
+Once you've installed all the dependencies listed here, run `make init` to automatically install all remaining dependencies.
+
+
+#### Make
+
+We use `make` to build, test and deploy artifacts. To build and test issue the `make` command from the project root.
+
+### Running Development Code
+
+To run against a development version of AGC, first build your relevant changes and then run `./scripts/run-dev.sh`. This will
+set the required environment variables and then enter into an AGC command shell.
+
+If you want to run from development code manually, ensure you have the following environment variables set.
+
+```shell
+export ECR_CROMWELL_ACCOUNT_ID=<some-value>
+export ECR_CROMWELL_REGION=<some-value>
+export ECR_CROMWELL_TAG=<some-value>
+export ECR_WES_ACCOUNT_ID=<some-value>
+export ECR_WES_REGION=<some-value>
+export ECR_WES_TAG=<some-value>
+```
+
+These environment variables point to the ECR account, region, and tags of the Cromwell engine and WES adaptor containers respectively
+that will be deployed for your contexts. They are written as Systems Manager Parameter Store variables when you activate
+your AGC account region (`agc account activate`). The `./scripts/run-dev.sh` contains logic to determine the current
+dev versions of the images which you would typically use. You may also use production images, the current values of which will
+be written when you activate an account with the production version of AGC. If you have customized containers that you 
+want to develop against you can specify these however you will need to make these available if you wish to make pull requests
+with code that depends on them.
+
+### Building locally with CodeBuild
+
+This package is buildable with AWS CodeBuild. You can use the AWS CodeBuild agent to run CodeBuild builds on a local
+machine.
+
+You only need to set up the build image the first time you run the agent, or when the image has changed. To set up the
+build image, use the following commands:
+
+```bash
+git clone https://github.com/aws/aws-codebuild-docker-images.git
+cd aws-codebuild-docker-images/ubuntu/standard/5.0
+docker build -t aws/codebuild/standard:5.0 .
+docker pull amazon/aws-codebuild-local:latest --disable-content-trust=false
+```
+
+Create an environment file (e.g. `env.txt`) with the appropriate entries depending on which image tags you want to use.
+
+```shell
+CROMWELL_ECR_TAG=2021-06-17T23-48-54Z
+WES_ECR_TAG=2021-06-17T23-48-54Z
+```
+
+In the root directory for this package, download and run the CodeBuild build script:
+
+```bash
+wget https://raw.githubusercontent.com/aws/aws-codebuild-docker-images/master/local_builds/codebuild_build.sh
+chmod +x codebuild_build.sh
+./codebuild_build.sh -i aws/codebuild/standard:5.0 -a ./output -c -e env.txt
+```
+
+### Configuring docker image location
+
+The default values for all variables are placeholders (e.g. 'WES_ECR_TAG_PLACEHOLDER'). It is replaces by the actual
+value during a build process.
+
+#### WES adapter for Cromwell
+
+Local environment variables:
+
+- `ECR_WES_ACCOUNT_ID`
+- `ECR_WES_REGION`
+- `ECR_WES_TAG`
+
+The corresponding AWS Systems Manager Parameter Store property names:
+
+- /agc/_common/wes/ecr-repo/account
+- /agc/_common/wes/ecr-repo/region
+- /agc/_common/wes/ecr-repo/tag
+
+#### Cromwell engine
+
+Local environment variables:
+
+- `ECR_CROMWELL_ACCOUNT_ID`
+- `ECR_CROMWELL_REGION`
+- `ECR_CROMWELL_TAG`
+
+The corresponding AWS Systems Manager Parameter Store property names:
+
+- /agc/_common/cromwell/ecr-repo/account
+- /agc/_common/cromwell/ecr-repo/region
+- /agc/_common/cromwell/ecr-repo/tag
+
+## Contributing
+
+### Issues
+
+See [Reporting Bugs/Feature Requests](CONTRIBUTING.md#reporting-bugsfeature-requests) for more information. For a list of
+open bugs and feature requests, please refer to our [issues](https://github.com/aws/amazon-genomics-cli/issues?q=is%3Aopen+is%3Aissue) page.
+
+### Pull Requests
+
+See [Contributing via Pull Requests](CONTRIBUTING.md#contributing-via-pull-requests)
 
 ## Security
 
-See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.
+See [Security Issue Notification](CONTRIBUTING.md#security-issue-notifications) for more information.
+
+## Attributions
+
+TODO, link to attribution docs
 
 ## License
 

buildspec.yml

@@ -0,0 +1,41 @@
+version: 0.2
+
+env:
+  shell: bash
+  variables:
+    # These variables should be over-ridden as appropriate by the CI/CD pipeline
+    BUCKET_NAME: "healthai-public-assets-us-east-1"
+    WES_ECR_TAG: ""
+    CROMWELL_ECR_TAG: ""
+    NEXTFLOW_ECR_TAG: ""
+
+phases:
+  install:
+    commands:
+      - goenv install
+      - go env -w GOPROXY=direct
+      - make init
+  pre_build:
+    commands:
+      - make test
+      - SEMANTIC_VERSION=$(./scripts/version.sh "$BUCKET_NAME")
+      - echo "Using semantic version '${SEMANTIC_VERSION}'"
+      - ./scripts/update-tags.sh
+  build:
+    on-failure: ABORT
+    commands:
+      - make release
+  post_build:
+    commands:
+      # Package Amazon Genomics CLI
+      - mkdir -p tmp
+      - cp -a scripts/cli/. tmp
+      - cp -a examples tmp
+      - cp -a packages/cli/bin/local/. tmp
+      - cp packages/cdk/cdk.tgz tmp
+      - mkdir -p amazon-genomics-cli amazon-genomics-cli/$SEMANTIC_VERSION
+      - (cd tmp && zip -r ../amazon-genomics-cli/$SEMANTIC_VERSION/amazon-genomics-cli.zip ./*)
+
+artifacts:
+  files:
+    - "amazon-genomics-cli/**/*"