Skip to content

Commit 03aa986

Browse files
author
Achille Roussel
committed
initial commit
0 parents  commit 03aa986

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

56 files changed

+4337
-0
lines changed

‎.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
example/keys
2+
.terraform
3+
*.tfstate
4+
*.tfstate*
5+
*.swp
6+
*~

‎License.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2016 Segment.io, Inc.
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎Makefile

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
2+
SRC = $(wildcard *.tf ./*/*.tf)
3+
platform := $(shell uname)
4+
pydeps := pyyaml boto3
5+
modules = $(shell ls -1 ./*.tf ./*/*.tf | xargs -I % dirname %)
6+
7+
tools := \
8+
./tools/pack-ami \
9+
./tools/roll-ami \
10+
./tools/tfvar-ami
11+
12+
tools := $(patsubst ./tools/%,/usr/local/bin/%,${tools})
13+
14+
# The install rule sets up the development environment on the machine it's ran
15+
# on.
16+
install: install-third-party-tools install-python-dependencies install-tools
17+
18+
ifeq (${platform},Darwin)
19+
install-third-party-tools:
20+
brew install packer terraform python3
21+
else
22+
install-third-party-tools:
23+
@echo "${platform} is a platform we have no presets for, you'll have to install the third party dependencies manually (packer, terraform, python3)"
24+
endif
25+
26+
ifeq (${platform},Darwin)
27+
install-python-dependencies:
28+
sudo -H pip install --upgrade ${pydeps}
29+
else
30+
install-python-dependencies:
31+
pip install --upgrade pyyaml boto3
32+
endif
33+
34+
install-tools: $(tools)
35+
36+
/usr/local/bin/%: ./tools/%
37+
install -S -m 0755 $< /usr/local/bin
38+
39+
amis:
40+
pack-ami build -p ./packer -t base -r
41+
42+
plan-ami:
43+
pack-ami plan -p ./packer -t ${template}
44+
45+
validate-ami:
46+
pack-ami validate -p ./packer -t ${template}
47+
48+
build-ami:
49+
pack-ami build -p ./packer -t ${template} | tee /tmp/build-ami.output
50+
@cat /tmp/build-ami.output | tfvar-ami ${template} > ${template)_ami.tfvars
51+
@rm -f /tmp/build-ami.output
52+
53+
test:
54+
@bash scripts/test.sh
55+
56+
docs.md: $(SRC)
57+
@bash scripts/docs.sh
58+
59+
.PHONY: install-third-party-tools install-python-dependencies build-ami plan-ami validate-ami amis

‎Readme.md

Lines changed: 268 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,268 @@
1+
# Segment Stack [![CircleCI](https://circleci.com/gh/segmentio/stack.svg?style=shield&circle-token=21d1df0dfd7e405582403f65cd1a270f9f52d7a4)](https://circleci.com/gh/segmentio/stack)
2+
3+
[terraform]: https://terraform.io
4+
[remote-state]: https://www.terraform.io/docs/commands/remote-config.html
5+
[aws-credentials]: http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#cli-quick-configuration
6+
[aws-vault]: https://github.com/99designs/aws-vault
7+
[aws]: http://aws.amazon.com/
8+
[docker-hub]: https://hub.docker.com/
9+
[keypair]: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#having-ec2-create-your-key-pair
10+
11+
The Segment Stack is a set of [Terraform][terraform] modules for configuring production infrastructure with AWS, Docker, and ECS.
12+
It's a more 'curated' set of defaults for configuring your AWS environment, while still allowing you to fully customize it.
13+
14+
*To get more background on the Segment Stack you can read [this blog post](https://segment.build/blog/the-segment-aws-stack/) about its history.*
15+
16+
The Stack comes with:
17+
18+
- an auto-scaling group of instances to run your services
19+
- a multi-az VPC with different subnets for availability
20+
- self-managed services run via docker and ECS
21+
- an ELB and ECS definition for each service
22+
- docker logs that populate in CloudWatch
23+
- a bastion node for manual SSH access
24+
- automatic ELB logging to S3
25+
26+
Start from scratch or selectively add it to your existing infrastructure, the Stack is yours to customize and tweak.
27+
28+
## Quickstart
29+
30+
_To run the stack, you'll need AWS access and Terraform installed, check out the [requirements](#Requirements) section._
31+
32+
The easiest way to get the Stack up and running is by creating a Terraform definition for it, copy this snippet in a file
33+
named `terraform.tf`:
34+
```hcl
35+
module "stack" {
36+
source = "github.com/segmentio/stack"
37+
environment = "prod"
38+
key_name = "my-key-name"
39+
name = "my-app"
40+
}
41+
```
42+
This is the _base_ configuration, that will provision everything you need to run your services.
43+
44+
From there, you'll want to plan, which will stage the changeset
45+
46+
$ terraform plan
47+
48+
And if the changes look good, apply them to your infrastructure
49+
50+
$ terraform apply
51+
52+
This will automatically setup your basic networking configuration with an auto-scaling default cluster running ECS.
53+
54+
Now that we've got all the basics setup, how about adding a service?
55+
56+
Services pull images from Docker Hub and then run the images as contianers via ECS. They are automatically discoverable at `<service-name.stack.local>` and will run with zero-downtime deploys.
57+
We can can use the `stack//service` module to automatically provision all of the required parts of the service, including a load balancer, ECS service, and Route53 DNS entry.
58+
59+
Here's a sample service definition, try adding it to your `terraform.tf` file.
60+
61+
```hcl
62+
module "nginx" {
63+
# this sources from the "stack//service" module
64+
source = "github.com/segmentio/stack//service"
65+
environment = "prod"
66+
name = "my-app"
67+
image = "nginx"
68+
}
69+
```
70+
71+
Once the nginx service has been added, simply run another plan and apply:
72+
73+
$ terraform plan
74+
$ terraform apply
75+
76+
Your service should automatically be up and running. You can SSH into your bastion host (find the ip by running `terraform output`) and connect using the service name:
77+
78+
$ ssh -i <path-to-key> ubuntu@<bastion-ip>
79+
$ curl http://nginx.stack.local/
80+
81+
*The bastion IP should have been shown by the terraform output when it created the stack for the first time. If you missed it you can still get it from the AWS console.*
82+
83+
## Requirements
84+
85+
Before we start, you'll first need:
86+
87+
- [ ] an [AWS account][aws] with API access
88+
- [ ] locally configured [AWS credentials][aws-credentials] or a tool like [aws-vault][aws-vault]
89+
- [ ] to [create a keypair][keypair] in AWS
90+
- [ ] Docker images of your services uploaded to [Docker Hub][docker-hub]
91+
- [ ] download and install [terraform][terraform]
92+
93+
## Architecture
94+
95+
At a high level, the Stack creates a multi-az VPC, a single auto-scaling cluster, and service definitions within ECS.
96+
97+
![](./images/stack.png)
98+
99+
Your instances are automatically distributed across the VPC, addresses are translated by NAT gateways, and services are all discoverable via route53 and ELBs.
100+
101+
We'll walk through how each of these fit together in this architecture document.
102+
103+
### Networking
104+
105+
![](./images/networking.png)
106+
107+
By default, the Stack will create a VPC in a single region, amongst multiple availability zones (AZs). The default mask for this VPC is
108+
109+
10.30.0.0/16
110+
111+
The address was chosen to be internal, and to not conflict with other pieces of infrastructure you might run. But, it can also be configured with its own CIDR range.
112+
113+
Each availability zone will get its own external and internal subnets. Most of our infrastructure will live in the *internal* subnet so that they are not externally accessible to the internet.
114+
115+
If you'd like to scale to multiple regions (outside the scope of the current stack), simply add one to the second octet.
116+
117+
10.31.0.0/16 -- my new region
118+
119+
To span across availability zones, the regional 16-bit mask becomes 18-bits.
120+
121+
10.30.0.0/18 - AZ A
122+
10.30.64.0/18 - AZ B
123+
10.30.128.0/18 - AZ C
124+
10.30.192.0/18 - Spare
125+
126+
To subdivide each availability zone into spaces for internal, external and to have spare room for growth; use a 19-bit mask for internal, and a 20-bit mask for external. The external space is smaller because only a few instances and load-balancers should be provisioned into it.
127+
128+
10.30.0.0/18 - AZ A
129+
130+
10.30.0.0/19 internal
131+
10.30.32.0/20 external
132+
10.30.48.0/20 spare
133+
134+
10.30.64.0/18 - AZ B
135+
136+
10.30.64.0/19 internal
137+
10.30.96.0/20 external
138+
10.30.112.0/20 spare
139+
140+
10.30.128.0/18 - AZ C
141+
142+
10.30.128.0/19 internal
143+
10.30.160.0/20 external
144+
10.30.176.0/20 spare
145+
146+
The VPC itself will contain a single network gateway to route
147+
traffic in and out of the different subnets. The Stack terraform will automatically create 3 separate [NAT Gateways][nat-gateway] in each of the different subnets.
148+
149+
Traffic from each internal subnet to the outside world will run through the associated NAT gateway.
150+
151+
For further reading, check out these sources:
152+
153+
- [Recommended Address Space](http://serverfault.com/questions/630022/what-is-the-recommended-cidr-when-creating-vpc-on-aws)
154+
- [Practical VPC Design](https://medium.com/aws-activate-startup-blog/practical-vpc-design-8412e1a18dcc)
155+
156+
[nat-gateway]: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-nat-gateway.html
157+
158+
### Instances
159+
160+
![](./images/instance.png)
161+
162+
Each instance in an ecs-cluster is provisioned using an AMI built in the `./packer` directory. By default, this AMI is based off the 16.04 Ubuntu LTS image, and runs all the base programs within **systemd**.
163+
164+
After boot, systemd will run each of its targets, which includes booting Docker and the ECS agent. The ECS agent will register the instance with a particular cluster, pulled from the environment variables on the instance.
165+
166+
### Services
167+
168+
![](./images/service.png)
169+
170+
Stack services run within ECS. They include a few key pieces:
171+
172+
- an ECS task definition
173+
- an ECS service definition
174+
- an internal ELB
175+
- an internal route53 entry
176+
177+
The task definition tells ECS _what_ docker image to run (nginx), and _how_ to run it (env vars, arguments, etc). The service definition tells ECS how many containers of a task to run, and on which cluster to run the containers. The ELB routes traffic to the containers in a service, and route53 assigns a 'nice' name to the ELB.
178+
179+
Service discovery works via vanilla DNS. Whenever a service is provisioned, it will also create an accompanying ELB that routes to the containers in the service. The route53 entry for the ELB provisioned by the 'auth' service would be:
180+
181+
$ curl http://auth.stack.local
182+
183+
For more complicated service discovery which handles cases like versioning, we'd recommend using a service like [Consul][consul] or [etcd][etcd].
184+
185+
[consul]: https://www.consul.io/
186+
[etcd]: https://github.com/coreos/etcd
187+
188+
### Bastion
189+
190+
The bastion host acts as the "jump point" for the rest of the infrastructure. Since most of our instances aren't exposed to the external internet, the bastion acts as the gatekeeper for any direct SSH access.
191+
192+
The bastion is provisioned using the key name that you pass to the stack (and hopefully have stored somewhere). If you ever need to access an instance directly, you can do it by "jumping through" the bastion:
193+
194+
$ terraform output # print the bastion ip
195+
$ ssh -i <path/to/key> ubuntu@<bastion-ip> ssh ubuntu@<internal-ip>
196+
197+
### Logging
198+
199+
The default AMI that instances of the ECS cluster are running ships with the ecs-agent and a program called ecs-logs pre-configured. While ecs-agent takes care of scheduling services, ecs-logs is in charge of reading the service logs and uploading them to CloudWatch.
200+
This is all configured automatically by the default Stack settings.
201+
202+
ecs-logs creates one CloudWatch Logs Group for each service, then in each of the groups, a CloudWatch Logs Stream named after the docker container running the service will hold all the logs generated by the service.
203+
204+
If you're interested in digging further into how ecs-logs work here is the github repository where it's hosted:
205+
206+
- https://github.com/segmentio/ecs-logs
207+
208+
## Module Reference
209+
210+
To see the full reference for each individual module, see our [reference page](./docs.md).
211+
212+
You can reference modules individually by name:
213+
214+
```hcl
215+
module "vpc" {
216+
source = "github.com/segmentio/stack//vpc"
217+
name = "${var.name}"
218+
environment = "${var.environment}"
219+
cidr = "${var.cidr}"
220+
internal_subnets = "${var.internal_subnets}"
221+
external_subnets = "${var.external_subnets}"
222+
availability_zones = "${var.availability_zones}"
223+
}
224+
```
225+
226+
## Developing
227+
228+
You can customize any part of the stack you'd like.
229+
230+
### AMIs
231+
232+
All of the default AMIs that ship with stack are build using [packer][packer]. If you'd like to build your own, you can make changes to the `./packer` directory and then re-build using:
233+
234+
$ make amis
235+
236+
[packer]: https://www.packer.io
237+
238+
### Terraform
239+
240+
Stack is all vanilla Terraform and AWS, so you can customize it by simply forking the repository and referencing your own modules internally.
241+
242+
## Examples
243+
244+
To dig further down into what you can build with the Segment Stack we have put together an example app that shows how to configure a small infrastructure from scratch:
245+
246+
- https://github.com/segmentio/pingdummy
247+
248+
## Authors
249+
250+
- [Calvin French-Owen](github.com/calvinfo)
251+
- [Amir Aboushareb](https://github.com/yields)
252+
- [Achille Roussel](https://github.com/achille-roussel)
253+
- [Kevin Lo](https://github.com/liquidy)
254+
- [Rick Branson](https://github.com/rbranson)
255+
256+
## License
257+
258+
Released under the MIT License
259+
260+
(The MIT License)
261+
262+
Copyright (c) 2016 Segment [email protected]
263+
264+
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
265+
266+
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
267+
268+
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

‎base_ami.tf

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
variable "base_ami" {
2+
default = "ami-729c5912"
3+
}

0 commit comments

Comments
 (0)