Skip to content

Commit 5e4ab3c

Browse files
denstortidenstorti
authored
Class07/content: Serverless part 2 (#1438)
* Move SSM to class4. Move KMS to class05. Fix scope class06, class07 * Small fixes in the exercise * update commit sha * class 7 content WIP * Add SQS queue * Add exercise details * Add Cloudwatch and S3 * Fix exercise image path * Format exercises * Format ex 1 in class 7 * Update ex sha * Fix titles * Updating commit shas
1 parent 157dd85 commit 5e4ab3c

25 files changed

+479
-23
lines changed

‎README.md

+10-9
Original file line numberDiff line numberDiff line change
@@ -162,6 +162,7 @@ The content can be accessed through each specific README link below.
162162
- AWS ECS (Elastic Container Service)
163163
- AWS ECR (Elastic Container Repository)
164164
- AWS Fargate
165+
- AWS SSM - Parameter Store
165166
- IaC - Terraform
166167
- Project #1 Kick-off
167168
- [Class 05](classes/05class/README.md)
@@ -171,24 +172,25 @@ The content can be accessed through each specific README link below.
171172
- CI/CD Concepts
172173
- GitHub Actions
173174
- AWS CloudWatch
175+
- AWS KMS (Key Management Service)
174176
- [Class 06](classes/06class/README.md)
175177
- Project #1 Delivery
176178
- Serverless introduction
177-
- AWS Lambda
178179
- AWS API Gateway
179-
- Project #2 Kick-off
180-
- [Class 07](classes/07class/README.md)
180+
- AWS Lambda
181181
- AWS DynamoDB
182-
- AWS KMS (Key Management Service)
183-
- AWS SSM - Parameter Store
182+
- [Class 07](classes/07class/README.md)
183+
- Serverless integration patterns
184+
- AWS DynamoDB Streams
184185
- AWS SNS
185186
- AWS SQS
187+
- AWS S3 Events
188+
- AWS CloudWatch Events (EventBridge)
186189
- [Class 08](classes/08class/README.md)
187-
- Project #2 Delivery
190+
- AWS Certifications
188191
- Next steps
189192
- Course Closure
190193

191-
192194
## Exercises
193195

194196
We highly recommend doing the exercises from this repository. However, you will first need to raise a Pull Request from a [fork](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) to add yourself permission to push branches.
@@ -321,7 +323,7 @@ make dashboard \
321323

322324
## Presentation format
323325

324-
* We are using plain README.md files with markdown or [GitPitch](https://gitpitch.com/docs/markdown-features/basics/) for slideshow presentations
326+
* We are using plain `README.md` files with markdown or [GitPitch](https://gitpitch.com/docs/markdown-features/basics/) for slideshow presentations
325327

326328
* Generate the presentation by running:
327329
* `make presentation`
@@ -334,7 +336,6 @@ make dashboard \
334336
* Offline: use a [Docker container](https://github.com/kns-it/Docker-GitPitch/), run `make presentation` and `open http://localhost:9000/${ORG_NAME}/${REPO_NAME}/${BRANCH_NAME}?p=${FOLDER_NAME}`
335337
For running in a specific folder: `FOLDER_NAME=class2 make presentation`
336338

337-
338339
## Clean up Ready to Review tag from Closed PRs
339340

340341
To clean up the `Read to Review` tags of the closed PRs, the below make target can be executed.

‎classes/04class/README.md

+1
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
## Agenda
44

55
- [ECS, ECR and Fargate](/classes/04class/ecs_ecr_fargate/README.md)
6+
- [AWS SSM Parameter Store](/classes/04class/ssm/README.md)
67
- [IaC](/classes/04class/iac/README.md)
78
- [Project #1 Introduction and Kick off](/projects/project01/README.md)
89

‎classes/07class/ssm/README.md ‎classes/04class/ssm/README.md

+5-3
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,11 @@ This class is an introduction to AWS System Manager.
33
The main goal is to explain the purpose, the benefits and how to use it.
44

55
***Contents***
6-
- [What is AWS SSM?](#what-is-aws-ssm)
7-
- [SSM Parameter Store](#ssm-parameter-store)
8-
- [Main Use cases](#main-use-cases)
6+
- [System Manager](#system-manager)
7+
- [What is AWS SSM?](#what-is-aws-ssm)
8+
- [SSM Parameter Store](#ssm-parameter-store)
9+
- [Main use cases](#main-use-cases)
10+
- [Appendix](#appendix)
911

1012
## What is AWS SSM?
1113
AWS Systems Manager is an AWS service that you can use to view and control your infrastructure on AWS. Using the Systems Manager console, you can view operational data from multiple AWS services and automate operational tasks across your AWS resources.

‎classes/06class/README.md

+1-2
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,7 @@
66
- [AWS API Gateway](/classes/06class/apigateway/README.md)
77
- [AWS Lambda](/classes/06class/lambda/README.md)
88
- [AWS DynamoDB - NoSQL database](/classes/06class/dynamodb/README.md)
9-
- [AWS KMS - Key Management](/classes/06class/kms/README.md)
10-
- [AWS SSM Parameter Store](/classes/06class/parameterstore/README.md)
9+
1110
## [Exercises](/classes/06class/exercises/README.md)
1211

1312
## What's serverless?

‎classes/06class/exercises/c06-serverless01/ANSWER.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -13,4 +13,4 @@
1313
<!-- Don't change anything below this point-->
1414
<!-- Before commiting, remove both commented lines-->
1515
***
16-
Answer for exercise [c06-serverless01](https://github.com/devopsacademyau/academy/blob/cdbf8008dd42eacd5aa6f43f66b152e8f91d0e78/classes/06class/exercises/c06-serverless01/README.md)
16+
Answer for exercise [c06-serverless01](https://github.com/devopsacademyau/academy/blob/22bc97543c8193afc157cb3bc12e876d642c75ac/classes/06class/exercises/c06-serverless01/README.md)

‎classes/06class/exercises/c06-serverless01/README.md

+5-5
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
# Creating a POST API using API Gateway, Lambda and DynamoDB
22

3-
For this exercise you will create a POST API for creating a record in the database using APIs. This is a common scenario that we see in several software projects.
3+
For this exercise you will create a POST API for creating a record in DynamoDB using API Gateway and Lambda. This is a common use case that we've seen in several software projects.
44

55
For that you will use:
66
- AWS API Gateway, Lambda, DynamoDB, KMS, SSM
77
- Terraform for IaC
88
- 3 Musketeers
99

10-
**IMPORTANT**: copy the files [src/](./src) folder to your exercise folder under your user name. You will use them as a base to develop the solution.
10+
**IMPORTANT**: copy the files [src/](./src) folder to your exercise folder under your user name in the exercises folder (example `exercises/c06-serverless01/<username>/src`). You will use them as a base to develop the solution.
1111

1212
Requirements:
1313

@@ -18,11 +18,11 @@ Requirements:
1818
1. You need to create a public REST API that will expose the following endpoints:
1919
1. `POST /customers`: For creating a new customer in the database.
2020
2. API Gateway: All resources will have Proxy integration with Lambda
21-
3. Authentication: Use API Key to protect your API. Push the value of the API key to SSM parameter store after creation.
22-
3. **Lambda**:
21+
3. Authentication: Use API Key to protect your API. Push the value of the `API key` to `SSM parameter store` after creation. Do not store secrets in your source code.
22+
3. **Lambda**:
2323
1. Source code is provided for you in Python (it is in the `src/lambda.zip`). Check it in [src/ folder](./src). You will point to this zip file in your terraform Lambda resource.
2424
2. This lambda will have an environment variable called `DB_NAME`. Make sure the value for this variable is fetched from SSM Parameter Store for deployment.
25-
3. *Tip: Lambda permissions: Lambda will need `dynamodb:PutItem` access*. Lambda will also need to grant invokeFunction permission to the API Gateway.
25+
3. *Tip: Lambda permissions: Lambda will need `dynamodb:PutItem` access*. Lambda will also need to grant `invokeFunction` permission to the API Gateway.
2626
4. **SSM Parameter store**: Use parameter store to store and retrieve the value of `DB_NAME` in your 3 Musketeers scripts.
2727
5. **KMS Key**: Create a new key used for encrypting the DynamoDB table.
2828
1. *Tip: lambda will require access to Encrypt the data using this key. This needs to be described in the KMS Key policy.*

‎classes/07class/README.md

+6-2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,9 @@
22

33
## Agenda
44

5-
- [KMS](kms/README.md)
6-
- [SSM](ssm/README.md)
5+
- [Serverless Integration patterns](serverless_integration/README.md)
6+
- [AWS DynamoDB Streams](serverless_integration/README.md#dynamodb-streams)
7+
- [AWS SNS - Notification service](serverless_integration/README.md#simple-notification-service-sns)
8+
- [AWS SQS - Queue service](serverless_integration/README.md#simple-queue-service-sqs)
9+
- [AWS S3 event notifications](serverless_integration/README.md#flat-files-integration)
10+
- [AWS Cloudwatch Events (EventBridge)](serverless_integration/README.md#event-triggers)

‎classes/07class/exercises/README.md

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Labs and Exercises
2+
3+
For instructions on how to submit your work, [please check the main repository README here](/README.md#exercises)
4+
5+
Exercises:
6+
- [Uploading profile images and notifying](c07-serverless01/README.md)
7+
- [Compliance - Shutdown EC2 instances not tagged](c07-serverless02/README.md)

‎classes/07class/exercises/c07-serverless01/ANSWER.md

+23
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# c07-serverless02
2+
3+
## Uploading profile images and notifying
4+
5+
- [terraform/_provider.tf](terraform/_provider.tf)
6+
- [`terraform/main.yaml`](terraform/main.yaml): terraform code for all resources
7+
- [`src/photo_handler.py`](src/): Lambda for handling S3 event (the lambda DOES NOT need to be Python, you can use other languages)
8+
- [`src/report_customers_count.py`](src/): Lambda for consuming from DDB stream and calling SNS (the lambda DOES NOT need to be Python, you can use other languages)
9+
- [`docker-compose.yaml`](docker-compose.yaml): part of 3M
10+
- [`Makefile`](Makefile): 3 Musketeers implementation
11+
- [docker-compose.yaml](docker-compose.yaml): part of 3M implementation
12+
13+
> Solution might contain other files than the ones listed above.
14+
15+
## Comments and Challenges faced
16+
17+
> List here any challenges faced and how you solved them
18+
19+
20+
<!-- Don't change anything below this point-->
21+
<!-- Before commiting, remove both commented lines-->
22+
***
23+
Answer for exercise [c07-serverless02](https://github.com/devopsacademyau/academy/blob/b06b4cc323b9349d904562e45551c22974928952/classes/07class/exercises/c07-serverless01/README.md)

‎classes/07class/exercises/c07-serverless01/README.md

+31
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
## Uploading profile images and notifying
2+
3+
> Use as base for this exercise your solution for the exercise from the last class (API gateway -> Lambda -> DynamoDB).
4+
5+
**Requirements**:
6+
- the name of the image is the email address of the user (to keep codification simpler)
7+
- Using a new Lambda (you need to create it) triggered by the S3 object creation, update the DynamoDB Table you created in the previous exercise to store the photo location path (S3 prefix) new attribute `photo_location` for that user by email address.
8+
- Example:
9+
- If I upload the file PNG `[email protected]` to the S3 bucket `s3://mycustomersphotos`, then the Item in DDB for the email address `[email protected]` needs to update the attribute `photo_location` to `[email protected]`
10+
- Also, in the previous lambda, if the update was successful, send an email (SES) to the user saying "Thanks for adding your awesome photo :)"
11+
12+
- As soon as a new customer is created in the DynamoDB Table, stream it (DDB Streams) and send a report message by email (SNS) to the admin email (your email here) saying "Number of records: X", where X is the number of customers in the table
13+
14+
15+
At this stage you probably already have this:
16+
17+
![](assets/current_state.png)
18+
19+
Target state:
20+
21+
![](assets/target_state.png)
22+
23+
## Submit a PR with the following files:
24+
25+
- README.md based on the [ANSWER.md file](ANSWER.md) with a link to the following files from your answer (the actual solution may have more files):
26+
- `terraform/main.yaml`: terraform code for all resources
27+
- `src/photo_handler.py`: Lambda for handling S3 event (the lambda DOES NOT need to be Python, you can use other languages)
28+
- `src/report_customers_count.py`: Lambda for consuming from DDB stream and calling SNS (the lambda DOES NOT need to be Python, you can use other languages)
29+
- `docker-compose.yaml`: part of 3M
30+
- `Makefile`: 3 Musketeers implementation
31+
- In the `README.md` file include details on the execution

‎classes/07class/exercises/c07-serverless01/assets/current_state.png

17 KB
Loading

‎classes/07class/exercises/c07-serverless01/assets/target_state.png

55.7 KB
Loading

‎classes/07class/exercises/c07-serverless02/ANSWER.md

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# c07-serverless02
2+
3+
## Compliance - Shutdown EC2 instances not tagged
4+
5+
- [_provider.tf](_provider.tf)
6+
- [main.tf](main.tf)
7+
- [docker-compose.yaml](docker-compose.yaml)
8+
- [Makefile](Makefile)
9+
10+
> Solution might contain other files than the ones listed above.
11+
12+
## Command Execution Output
13+
14+
<!-- Don't change anything below this point-->
15+
<!-- Before commiting, remove both commented lines-->
16+
***
17+
Answer for exercise [c07-serverless02](https://github.com/devopsacademyau/academy/blob/b06b4cc323b9349d904562e45551c22974928952/classes/07class/exercises/c07-serverless02/README.md)

‎classes/07class/exercises/c07-serverless02/README.md

+33
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
## Compliance - Shutdown EC2 instances not tagged
2+
3+
Requirement:
4+
- If an EC2 instance is running and is not tagged with `CostCentre` (any value), terminate it and notify subscribed admins via a SNS topic that has only your email subscribed.
5+
6+
Tips:
7+
- Solution will include a Cloudwatch event rule (see below) and a Lambda ([check the ./src folder](../c07-serverless02/src/))
8+
9+
![](assets/ex2_diagram.png)
10+
11+
Sample CloudWatch Event rule:
12+
```json
13+
{
14+
"source": [
15+
"aws.ec2"
16+
],
17+
"detail-type": [
18+
"EC2 Instance State-change Notification"
19+
],
20+
"detail": {
21+
"state": [
22+
"running"
23+
]
24+
}
25+
}
26+
```
27+
28+
## Submit a PR with the following files:
29+
- README.md based on the [ANSWER.md file](ANSWER.md) with a link to the following files from your answer (the actual solution may have more files):
30+
- `main.tf`: terraform code for all resources
31+
- `docker-compose.yaml`: part of 3M
32+
- `Makefile`: 3 Musketeers implementation
33+
- In the `README.md` file include details on the execution of the test below:

‎classes/07class/exercises/c07-serverless02/assets/ex2_diagram.png

22.6 KB
Loading

‎classes/07class/exercises/c07-serverless02/src/remove_untagged_instances.py

+52
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
import json
2+
import os
3+
import boto3
4+
import jmespath
5+
6+
def lambda_handler(event, context):
7+
print(event)
8+
9+
tag_name=os.getenv('TAG_NAME', 'CostCentre')
10+
11+
ec2 = boto3.client('ec2')
12+
13+
# list all running instances
14+
response = ec2.describe_instances(
15+
Filters=[
16+
{
17+
'Name': 'instance-state-name',
18+
'Values': [
19+
'running',
20+
]
21+
}
22+
]
23+
)
24+
print(response)
25+
26+
# filter instances missing CostCentre tag
27+
result = jmespath.search(f'Reservations[].Instances[?!not_null(Tags[?Key == `{tag_name}`].Value)] | [].InstanceId', response)
28+
print('Filtered results: ' + json.dumps(result))
29+
30+
if result:
31+
# terminate instances
32+
ec2.terminate_instances(InstanceIds=result)
33+
ec2.create_tags(
34+
Resources=result,
35+
Tags=[
36+
{
37+
'Key': 'Name',
38+
'Value': 'INVALID_TAGS'
39+
}
40+
]
41+
)
42+
43+
if not os.getenv('SNS_TOPIC', None):
44+
raise ValueError("Could not find the environment variable SNS_TOPIC")
45+
46+
sns = boto3.client('sns')
47+
sns.publish(
48+
TopicArn=os.getenv('SNS_TOPIC', None),
49+
Message=f'Instances terminated: {json.dumps(result)}'
50+
)
51+
52+
print('Instances terminated: ' + json.dumps(result))

0 commit comments

Comments
 (0)