Repository for the course project for CSCI-B 649 Applied Distributed Systems in Spring 2022. This project is based on a multi-user system with a distributed architecture for consuming and processing weather data in various forms, logging its usage and presenting the weather information to the user in a web-based interface.
The project is in its initial stages of development and the README will be updated as new features are added.
The microservices from the architecture above is built with the following tech stack
- Java 17
- SpringBoot
- Maven
- Lombok
- JPA
- Python 3
- FastAPI
- Express
- NodeJS
- MongoDB
- PostgresSQL
- Docker
- Docker
The application is containerized and is hosted in docker registry. To setup the application run the docker-compose file.
- Minikube cluster is used to deploy scale each of the microservices. The application was tested for various amounts of load with Apache Jmeter.
- Minikube cluster with kubectl installed and configured to use your cluster
- Docker cli installed, you must be signed into your Docker Hub account
-
Navigate to this folder and run the following command deploy the application in the Kubernetes cluster
kubectl apply -f kubernetes/. --recursive -
To manually scale the application to 3 replicas run the following kube-scale.sh script file with -r as 3
bash kube-scale.sh -r 3
Jmeter Test details and Report
- The REST based communication between the services from project 2 is replaced with Kafka based message streaming.
- Github Actions were used for Continuous integration and deployment.
- On pushing the code to Github the Github workflow is triggered to build and push the image to registry and simultaneously a rolling update is made on the Kubernetes cluster in the remote server.
- Each service has a feature and a development branch.
- The initial development for the service is done in the feature branch and on completion it is pushed to the develop branch
- A push to the develop branch triggers a workflow where the Github actions are implemented to build, create image and push the image to docker registry.
- The individual service development branches are merged to the develop branch and from there on push to the main branch an ssh connection is made to the remote server where the Kubernetes cluster is hosted and a rolling update is made.
- All the credentials and sensitive data are stored in the Github secrets.
- Using terraform and OpenStack CLI we will create the instances. (master and worker nodes).
- Using Ansible playbooks we will connect to the above instances and run the playbooks to setup kubernetes.
- Finally connect to the master node and run Kubernetes yaml files for starting deployments and services.
We need an instance to set up Ansible first. This can be a local or a remote machine. We chose a remote machine with Ubuntu image. Login to the machine and open the terminal
- Create SSH-key.
sudo su
ssh-keygen
This will generate public and private keys. we will use this key later to SSH connect to the instances we create.
- Copy the openrc.sh file to this instance. You can follow the steps in this Link to download the openrc file. And then Run the following command,
source <openrc.sh>
This will just save the environment variables to our localhost which we need to connect to the OpenStack python client.
- Make sure you have a public IP available. This can be created in the exosphere environment or running the following command. Ignore this step if you have an public IP address available.
pip3 install python-openstackclient
openstack floating ip create public
- Paste the IP address you created in the first line below and run the following code. We need to modify certain attributes from the cluster.tfvars file
- master node IP address (k8s_master_fips)
- availability zones(az_list)
- external network(external_net)
- number of nodes (number_of_k8s_nodes)
Note: The shell script below only modifies the IP address and the rest of the above attributes are already set based on the cloud environment provided to us.
export IP=149.165.152.125
bash instance-creation.sh
What does the instance-creation.sh shell script do?
- It will install terraform, ansible.
- Initiate the terraform scripts for creation of the VM instances.
- Setup SSH as an agent to connect to the remote machines
- Run ansible playbooks to install kubernetes.
- Kubernetes is installed successfully in our cluster( at the end of the above script you will be logged into the master node) and all the necessary deployment files are copied to the master node.
cd deploy
bash deploy.sh
- To test the setup, you can ssh to the master machine and check the nodes connected. (Optional)
ssh ubuntu@$IP
sudu su
kubectl get nodes
- The ansible implementation to setup jetstream instance and Kubernetes Cluster is adapted from Team-terra 's implementation