Cloud infrastructure setup

For all remote testing and logging, we use google cloud VM instances and associated tools. With terraform, most of the setup is automated, however, this tutorial provides a short overview of what needs to be done to apply the initial deployment.

Prerequisites

As mentioned in the main README, you need to have Terraform CLI tools installed. Google cloud CLI is also beneficial, but not necessary.

Shown by this tutorial, the first steps are as follows:

1. Create a new google cloud project. Make sure to change the project ID to yours in variables.tf (line 3).
2. Create a service account and a service account key. The details are outlined in the link above.
3. Add the following permissions in google cloud IAM to your service account:
   • "Resources Admin"
   • "Storage Admin"
   • "BigQuery Admin"
   • "Project IAM Admin"
   • "Service Usage Admin"
   • "Compute Network Admin"
   • "Editor"
   • "Logging Admin"
4. Copy or move the downloaded key to the infrastructure folder, and change the name to credentials.json

After this, you have the necessary steps completed to start using the actual setup script.

Using the setup script

As previously mentioned above, we use terraform and all related tools for deploying tests, instances, and other objects to the cloud.

To instantiate the terraform setup, run these commands in the given order:

• terraform init
• terraform plan --out tfplan
• terraform apply tfplan

this will deploy a set of instances and resources. After this, you can use the setup scripts for different scenarios, for example:

terraform apply -var-file ./experiment-3-baseline.tfvars

If you need inspiration or we might have forgotten something have a look at a working action

Teardown

To teardown all resources, use terraform destroy

Available scenarios and setup scripts

Use any of these files to perform a terraform apply and deploy the scenario:

• 3 syncmesh nodes - experiment-3-syncmesh.tfvars
• 3 baseline instances - experiment-3-baseline.tfvars
• 3 advanced MongoDB nodes - experiment-3-advanced-mongo.tfvars

Accessing the nodes/instances

There are multiple ways to access VM instances. The fastest and most convenient one is to ssh into it. If you have installed gcloud as recommended above, follow this tutorial to connect to a VM.

You can also use the cloud shell to ssh inside your browser, or do it manually by using ssh:

ssh -L 8080:ip:8080 username@ip

Port forwarding

With gcloud, you can use

gcloud compute ssh --project=project_name  INSTANCE_NAME -- -NL YOUR_LOCAL_PORT:localhost:8080

to forward the OpenFaaS UI and function namespace to your local machine. For regular ssh, see above.

To forward the connected MongoDB instance, use the port 27017 and the internal IP of the node instead of localhost:

gcloud compute ssh INSTANCE_NAME -- -NL 27017:internal_ip_here:27017