add results & adjust docs+utils

Author: chr1sfr4

README.md

@@ -26,8 +26,6 @@ This Repository is structured as follows:
 
 ## Reproduction of Results
 
-We were not yet able to make all configurations in this repository independent of our AWS account and to upload our measurements as CSV files. We will improve this in the coming days.
-
 All plots and intermediate results can be reproduced with the code and README descriptions in the microbenchmarks directories mentioned above.
 Our results are part of this repo as well for exploration and replotting. Make sure to [prepare the according tools](#plotting).
 To rerun the benchmarks make sure to follow the [requirements instructions](#requirements) below.

aws/.gitignore

@@ -1 +1,2 @@
 venv
+awsenv

aws/README.md

@@ -1,5 +1,7 @@
 # EC2 Provisioning scripts
 
+This page contains a short guide to setup the AWS specific tools and configurations in order to create and manage EC2 Instances and access S3 resources on AWS. We used the AWS SSO feature for authentication, towards which this guide is taylord. If you want or need to use another authentication method, some steps and provided utilities might not be compatible without manual adjustments.
+
 ## Getting started
 
 1) Install aws cli2:
@@ -18,18 +20,24 @@
    sudo installer -pkg AWSCLIV2.pkg -target /
    ```
 
-2) Configure AWS cli: `aws configure sso`
+1) Create an `awsenv` file in this directory (`{projectRoot}/aws/awsenv`). It is excluded from git and stores all private AWS specific configuration used by the [`awsrc`](./awsrc) and some other places throughout this project. The variables should be insertet line by line in the format `KEY="value"` and will be exported as environment variables via `source awsrc`. You don't need to fill them out now as they will be explained in the following steps along their respective AWS configurations. However as an overview, they will include:
+   - your SSH key name (`AWS_SSH_KEY_NAME`)
+   - your AWS SSO profile (`AWS_PROFILE`, `S3_PROFILE`)
+   - the AWS subnet ID(s) of your private AWS cloud network(s) (`AWS_SUBNET_DEFAULT`, `AWS_SUBNET_ALL`)
+
+1) Configure AWS cli: `aws configure sso`
    - As SSO start URL use <https://d-[your-project-id].awsapps.com/start/#>
    - As SSO region use [your-sso-region]
    - Confirm in web browser
    - choose your CLI default client region where you want to deploy your EC2 instances and store your results
    - choose your CLI profile name, e.g. nitro (preferrably something short and easy to remember, since you will need this in several other places)
-   - set your `AWS_PROFILE` for use in the [`awsrc`](./awsrc)
-     ```bash
-     export AWS_PROFILE=[your-sso-profile]
+   - set your `AWS_PROFILE` for use in the [`awsrc`](./awsrc) and `S3_PROFILE` for downloading the results in the microbenchmarks in your [`awsenv`](./awsenv) file
+     ```shell
+     AWS_PROFILE=[your-sso-profile]
+     S3_PROFILE=$AWS_PROFILE
      ```
    - Other options can be set as you want.
-3) Setup python venv with dependencies:
+1) Setup python venv with dependencies:
    Run the `setup_venv.sh` script or use manual steps:
 
    ```bash
@@ -38,21 +46,25 @@
    pip3 install -r requirements.txt
    ```
 
-4) Setup VPC (a default VPC should be created automatically in each region)
-5) Set AWS SSH Key pair in the EC2 console under `Network & Security > Key Pairs`. Importing existing key pairs is possible under Actions. Save the key name in the environment variable `AWS_SSH_KEY_NAME` in your local shell. This is required for the shell functions and aliases in `awsrc`
-6) Allow SSH access from the internet to instances you create:
+1) Setup VPC (a default VPC should be created automatically in each region). The IDs should be stored to the [`awsenv`](./awsenv) file and can be looked up in the AWS console in the browser or via `ec2laz`. Those settings specify where instances, created via the shell functions from [`awsrc`](./awsrc) will be started.
+   ```
+   AWS_SUBNET_DEFAULT=[default-subnet]
+   AWS_SUBNET_ALL="subnet-id-1 subnet-id-2 ..."
+   ```
+1) Set AWS SSH Key pair in the EC2 console under `Network & Security > Key Pairs`. Importing existing key pairs is possible under Actions. Save the key name as `AWS_SSH_KEY_NAME` in your [`awsenv`](./awsenv) file.
+1) Allow SSH access from the internet to instances you create:
    1) Go to Security Groups in the EC2 console: <https://[your-client-region].console.aws.amazon.com/ec2/home?region=[your-client-region]#SecurityGroups:>
    2) Select the existing default security group
    3) Go to `Inbound Rules > Edit inbound rules`
    4) Create a rule that allows SSH access from your IP/from the Internet
    5) Save
-7) Create an S3 access role that can be attached to EC2 instances:
+1) Create an S3 access role that can be attached to EC2 instances:
    1) Go to IAM/Roles <https://[your-client-region].console.aws.amazon.com/iam/home#/roles>
    2) Create role > AWS Service > EC2 > Next > tick AmazonS3FullAccess > Next > name the role "EC2-S3-access-role" > Create Role
-8) Create an S3 Bucket for benchmarking
-9) For some useful settings, aliases, and functions. Load `awsrc` in your shell with `source awsrc`.
-10) Try if ec2 instance creation works with: `ec2c c6i.2xlarge`
-11) Set up SSH key forwarding to easily clone this git repository onto the created machines. For example, add the following to your `.ssh/config`:
+1) Create an S3 Bucket for benchmarking, e.g. `nitro-enclaves-result-bucket`
+1) For some useful settings, aliases, and functions (re)load `awsrc` in your shell with `source awsrc`.
+1) Try if ec2 instance creation works with: `ec2c c6i.2xlarge`
+1) Set up SSH key forwarding to easily clone this git repository onto the created machines. For example, add the following to your `.ssh/config`:
 
     ```bash
     Host *.compute.amazonaws.com
@@ -63,5 +75,5 @@
         ForwardAgent yes
     ```
 
-12) Get the instance public DNS name with `ec2li`.
-13) Clone repository and install requirements on the new instance with `ec2setup INSTANCE_DNS` substitute INSTANCE_DNS with the result of the previous step.
+1) Get the instance public DNS name with `ec2li`.
+1) Clone repository and install requirements on the new instance with `ec2setup INSTANCE_DNS`. substitute INSTANCE_DNS with the result of the previous step.

aws/awsrc

@@ -1,3 +1,11 @@
+
+# get the script directory
+AWSRC_DIR=$(readlink -f "$(dirname "$0")")
+export AWSRC_DIR="$AWSRC_DIR"
+set -a  # Automatically export variables
+source "$AWSRC_DIR"/awsenv
+set +a  # Disable automatic export
+
 # List instances
 alias ec2li="aws --profile $AWS_PROFILE ec2 describe-instances --output table --query \"Reservations[*].Instances[*].{Instance:InstanceId,State:State.Name,PrivateIp:PrivateIpAddress,PublicDnsName:PublicDnsName,Subnet:SubnetId,InstanceType:InstanceType,LaunchTime:LaunchTime}\""
 
@@ -13,11 +21,11 @@ export AWS_PAGER=""
 # Create instance
 # Parameter 1 (required): instance type
 # Parameter 2 (default 1): count
-# Parameter 3 (default subnet-08b661064a57e5dbd): subnet, determines availability zone (default 2a)
+# Parameter 3 (default $AWS_SUBNET_DEFAULT): subnet, determines availability zone (default 2a)
 # Use ec2laz to list availability zones and connected subnets incase you need to start the VM in a different AZ than the default 2a.
 ec2c() {
     count=${2:-1}
-    subnet=${3:-"subnet-08b661064a57e5dbd"}
+    subnet=${3:-"$AWS_SUBNET_DEFAULT"}
     network_config="{\"SubnetId\":\"$subnet\",\"AssociatePublicIpAddress\":true,\"DeviceIndex\":0,\"Groups\":[\"sg-0772ae4b8cfc0e188\"]}"
     echo "$network_config"
     aws --profile $AWS_PROFILE ec2 run-instances \
@@ -27,15 +35,15 @@ ec2c() {
     --key-name "$AWS_SSH_KEY_NAME" \
     --enclave-options 'Enabled=true' \
     --network-interfaces=$network_config \
-    --instance-market-options file://spot-options.json \
+    --instance-market-options file://"$AWSRC_DIR"/spot-options.json \
     --block-device-mappings '[{"DeviceName":"/dev/xvda","Ebs":{"VolumeSize":16}}]' \
     --iam-instance-profile 'Name="EC2-S3-access-role"'
 }
 
 # Create an instance with spot pricing and Graviton CPU
 ec2cg() {
     count=${2:-1}
-    subnet=${3:-"subnet-08b661064a57e5dbd"}
+    subnet=${3:-"$AWS_SUBNET_DEFAULT"}
     network_config="{\"SubnetId\":\"$subnet\",\"AssociatePublicIpAddress\":true,\"DeviceIndex\":0,\"Groups\":[\"sg-0772ae4b8cfc0e188\"]}"
     echo "$network_config"
     aws --profile $AWS_PROFILE ec2 run-instances \
@@ -45,7 +53,7 @@ ec2cg() {
     --key-name "$AWS_SSH_KEY_NAME" \
     --enclave-options 'Enabled=true' \
     --network-interfaces=$network_config \
-    --instance-market-options file://spot-options.json \
+    --instance-market-options file://"$AWSRC_DIR"/spot-options.json \
     --block-device-mappings '[{"DeviceName":"/dev/xvda","Ebs":{"VolumeSize":16}}]' \
     --iam-instance-profile 'Name="EC2-S3-access-role"'
 }
@@ -59,27 +67,37 @@ ec2ce() {
     --instance-type "$1" \
     --key-name "$AWS_SSH_KEY_NAME" \
     --enclave-options 'Enabled=true' \
-    --network-interfaces '{"SubnetId":"subnet-08b661064a57e5dbd","AssociatePublicIpAddress":true,"DeviceIndex":0,"Groups":["sg-0772ae4b8cfc0e188"]}' \
+    --network-interfaces '{"SubnetId":"$AWS_SUBNET_DEFAULT","AssociatePublicIpAddress":true,"DeviceIndex":0,"Groups":["sg-0772ae4b8cfc0e188"]}' \
     --block-device-mappings '[{"DeviceName":"/dev/xvda","Ebs":{"VolumeSize":16}}]' \
     --iam-instance-profile 'Name="EC2-S3-access-role"'
 }
 
-# Create one instance in each availability zone that we have a subnet for. Subnets are hard-coded.
+# Create Instance(s) in each availability zone in $AWS_SUBNET_ALL
+# Parameter 1 (required): instance type
+# Parameter 2 (default 1): count
+# Parameter 3 (default empty): g for Graviton CPU
 ec2caz() {
-    subnets=("subnet-08b661064a57e5dbd" "subnet-05994991e88c444ed" "subnet-0f46f92778eebc592")
-    for subnet in "${subnets[@]}";
+    count=${2:-1}
+    for subnet in $(echo "$AWS_SUBNET_ALL" | tr " " "\n");
     do
-        if [[ "$2" == "1" ]]; then
-            ec2cg "$1" 1 "$subnet"
+        echo "starting $count instances on subnet: $subnet"
+        if [[ "$3" == "g" ]]; then
+            ec2cg "$1" "$count" "$subnet"
         else
-            ec2c "$1" 1 "$subnet"
+            ec2c "$1" "$count" "$subnet"
         fi
     done
 }
 
 # Setup instance
 ec2setup() {
-    ssh -o "StrictHostKeyChecking no" "$1" 'sudo dnf install git -y && ssh -T -o "StrictHostKeyChecking accept-new" [email protected] ; mkdir -p ~/AWSNitroBenchmark && cd ~/AWSNitroBenchmark && git clone [email protected]:DataManagementLab/nitro-enclaves-benchmarks.git . && cd aws && chmod +x setup_ec2.sh && ./setup_ec2.sh'
+    ssh -o "StrictHostKeyChecking no" "$1" '
+        sudo dnf install git -y &&
+        ssh -T -o "StrictHostKeyChecking accept-new" [email protected] ;
+        mkdir -p ~/AWSNitroBenchmark && cd ~/AWSNitroBenchmark &&
+        git clone '"$(cd "$AWSRC_DIR" && git remote get-url origin)"' . &&
+        cd aws && chmod +x setup_ec2.sh && ./setup_ec2.sh
+    '
 }
 
 # Terminate instance
@@ -94,11 +112,11 @@ s3sr() {
 
 # Spot & Pricing Information
 # Parameter 1 (required): instance type(s)
-# Parameter 2 (default us-west-2): region
+# Parameter 2 (default derived from profile): region
 # Parameter 3 (only ec2sp, default 0): lookback hours
 
 ec2sp() {
-    region=${2:-"us-west-2"}
+    region=${2:-"$(aws configure get region --profile $AWS_PROFILE)"}
     lookback_hours=${3:-0}
     start_time=$(date -u -d "-${lookback_hours} hours" +%FT%TZ 2>/dev/null || date -u -v-"${lookback_hours}"H +%FT%TZ)
     aws --profile $AWS_PROFILE ec2 describe-spot-price-history \
@@ -110,7 +128,7 @@ ec2sp() {
 }
 
 ec2ssc() {
-    region=${2:-"us-west-2"}
+    region=${2:-"$(aws configure get region --profile $AWS_PROFILE)"}
     aws --profile $AWS_PROFILE ec2 get-spot-placement-scores \
     --instance-types "$1" \
     --target-capacity 2 \

microbenchmarks/SockLatency/plot/plot.R

@@ -44,24 +44,32 @@ df <- lapply(csv_files, function(file) {
     return(NULL)  # Skip invalid files
   }
 }) %>%
-  bind_rows() %>%
-  mutate_if(is.character, trimws) %>%
-  mutate(
-    architecture_id = case_when(
-      scenario == "single_instance" & protocol == "inet"  ~ 1,
-      scenario == "single_instance" & protocol == "vsock"  ~ 2,
-      scenario == "single_instance_proxy" ~ 3,
-      scenario == "cross_instance_host2host" ~ 4,
-      scenario == "cross_instance_host2enclave" ~ 5,
-      scenario == "cross_instance_proxy" ~ 6,
-      TRUE ~ NA
-    )
-  )
+  bind_rows()
 
-# store the data
 agg_store_path <- normalizePath("../results/data/results.csv", mustWork = FALSE)
-write.csv(df, file = agg_store_path, row.names = FALSE)
-print(paste("Combined data stored to:", agg_store_path))
+if (nrow(df)) {
+  # process combined data & cleanup
+  df <- df %>%
+    mutate_if(is.character, trimws) %>%
+    mutate(
+      architecture_id = case_when(
+        scenario == "single_instance" & protocol == "inet"  ~ 1,
+        scenario == "single_instance" & protocol == "vsock"  ~ 2,
+        scenario == "single_instance_proxy" ~ 3,
+        scenario == "cross_instance_host2host" ~ 4,
+        scenario == "cross_instance_host2enclave" ~ 5,
+        scenario == "cross_instance_proxy" ~ 6,
+        TRUE ~ NA
+      )
+    )
+
+  # store the data
+  write.csv(df, file = agg_store_path, row.names = FALSE)
+  print(paste("Combined data stored to:", agg_store_path))
+} else {
+  df <- read.csv(agg_store_path, stringsAsFactors = TRUE)
+  print(paste("Results read from", agg_store_path))
+}
 
 # all plots from the paper were generated via plot.py
 # for your own exploration you can use the following code

microbenchmarks/SockLatency/plot/plot.py

@@ -26,7 +26,7 @@ def plot_paper():
             & (df["architecture_id"] <= 5)
             ]
 
-    df.to_csv(f"{DATA_DIR}/filtered.csv")
+    df.to_csv(f"{DATA_DIR}/filtered.csv", index=False)
 
     # Project to required columns
     x_axis = "Message Size [Byte]"