Add files via upload

README.md

@@ -0,0 +1,136 @@
+Point Cloud Classification and Segmentation
+========================
+**Name: Omkar Chittar**  
+**UID - 119193556**
+------------------------
+```
+PointCloud_Classification_and_Segmentation
++-checkpoints
++-data
++-logs
++-output
++-output_cls
++-output_cls_numpoints
++-output_cls_rotated
++-output_seg
++-output_seg_numpoints
++-output_seg_rotated
++-README.md
++-report
+data_loader.py
+eval_cls_numpoints.py
+eval_cls_rotated.py
+eval_cls.py
+eval_seg_numpoints.yml
+eval_seg_rotated.py
+eval_seg.py
+models.py
+train.py
+utils.py
+```
+
+# **Installation**
+
+- Download and extract the files.
+- Make sure you meet all the requirements given on: https://github.com/848f-3DVision/assignment2/tree/main
+- Or reinstall the necessary stuff using 'environment.yml':
+```bash
+conda env create -f environment.yml
+conda activate pytorch3d-env
+```
+## Data Preparation
+Download zip file (~2GB) from https://drive.google.com/file/d/1wXOgwM_rrEYJfelzuuCkRfMmR0J7vLq_/view?usp=sharing. Put the unzipped `data` folder under root directory. There are two folders (`cls` and `seg`) corresponding to two tasks, each of which contains `.npy` files for training and testing.
+- The **data** folder consists of all the data necessary for the code.
+- There are 6 output folders: 
+    1. **output_cls** folder has all the images/gifs generated after running ```eval_cls.py```.
+    2. **output_seg** folder has all the images/gifs generated after running ```eval_seg.py```.
+    3. **output_cls_numpoints** folder has all the images/gifs generated after running ```eval_cls_numpoints.py```.
+    4. **output_seg_numpoints** folder has all the images/gifs generated after running ```eval_seg_numpoints.py```.
+    5. **output_cls_rotated** folder has all the images/gifs generated after running ```eval_cls_rotated.py```.
+    6. **output_seg_rotated** folder has all the images/gifs generated after running ```eval_seg_rotated.py```.
+- All the necessary instructions for running the code are given in **README.md**.
+- The folder **report** has the html file that leads to the webpage.
+
+
+# **1. Classification Model**
+- After making changes to:
+    1. `models.py` 
+    2. `train.py`
+    3. `eval_cls.py`
+
+Run the code:  
+```bash
+python train.py --task cls
+```
+The code trains the model for the classification task. 
+
+Evaluate the trained model by running the code:
+```bash
+python eval_cls.py
+```
+Evaluates the model for the classification task by rendering point clouds named with their ground truth class and their respective predicted class. Displays the accuracy of the trained model in the terminal. The rendered point clouds are saved in the **output_cls** folder.
+
+
+# **2. Segmentation Model**
+- After making changes to:
+    1. `models.py` 
+    2. `train.py`
+    3. `eval_seg.py`
+
+Run the code:  
+```bash
+python train.py --task seg
+```
+The code trains the model for the Segmentation task. 
+
+Evaluate the trained model by running the code:
+```bash
+python eval_seg.py
+```
+Evaluates the model for the Segmentation task by rendering point clouds with segmented areas with different colors. The rendered point clouds are saved in the **output_seg** folder. Displays the accuracy of the trained model in the terminal.
+
+
+# **3. Robustness Analysis**
+## **3.1. Rotating the point clouds** 
+Here we try to evaluate the accuracy of the classification as well as the segmentation models by rotating the point clouds around any one axis (x/y/z) or their permutations.
+
+### 3.1.1. Classification
+Run the code:
+```bash
+python eval_cls_rotated.py
+```
+Evaluates the model with rotated inputs for the classification task by rendering point clouds named with their rotated angle, ground truth class and their respective predicted class. Displays the accuracy of the trained model in the terminal. The rendered point clouds are saved in the **output_cls_rotated** folder.
+
+### 3.1.2. Segmentation
+Run the code:
+```bash
+python eval_seg_rotated.py
+```
+Evaluates the model with rotated inputs for the segmentation task by rendering point clouds named with their rotated angle, and prediction accuracy. Displays the accuracy of the trained model in the terminal. The rendered point clouds are saved in the **output_seg_rotated** folder.
+
+
+## **3.2. Varying the sampled points in the point clouds** 
+Here we try to evaluate the accuracy of the classification as well as the segmentation models by varying the number of sampled points in the point clouds.
+
+### 3.2.1. Classification
+Run the code:
+```bash
+python eval_cls_numpoints.py
+```
+Evaluates the model with varying number of sampled points inputs for the classification task by rendering point clouds named with their index, number of points, ground truth class and their respective predicted class. Displays the accuracy of the trained model in the terminal. The rendered point clouds are saved in the **output_cls_numpoints** folder.
+
+### 3.2.2. Segmentation
+Run the code:
+```bash
+python eval_seg_numpoints.py
+```
+Evaluates the model with varying number of sampled points inputs for the segmentation task by rendering point clouds named with their index, number of points, and their respective predicted class accuracy. Displays the accuracy of the trained model in the terminal. The rendered point clouds are saved in the **output_seg_numpoints** folder.
+
+
+# **4. Webpage**
+The html code for the webpage is stored in the *report* folder along with the images/gifs.
+Clicking on the *webpage.md.html* file will take you directly to the webpage.
+
+
+
+

data_loader.py

@@ -0,0 +1,39 @@
+from torch.utils.data import DataLoader, Dataset
+import numpy as np
+import torch
+
+
+
+class CustomDataSet(Dataset):
+    """Load data under folders"""
+    def __init__(self, args, train=True):
+        self.main_dir = args.main_dir 
+        self.task = args.task 
+
+        if train:
+            data_path = self.main_dir + self.task + "/data_train.npy"
+            label_path = self.main_dir + self.task + "/label_train.npy"
+        else:
+            data_path = self.main_dir + self.task + "/data_test.npy"
+            label_path = self.main_dir + self.task + "/label_test.npy"
+
+        self.data = torch.from_numpy(np.load(data_path))
+        self.label = torch.from_numpy(np.load(label_path)).to(torch.long) # in cls task, (N,), in seg task, (N, 10000), N is the number of objects
+
+
+    def __len__(self):
+        return self.data.size()[0]
+
+    def __getitem__(self, idx):
+        return self.data[idx], self.label[idx]
+
+
+def get_data_loader(args, train=True):
+    """
+    Creates training and test data loaders
+    """
+    dataset = CustomDataSet(args=args, train=train)
+    dloader = DataLoader(dataset=dataset, batch_size=args.batch_size, shuffle=train, num_workers=args.num_workers)
+
+
+    return dloader

eval_cls.py

@@ -0,0 +1,103 @@
+import numpy as np
+import argparse
+
+import torch
+from models import cls_model
+from utils import create_dir, viz_cls
+from data_loader import get_data_loader
+
+import random
+import pytorch3d
+
+def create_parser():
+    """Creates a parser for command-line arguments.
+    """
+    parser = argparse.ArgumentParser()
+
+    parser.add_argument('--num_cls_class', type=int, default=3, help='The number of classes')
+    parser.add_argument('--num_points', type=int, default=10000, help='The number of points per object to be included in the input data')
+
+    # Directories and checkpoint/sample iterations
+    parser.add_argument('--load_checkpoint', type=str, default='best_model') #model_epoch_0
+    parser.add_argument('--i', type=int, default=0, help="index of the object to visualize")
+
+    parser.add_argument('--test_data', type=str, default='./data/cls/data_test.npy')
+    parser.add_argument('--test_label', type=str, default='./data/cls/label_test.npy')
+    parser.add_argument('--output_dir', type=str, default='./output_cls')
+
+    parser.add_argument('--exp_name', type=str, default="exp", help='The name of the experiment')
+
+    parser.add_argument('--main_dir', type=str, default='./data/')
+    parser.add_argument('--task', type=str, default="cls", help='The task: cls or seg')
+    parser.add_argument('--batch_size', type=int, default=16, help='The number of images in a batch.')
+    parser.add_argument('--num_workers', type=int, default=0, help='The number of threads to use for the DataLoader.')
+
+    return parser
+
+
+if __name__ == '__main__':
+    parser = create_parser()
+    args = parser.parse_args()
+    args.device = torch.device("cuda" if torch.cuda.is_available() else 'cpu')
+
+    create_dir(args.output_dir)
+
+    # ------ TO DO: Initialize Model for Classification Task ------
+    model = cls_model().to(args.device)
+
+    # Load Model Checkpoint
+    model_path = './checkpoints/cls/{}.pt'.format(args.load_checkpoint)
+    with open(model_path, 'rb') as f:
+        state_dict = torch.load(f, map_location=args.device)
+        model.load_state_dict(state_dict)
+    model.eval()
+    print ("successfully loaded checkpoint from {}".format(model_path))
+
+    # Sample Points per Object
+    ind = np.random.choice(10000,args.num_points, replace=False)
+
+    # ------ TO DO: Make Prediction ------
+    test_dataloader = get_data_loader(args=args, train=False)
+
+    correct_obj = 0
+    num_obj = 0
+    predictions = []
+    for batch in test_dataloader:
+        point_clouds, labels = batch
+        point_clouds = point_clouds[:, ind].to(args.device)
+        labels = labels.to(args.device).to(torch.long)
+
+        with torch.no_grad():
+            pred_labels = torch.argmax(model(point_clouds), dim=-1, keepdim=False)
+        correct_obj += pred_labels.eq(labels.data).cpu().sum().item()
+        num_obj += labels.size()[0]
+
+        predictions.append(pred_labels)
+
+    accuracy = correct_obj / num_obj
+    print(f"test accuracy: {accuracy}")
+    predictions = torch.cat(predictions).detach().cpu()
+
+    # Visualize a few random test point clouds and failed test point clouds
+    fail_inds = torch.argwhere(predictions != test_dataloader.dataset.label)
+
+    for i in range(min(15, len(fail_inds))):
+        random_ind = random.randint(0, predictions.shape[0]-1)
+        while random_ind in fail_inds:
+            random_ind = random.randint(0, predictions.shape[0]-1)
+        verts = test_dataloader.dataset.data[random_ind, ind]
+        gt_cls = test_dataloader.dataset.label[random_ind].to(torch.long).detach().cpu().data
+        pred_cls = predictions[random_ind].detach().cpu().data
+
+        path = f"output_cls/1. random_vis_{random_ind}_with_gt_{gt_cls}_pred_{pred_cls}.gif"
+        viz_cls(verts, path, "cuda")
+
+    for i in range(len(fail_inds)):
+        fail_ind = fail_inds[i]
+        verts = test_dataloader.dataset.data[fail_ind, ind]
+        gt_cls = test_dataloader.dataset.label[fail_ind].detach().cpu().data
+        pred_cls = predictions[fail_ind].detach().cpu().data
+        path = f"output_cls/1.1 fail_vis_{fail_ind}_with_gt_{gt_cls}_pred_{pred_cls}.gif"
+        viz_cls(verts, path, "cuda")
+
+    print(f"test accuracy: {accuracy}")

eval_cls_numpoints.py

@@ -0,0 +1,92 @@
+import numpy as np
+import argparse
+
+import torch
+from models import cls_model
+from utils import create_dir, viz_cls
+from data_loader import get_data_loader
+
+import random
+import pytorch3d
+
+def create_parser():
+    """Creates a parser for command-line arguments.
+    """
+    parser = argparse.ArgumentParser()
+
+    parser.add_argument('--num_cls_class', type=int, default=3, help='The number of classes')
+    parser.add_argument('--num_points', type=int, default=10000, help='The number of points per object to be included in the input data')
+
+    # Directories and checkpoint/sample iterations
+    parser.add_argument('--load_checkpoint', type=str, default='best_model') #model_epoch_0
+    parser.add_argument('--i', type=int, default=0, help="index of the object to visualize")
+
+    parser.add_argument('--test_data', type=str, default='./data/cls/data_test.npy')
+    parser.add_argument('--test_label', type=str, default='./data/cls/label_test.npy')
+    parser.add_argument('--output_dir', type=str, default='./output_cls_numpoints')
+
+    parser.add_argument('--exp_name', type=str, default="exp", help='The name of the experiment')
+
+    parser.add_argument('--main_dir', type=str, default='./data/')
+    parser.add_argument('--task', type=str, default="cls", help='The task: cls or seg')
+    parser.add_argument('--batch_size', type=int, default=16, help='The number of images in a batch.')
+    parser.add_argument('--num_workers', type=int, default=0, help='The number of threads to use for the DataLoader.')
+
+    return parser
+
+
+if __name__ == '__main__':
+    parser = create_parser()
+    args = parser.parse_args()
+    args.device = torch.device("cuda" if torch.cuda.is_available() else 'cpu')
+
+    create_dir(args.output_dir)
+
+    # ------ TO DO: Initialize Model for Classification Task ------
+    model = cls_model().to(args.device)
+
+    # Load Model Checkpoint
+    model_path = './checkpoints/cls/{}.pt'.format(args.load_checkpoint)
+    with open(model_path, 'rb') as f:
+        state_dict = torch.load(f, map_location=args.device)
+        model.load_state_dict(state_dict)
+    model.eval()
+    print ("successfully loaded checkpoint from {}".format(model_path))
+
+    index = [94, 702, 870]
+
+    for j in index:
+        n_points = [10, 50, 100, 500, 1000, 10000]
+
+        for n in n_points:
+            # Sample Points per Object
+            ind = np.random.choice(10000, n, replace=False)
+
+            # ------ TO DO: Make Prediction ------
+            test_dataloader = get_data_loader(args=args, train=False)
+
+            correct_obj = 0
+            num_obj = 0
+            predictions = []
+            for batch in test_dataloader:
+                point_clouds, labels = batch
+                point_clouds = point_clouds[:, ind].to(args.device)
+                labels = labels.to(args.device).to(torch.long)
+
+                with torch.no_grad():
+                    pred_labels = torch.argmax(model(point_clouds), dim=-1, keepdim=False)
+                correct_obj += pred_labels.eq(labels.data).cpu().sum().item()
+                num_obj += labels.size()[0]
+
+                predictions.append(pred_labels)
+
+                accuracy = correct_obj / num_obj
+            print(f"test accuracy for num of points {n} : {accuracy}")
+            predictions = torch.cat(predictions).detach().cpu()
+
+            verts = test_dataloader.dataset.data[j, ind]       # change j to args.i for a particular index visualization
+            gt_cls = test_dataloader.dataset.label[j].to(torch.long).detach().cpu().data
+            pred_cls = predictions[j].detach().cpu().data
+
+            path = f"output_cls_numpoints/3.2. vis_{j}_numpoints_{n}_with_gt_{gt_cls}_pred_{pred_cls}_acc_{accuracy}.gif"
+            viz_cls(verts, path, "cuda")