|
1 |
| -# 3d-object-detection.pytorch |
| 1 | +# 3d-object-detection.pytorch |
| 2 | + |
| 3 | +This project provides code to train a two stage 3d object detection models on the [Objectron](https://github.com/google-research-datasets/Objectron) dataset. |
| 4 | + |
| 5 | +Training includes the following stages: |
| 6 | +- Converting the original Objectron annotation to COCO-like format |
| 7 | +- Training a 2d bounding box detection model |
| 8 | +- Training a 3d bounding box regression model |
| 9 | + |
| 10 | +Trained models can be deployed on CPU using [OpenVINO](https://docs.openvinotoolkit.org) framework and then run in [live demo](). |
| 11 | + |
| 12 | +## Installation guide |
| 13 | +```bash |
| 14 | +git clone https://github.com/sovrasov/3d-object-detection.pytorch.git --recursive |
| 15 | +python setup.py develop |
| 16 | +``` |
| 17 | +All the mentioned below scripts should be launched from the repo root folder because they depend on the Objectron python package which is distributed in source codes only. This issue is planned to be addressed. |
| 18 | + |
| 19 | +## Converting data |
| 20 | + |
| 21 | +Download raw Objectron data and extract it preserving the following layout: |
| 22 | +``` |
| 23 | +Objectron_root |
| 24 | +├── annotation |
| 25 | +│ ├── bike |
| 26 | +| | └── <batch-* folders with .pbdata files> |
| 27 | +│ ├── book |
| 28 | +| | └── <batch-* folders with .pbdata files> |
| 29 | +│ |── .... |
| 30 | +│ └── shoe |
| 31 | +| └── <batch-* folders with .pbdata files> |
| 32 | +│ |
| 33 | +└── videos |
| 34 | + ├── bike |
| 35 | + | └── <batch-* folders with videos> |
| 36 | + ├── book |
| 37 | + | └── <batch-* folders with videos> |
| 38 | + └── .... |
| 39 | + └── shoe |
| 40 | + └── <batch-* folders with videos> |
| 41 | +``` |
| 42 | + |
| 43 | +Then run the converter: |
| 44 | + |
| 45 | +```bash |
| 46 | +python annotation_converters/objectron_2_coco.py --data_root <Objectron_root> --output_folder <output_dir> --fps_divisor 5 --res_divisor 2 --obj_classes all |
| 47 | +``` |
| 48 | + |
| 49 | +Adjacent frames on the 30 FPS videos are close to each other, so we can take only each `fps_divisor` frame and downscale them from the FullHD resolution by a factor `res_divisor` without significant loss of information. |
| 50 | + |
| 51 | +## Train 2d detector |
| 52 | + |
| 53 | +Detector trained on this step is supposed to retrieve 2d bounding boxes that enclose 3d boxes from the original Objectron annotation. |
| 54 | +At the next stage a multi-class regression model is launched on detected regions to finally obtain 2d coordinates of projected 3d bounding box vertexes. |
| 55 | + |
| 56 | +To launch training refer to the instructions from the modified [mmdetection](https://github.com/openvinotoolkit/mmdetection/) repo. |
| 57 | +Config for detector is stored in `configs/detection/mnv2_ssd_300_2_heads.py`, COCO-formatted data required for training is obtained at the previous step. |
0 commit comments