Skip to content

Commit

Permalink
update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
anl13 committed Aug 30, 2022
1 parent f39811c commit 48815a3
Show file tree
Hide file tree
Showing 5 changed files with 44 additions and 59 deletions.
101 changes: 43 additions & 58 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,58 +1,43 @@
# MAMMAL core

This repository contains the core code of MAMMAL system. The system is written with C++17.

# Environment
The code has been tested on Windows 10 system with NVIDIA RTX 2080Ti GPU and CUDA 10.2.

After unzip this project, you need to download some necessary data from [Google Drive](https://drive.google.com/file/d/1m9hKCMhI_VJb1muM-sS_01TgYdDqaz3D/view?usp=sharing). The file you download is `data.zip`, just unzip it and put it under the MAMMAL_core main folder. Besides, download my pre-compiled third-party libraries from the link (https://drive.google.com/file/d/1Df-p3nQLE5lPo999eOvaKY59sMQurmW9/view?usp=sharing). Then unzip the third-party libraries and put it under the MAMMAL_core main folder. You will get a folder structure like
```
\-- MAMMAL_core\
|-- 3rdparty\
|-- data\
|-- annotator\
|-- articulation\
|-- configs\
|-- posesolver\
|-- props\
|-- render\
|-- utils\
|-- .gitignore
|-- README.md
|-- MAMMAL_core.sln
```
Then, download data files for render from [Google Drive](https://drive.google.com/file/d/1xZqepoemvG6aPBnMBn69cvaUKz5JMlZd/view?usp=sharing) and unzip it under `render/` folder like
```
\-- MAMMAL_core\
\-- 3rdparty\
|-- data\
|-- annotator\
|-- articulation\
|-- configs\
|-- posesolver\
|-- props\
|-- render\
\-- data\
|-- shader\
|-- utils\
|-- .gitignore
|-- README.md
|-- MAMMAL_core.sln
```
To open and run the project, we recommend to download and install Visual Studio 2017 Community from their official website (https://visualstudio.microsoft.com/). After installing Visual Studio 2017, you should install CUDA Runtime API 10.2 (or higher version. However, if you use higher version, you need to change some property configurations after opening the project. ). To install it, you should download from NVIDIA developer website (https://developer.nvidia.com/cuda-10.2-download-archive) and run the .exe file directly. Usually, both Visual Studio 2017 and CUDA are installed under C:/ drive by default. After installing above softwares, you can open the project now!

# Open the project
By simply clicking `MAMMAL_core.sln` file, the Visual Studio 2017 would open it.
Then, you will see five `projects` under the `MAMMAL_core` solution:
```
annotator
articulation
posesolver
render
utils
```
You can choose to set one of `annotator`, `articulation`, `posesolver` or `render` as the "start project". We recommend to try `render` first because its dependencies are the least. The, set the project compile properties as `Release` + `x64`. Finally, click the green triangle button to compile and run the project. If everything goes right, you will see a rendering of pig liveing environment model.


# Run demo on BamaPig3D dataset
The tutorial on how to run demo on BamaPig3D will be provided soon.
# MAMMAL
![](pics/teaser.png)
This repository contains the core code of **MAMMAL** system proposed in paper _Three-dimensional surface motion capture of multiple freely moving pigs using MAMMAL_.
This is the **first** system demonstrating pose and shape reconstruction of socially interacting animals. Moreover, it is the **first** system to analyse pig behavior in three-dimensional space.

## Installation
The code is written using C++ Std 17 on Visual Studio Community 2017. Please refer to `docs/INSTALL.md` for installation guide.

## Quick demo
Please refer to `docs/Demo.md`.

## The PIG model
To enable the MAMMAL system, we proposed a new pig articulation model named `PIG` model, as in the link https://github.com/anl13/PIG_model. This link contains necessary details and code for you to understand the model.
![](pics/model.jpg)

## BamaPig2D and BamaPig3D dataset
Check out link https://github.com/anl13/MAMMAL_datasets for descriptions of these datasets. Briefly, BamaPig2D dataset contains 3340 images with more than 10k pig instances fully labeled with bounding boxes, silhouettes and keypoints, and is formated in COCO style. We also provide a modified COCO annotation file suitable for training [SLEAP](https://sleap.ai). BamaPig3D dataset provides synchronized 70-second videos of 10 views (1750 frames, 25FPS). We manually labeled 2D keypoints, 2D silhouettes and 3D mesh/keypoints every 25 frames, resulting in 70 fully labeled frames. BamaPig3D dataset is used for evaluating the performance of MAMMAL system. However, it is also suitable for training 3D algorithms (e.g. [DANNCE](https://github.com/spoonsso/dannce) or [LiftPose3D](https://github.com/NeLy-EPFL/LiftPose3D)). The figure shows examples of BamaPig2D dataset.
![](pics/BamaPig2D.jpg)

## MAMMAL detection
MAMMAL detection contains two seperate parts: pig bounding box and silhouette detection modified from Detectron2, and pig pose detection modified from HRNet. Please refer to https://github.com/anl13/pig_silhouette_det and https://github.com/anl13/pig_pose_det for more details.

## Evaluation
Key evaluation code is released at https://github.com/anl13/MAMMAL_evaluation. It helps reproduce some figures of the paper. The results used for evaluation are also provided together with the code. Our provided results perfactly match the paper figures. However, if run the algorithm again, it may change a little (because we have updated the code recently), but not much.

## Pig behavior analysis
Please refer to code https://github.com/anl13/MAMMAL_behavior for behavior analysis performed in the paper.

## Rendering
To effectively render the model in python environment, we developed a `pig_renderer` as in https://github.com/anl13/pig_renderer.

## Citation
If you use these datasets in your research, please cite the paper

```BibTex
@article{MAMMAL,
author = {An, Liang and Ren, Jilong and Yu, Tao and Hai, Tang and Jia, Yichang and Liu, Yebin},
title = {Three-dimensional surface motion capture of multiple freely moving pigs using MAMMAL},
booktitle = {},
month = {July},
year = {2022}
}
```
2 changes: 1 addition & 1 deletion docs/INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ Note that, on the left of the green triangle buttom "Local Windows Debugger", ch
The whole MAMMAL\_core solution contains five projects named `annotator`, `articulation`, `posesolver`, `render`, `utils`. Their functions are
* annotator

It is a GUI (graphical user interface) for pig pose tunning. See `Annotator.md` for more details.
It is a GUI (graphical user interface) for pig pose tunning. See `Demo.md` for more details.
* articulation

It contain the core code for how to pose the PIG model and how to optimize the poses according to observations.
Expand Down
Binary file added pics/BamaPig2D.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added pics/model.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added pics/teaser.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 48815a3

Please sign in to comment.