Skip to content

Commit

Permalink
Add dataset description
Browse files Browse the repository at this point in the history
  • Loading branch information
climbingdaily committed Mar 26, 2023
1 parent 3b1079d commit 8001172
Show file tree
Hide file tree
Showing 14 changed files with 104 additions and 6 deletions.
110 changes: 104 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@
·
<a href="http://www.cwang93.net/index_en.htm"><strong>Cheng Wang</strong></a>
</p>
<h2 align="center">CVPR 2023</h2>

<p align="center">
<a href="https://arxiv.org/pdf/2303.09095.pdf">
Expand All @@ -33,19 +32,118 @@
<img src='https://img.shields.io/badge/Homepage-orange?style=for-the-badge&logo=Google%20chrome&logoColor=white&labelColor=D35400' alt='Project Page'></a>
</p>

<div align="center">
<!-- <div align="center">
<video width="90%" autoplay loop muted>
<source src="./assets/teaser.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
</div>
</div> -->

<div align="center">
<img src="./assets/teaser.gif" alt="Logo" width="100%">
</div>



<!-- <div align="center">
<img src="./assets/teaser.gif" alt="Logo" width="90%">
</div> -->

## News and Updates
- *More info is coming soon…*
- **03/2023**: Initial release of the visualization Tool ([SMPL-Scene Viewer](https://github.com/climbingdaily/SMPL-Scene-Viewer)) (v1.0)

<br>
<br>
<br>

## Dataset
- 15 sequences of 12 human subjects in
- 10 scenes in urban environments (1k – 30k 𝑚^2)
- 100k+ frames multi-source data (20 Hz)
- including 2D / 3D annotations and 3D scenes;
7 km+ human motions.

*Every human subject signed permission to release their motion data for research purposes.*

<img src="assets/1_data_intro_202332215833.gif" width="50%"><img src="assets/1_data_intro_2_202332215837.gif" width="50%">

### Dataset breakdown
| Num | Sequence | Traj. length ($m$) | Area size ($m^2$) | Frames | Motions |
| --- | ------------ | ---------------- | ---------------- | ------ | -------------------------------------------- |
| 001 | campus_001 | 908 | 13,400 | 16,202 | Jogging downhill, tying shoelaces, jumping |
| 002 | football_002 | 221 | 200 | 4,665 | Juggling, passing, and shooting a football |
| 003 | street_002 | 291 | 1,600 | 6,496 | Taking photos, putting on/taking off a bag |
| 004 | library_001 | 440 | 2,300 | 9,949 | Borrowing books, reading, descending stairs |
| 005 | library_002 | 474 | 2,300 | 8,901 | Looking up, reading, returning a book |
| 006 | library_003 | 477 | 2,300 | 8,386 | Admiring paintings, throwing rubbish, greeting |
| 007 | garden_001 | 217 | 3,000 | 5,994 | Raising hand, sitting on bench, going upstairs |
| 008 | running_001 | 392 | 8,500 | 2,000 | Running |
| 009 | running_002 | 985 | 30,000 | 8,113 | Running |
| 010 | park_001 | 642 | 9,300 | 12,445 | Visiting a park, walking up a small hill |
| 011 | park_002 | 1,025 | 11,000 | 1,000 | Buying drinks, trotting, drinking |
| 012 | square_001 | 264 | 3,200 | 6,792 | Making phone calls, waving, drinking |
| 013 | sunlightRock001 | 386 | 1,900 | 10,116 | Climbing stairs, taking photos, walking |
| 014 | garden_002 | 209 | 4,200 | 5,604 | Stooping, crossing a bridge, sitting cross-legged |
| 015 | plaza_001 | 365 | 2,700 | 7,989 | Admiring sculptures, eating

### **File structure**
``` bash
├── seq{Num}_{Sequence}/
| ├── lidar_data/
| | ├── lidar_frames_rot/
| | | └── '*.pcd' # n frames pcd
| | ├── 'lidar_trajectory.txt'
| | └── 'tracking_traj.txt'
| ├── mocap_data/
| | └── '*.bvh'
| ├── rgb_data/
| | ├── 'labels.pkl'
| | └── '*.MP4'
| └── 'dataset_params.json'
├── ...
└── ...
```
For more information on data processing, please see [data-processing/README.md](./data_processing/README.md).

## More qualitative results
- Comparison between **IMU + ICP** and **our optimization** results.

<div align="center">
<img src="./assets/2_compare_202332215842.gif" alt="Logo" width="80%">
</div>
<div align="center">
Borrowing and reading a book on a sofa.
</div>

<div align="center">
<img src="./assets/2_compare_football_202332215855.gif" alt="Logo" width="80%">
</div>
<div align="center">
Playing football.
</div>

- Comparison between *original extrinsic parameters* and *our optimization results*.
<div align="center">
<img src="./assets/2_compare_extrinsic_202332215849.gif" alt="Logo" width="80%">
</div>


### Cross-Dataset Evaluation
- LiDAR-based human pose estimation (HPE)

<div align="center">
<img src="./assets/3_compare_lidarcap_20233221594.gif" alt="Logo" width="80%">
</div>

- Camera-based HPE
<div align="center">
<img src="./assets/3_compare_vibe_20233221598.gif" alt="Logo" width="80%">
</div>


- Global Human Pose Estimation Comparison
<div align="center">
<img src="./assets/4_compare_ghpe_glamr_202332215913.gif" alt="Logo" width="80%">
</div>

## Citation
```
@inproceedings{dai2023sloper4d,
Expand Down
Binary file added assets/0_pipeline_202332215830.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/1_data_intro_202332215833.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/1_data_intro_2_202332215837.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/1_hardware.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/2_compare_202332215842.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/2_compare_extrinsic_202332215849.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/2_compare_football_202332215855.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/3_compare_lidarcap_20233221594.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/3_compare_vibe_20233221598.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/4_compare_ghpe_glamr_202332215913.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/teaser.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed assets/teaser.mp4
Binary file not shown.
Empty file added data_processing/README.md
Empty file.

0 comments on commit 8001172

Please sign in to comment.