4A: An Animation-based Augmentation Approach for Action Recognition from Discontinuous Video

4A is a innovative dataset generation framework leveraging game engine technology to facilitate advancements in action recognition. 4A excels in automatically creating large-scale, well-annotated datasets with extensive action classes and superior video quality.

Project Setup

FiveM Server

To setup 4A, you need a Rockstar account and setup a FiveM server. For tutorial please refer to FiveM offical website

3ds MAX

To view your animations, you will need a 3D modeling tool. We recommend 3ds Max. You can setup by the offical website.

AnimKit

To make a dictionary for your costomized animation, please download Animkit from their website.

CodeWalker

To edit your map for presenting your animation, please download CodeWalker through their website.

OpenIV

To mangage your animation library, please download OpenIV from this website.

3D Pose Estimation

First, you need to obatin the human skeleton keypoint coordinates by 3D pose estimation. To adress this, we recommend HRNet for 2D wholebody pose extimatiom, and JointFormer for 2D-to-3D pose lifting. Also, we provide the inferencers for these two models, which can be found in /inferencer. Additionally, we recommend to use "COCO-Wholebody" for the 2D pose extimation training, and "H3WB" for the 3D lifting. 4A supports both COCO-Wholebldy and NTU-RGB+D layouts.

Action Animation

For Action Animation, please refer to "animation_generation" section.

Auto-collection

For Auto-collection, please refer to "dataset_generation" section.

Evaluation

For evaluation, please refer to "ecaluation_code" section.