EFRS-1254 - Add documentation of Pose plugin

gusdoe · Sep 29, 2022 · 7747415 · 7747415
1 parent 1ffcd6a
commit 7747415
Show file tree

Hide file tree

Showing 6 changed files with 24 additions and 14 deletions.
diff --git a/README.md b/README.md
@@ -7,7 +7,7 @@
  </a>
   <br>
   <i>Exadel CompreFace is a free and open-source face recognition service that can be easily integrated into any system without prior machine learning skills. 
-     CompreFace provides REST API for face recognition, face verification, face detection, landmark detection, age, and gender recognition and is easily deployed with docker.
+     CompreFace provides REST API for face recognition, face verification, face detection, landmark detection, mask detection, head pose detection, age, and gender recognition and is easily deployed with docker.
      </i>
   <br>
 </p>
@@ -72,7 +72,7 @@ Exadel CompreFace is a free and open-source face recognition GitHub project.
 Essentially, it is a docker-based application that can be used as a standalone server or deployed in the cloud. 
 You don’t need prior machine learning skills to set up and use CompreFace.
 
-The system provides REST API for face recognition, face verification, face detection, landmark detection, age, and gender recognition. 
+The system provides REST API for face recognition, face verification, face detection, landmark detection, mask detection, head pose detection, age, and gender recognition. 
 The solution also features a role management system that allows you to easily control who has access to your Face Recognition Services.
 
 CompreFace is delivered as a docker-compose config and supports different models that work on CPU and GPU. 
@@ -116,6 +116,7 @@ The system can accurately identify people even when it has only “seen” their
   - [age recognition plugin](/docs/Face-services-and-plugins.md#face-plugins)
   - [gender recognition plugin](/docs/Face-services-and-plugins.md#face-plugins)
   - [face mask detection plugin](/docs/Face-services-and-plugins.md#face-plugins)
+  - [head pose plugin](/docs/Face-services-and-plugins.md#face-plugins)
 - Use the CompreFace UI panel for convenient user roles and access management
 
 # Getting Started with CompreFace

diff --git a/docs/Custom-builds.md b/docs/Custom-builds.md
@@ -86,7 +86,7 @@ compreface-core:
     args:
       - FACE_DETECTION_PLUGIN=insightface.FaceDetector@retinaface_r50_v1
       - CALCULATION_PLUGIN=insightface.Calculator@arcface_r100_v1
-      - EXTRA_PLUGINS=insightface.LandmarksDetector,insightface.GenderDetector,insightface.AgeDetector,insightface.facemask.MaskDetector
+      - EXTRA_PLUGINS=insightface.LandmarksDetector,insightface.GenderDetector,insightface.AgeDetector,insightface.facemask.MaskDetector,insightface.PoseEstimator
       - BASE_IMAGE=compreface-core-base:base-cuda100-py37
       - GPU_IDX=0
   environment:

diff --git a/docs/Face-services-and-plugins.md b/docs/Face-services-and-plugins.md
@@ -8,6 +8,8 @@ CompreFace supports these face services and plugins:
 * Gender detection plugin
 * Landmarks detection plugin
 * Calculator plugin
+* Face mask detection plugin
+* Head pose plugin
 
 # Services
 
@@ -119,19 +121,20 @@ comma-separated needed plugins in the query `face_plugins` parameter. This param
 Example:
 
 ```shell
-curl  -X POST "http://localhost:8000/api/v1/recognition/recognize?face_plugins=age,gender,landmarks,mask" \
+curl  -X POST "http://localhost:8000/api/v1/recognition/recognize?face_plugins=age,gender,landmarks,mask,pose" \
 -H "Content-Type: multipart/form-data" \
 -H "x-api-key: <faces_recognition_api_key>" \
 -F file=<local_file>
 ```
 
-This request will recognize faces on the image and return additional information about age, gender, face mask, and landmarks.
+This request will recognize faces on the image and return additional information about age, gender, head pose, face mask, and landmarks.
 
 The list of possible plugins:
 * age - returns the supposed range of a person’s age in format [min, max]
 * gender - returns the supposed person’s gender
 * landmarks - returns face landmarks. This plugin is supported by all configurations and returns 5 points of eyes, nose, and mouth
 * calculator - returns face embeddings.  
+* pose - returns head pose in format: `{"pitch": 0.0,"roll": 0.0,"yaw": 0.0}`
 * mask - returns if the person wears a mask. Possible results: `without_mask`, `mask_worn_incorrectly`, `mask_worn_correctly`. Learn more about [mask plugin](Mask-detection-plugin.md)
 * landmarks2d106 - returns face landmarks. This plugin is supported only by the configuration that uses insightface library. It’s not 
   available by default. More information about landmarks [here](https://github.com/deepinsight/insightface/tree/ce3600a74209808017deaf73c036759b96a44ccb/alignment/coordinate_reg#visualization).
diff --git a/docs/Rest-API-description.md b/docs/Rest-API-description.md
@@ -473,6 +473,7 @@ Response body on success:
 |----------------------------|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | age                        | object  | detected age range. Return only if [age plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                       |
 | gender                     | object  | detected gender. Return only if [gender plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                       |
+| pose                       | object  | detected head pose. Return only if [pose plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                      |
 | mask                       | object  | detected mask. Return only if [face mask plugin](Face-services-and-plugins.md#face-plugins) is enabled.                                                     |
 | embedding                  | array   | face embeddings. Return only if [calculator plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                   |
 | box                        | object  | list of parameters of the bounding box for this face                                                                                                        |
@@ -559,6 +560,7 @@ Response body on success:
 |----------------------------|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | age                        | object  | detected age range. Return only if [age plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                       |
 | gender                     | object  | detected gender. Return only if [gender plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                       |
+| pose                       | object  | detected head pose. Return only if [pose plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                      |
 | mask                       | object  | detected mask. Return only if [face mask plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                      |
 | embedding                  | array   | face embeddings. Return only if [calculator plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                   |
 | box                        | object  | list of parameters of the bounding box for this face                                                                                                        |
@@ -639,6 +641,7 @@ Response body on success:
 |----------------------------|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | age                        | object  | detected age range. Return only if [age plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                       |
 | gender                     | object  | detected gender. Return only if [gender plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                       |
+| pose                       | object  | detected head pose. Return only if [pose plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                      |
 | mask                       | object  | detected mask. Return only if [face mask plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                      |
 | embedding                  | array   | face embeddings. Return only if [calculator plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                   |
 | box                        | object  | list of parameters of the bounding box for this face (on processedImage)                                                                                    |
@@ -757,6 +760,7 @@ Response body on success:
 | face_matches               | array   | result of face verification                                                                                                                                 |
 | age                        | object  | detected age range. Return only if [age plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                       |
 | gender                     | object  | detected gender. Return only if [gender plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                       |
+| pose                       | object  | detected head pose. Return only if [pose plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                      |
 | mask                       | object  | detected mask. Return only if [face mask plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                      |
 | embedding                  | array   | face embeddings. Return only if [calculator plugin](Face-services-and-plugins.md#face-plugins) is enabled                                                   |
 | box                        | object  | list of parameters of the bounding box for this face                                                                                                        |

diff --git a/embedding-calculator/Dockerfile b/embedding-calculator/Dockerfile
@@ -28,7 +28,7 @@ ARG GPU_IDX=-1
 ENV GPU_IDX=$GPU_IDX INTEL_OPTIMIZATION=$INTEL_OPTIMIZATION
 ARG FACE_DETECTION_PLUGIN="facenet.FaceDetector"
 ARG CALCULATION_PLUGIN="facenet.Calculator"
-ARG EXTRA_PLUGINS="facenet.LandmarksDetector,agegender.AgeDetector,agegender.GenderDetector,facenet.facemask.MaskDetector"
+ARG EXTRA_PLUGINS="facenet.LandmarksDetector,agegender.AgeDetector,agegender.GenderDetector,facenet.facemask.MaskDetector,facenet.PoseEstimator"
 ENV FACE_DETECTION_PLUGIN=$FACE_DETECTION_PLUGIN CALCULATION_PLUGIN=$CALCULATION_PLUGIN \
     EXTRA_PLUGINS=$EXTRA_PLUGINS
 COPY src src

diff --git a/embedding-calculator/README.md b/embedding-calculator/README.md
@@ -97,13 +97,15 @@ Pass to `EXTRA_PLUGINS` comma-separated names of plugins.
 |------------------------------------|----------------|-------------|------------|-------------|
 | agegender.AgeDetector              | age            | agegender   | Tensorflow |             |
 | agegender.GenderDetector           | gender         | agegender   | Tensorflow |             |
-| insightface.AgeDetector            | age            | insightface | MXNet      |      +      |
-| insightface.GenderDetector         | gender         | insightface | MXNet      |      +      |
-| facenet.LandmarksDetector          | landmarks      | Facenet     | Tensorflow |      +      |
-| insightface.LandmarksDetector      | landmarks      | insightface | MXNet      |      +      |
-| insightface.Landmarks2d106Detector | landmarks2d106 | insightface | MXNet      |      +      |
-| facenet.facemask.MaskDetector      | mask           | facemask    | Tensorflow |      +      |
-| insightface.facemask.MaskDetector  | mask           | facemask    | MXNet      |      +      |
+| insightface.AgeDetector            | age            | insightface | MXNet      | +           |
+| insightface.GenderDetector         | gender         | insightface | MXNet      | +           |
+| facenet.LandmarksDetector          | landmarks      | Facenet     | Tensorflow | +           |
+| insightface.LandmarksDetector      | landmarks      | insightface | MXNet      | +           |
+| insightface.Landmarks2d106Detector | landmarks2d106 | insightface | MXNet      | +           |
+| facenet.facemask.MaskDetector      | mask           | facemask    | Tensorflow | +           |
+| insightface.facemask.MaskDetector  | mask           | facemask    | MXNet      | +           |
+| facenet.PoseEstimator              | pose           | Facenet     | Tensorflow | +           |
+| insightface.PoseEstimator          | pose           | insightface | MXNet      | +           |
 
 Notes:    
 * `facenet.LandmarksDetector` and `insightface.LandmarksDetector` extract landmarks
@@ -116,7 +118,7 @@ Notes:
 ```
 FACE_DETECTION_PLUGIN=facenet.FaceDetector
 CALCULATION_PLUGIN=facenet.Calculator
-EXTRA_PLUGINS=agegender.AgeDetector,agegender.GenderDetector,facenet.facemask.MaskDetector 
+EXTRA_PLUGINS=facenet.LandmarksDetector,agegender.GenderDetector,agegender.AgeDetector,facenet.facemask.MaskDetector,facenet.PoseEstimator
 ```
 
 #### Pre-trained models