-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: DOC-231: Update docs for multi-page doc annotation (#6725)
Co-authored-by: caitlinwheeless <[email protected]> Co-authored-by: robot-ci-heartex <[email protected]> Co-authored-by: hlomzik <[email protected]> Co-authored-by: caitlinwheeless <[email protected]>
- Loading branch information
5 people
authored
Dec 3, 2024
1 parent
3f88d0a
commit b7a3b93
Showing
4 changed files
with
117 additions
and
170 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
283 changes: 115 additions & 168 deletions
283
docs/source/templates/multi-page-document-annotation.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,212 +1,159 @@ | ||
| --- | ||
| title: Multi-page Document Annotation | ||
|
|
||
| title: Multi-Page Document Annotation | ||
| type: templates | ||
| category: Computer Vision | ||
| cat: computer-vision | ||
| badge: <i class='ent'></i> | ||
| order: 112 | ||
| is_new: t | ||
| meta_title: Multi-page Document Annotation | ||
| meta_description: Template for labeling large, multi-page documents more easily and efficiently. | ||
| --- | ||
|
|
||
| With Label Studio Enterprise, the multi-page document annotation feature allows you to label large multi-page documents more easily and efficiently. You can now upload hundreds of pages as a single document, allowing navigation and annotation on multiple pages while maintaining the entire document's context. This release eliminates the significant amount of lag in loading the images into the browser and enhances the labeling speed. | ||
| <img src="/images/templates/multipage.png" alt="" class="gif-border" /> | ||
|
|
||
| Multi-page document annotation allows you to annotate documents that consist of multiple pages. By representing each page as an image, you can import them into a single task and annotate each page while maintaining the context of the entire document. | ||
|
|
||
| If you just need to classify a PDF document as a whole, you can also use the [PDF Classification template](pdf_classification). | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| The prerequisites to use the multi-page document annotation detection feature are as follows: | ||
| Multi-page document annotation requires that you first pre-process your document by converting it into separate images. There are multiple tools to do this, including a built-in option in [Adobe Acrobat](https://helpx.adobe.com/acrobat/using/pdf-to-jpg.html). | ||
|
|
||
| 1. Before uploading a PDF document to Label Studio, you must pre-process the document by converting it into separate images. However, you can still annotate PDF documents directly, for example, by performing [classification on each document page](/templates/pdf_classification.html). | ||
| 2. In Label Studio, import all the images into the browser using the following format: | ||
| ```json | ||
| [{ | ||
| "data": { | ||
| "pages": [{ | ||
| "image": "image-1-url" | ||
| }, { | ||
| "image": "image-2-url" | ||
| }, { | ||
| "image": "image-3-url" | ||
| }] | ||
| } | ||
| }] | ||
| To improve performance, we preload images by downloading an image via XHR and storing it in the local browser’s storage. For this to function properly, you must set up a CORS policy on the CDN/storage where the images are located. Specifically, the following headers must be set: | ||
|
|
||
| ``` | ||
| Access-Control-Allow-Origin: https://app.heartex.com | ||
| Access-Control-Allow-Methods: GET, OPTIONS | ||
| Access-Control-Allow-Headers: Content-Type | ||
| ``` | ||
| 3. Now, check if the projects using **Repeater** tag is paginated. | ||
|
|
||
| ## Limitations | ||
|
|
||
| ## Key capabilities supported | ||
| For performance reasons, we recommend that the maximum number of tasks per project be limited to around 100,000. | ||
|
|
||
| The following key capabilities are supported by the multi-page document annotation. | ||
| If you are creating tasks with multiple images, you may need to plan for a smaller task limit within each project. For example, a task with 100 images would be roughly equivalent to 100 tasks (in terms of performance considerations). | ||
|
|
||
| 1. Support one image per screen or one text per screen. This can include a large text, but it should be the only document that requires annotation. In several scenarios, there is a need to annotate multi-page documents. For example, when you have PDF documents that consist of multiple pages then you must annotate the whole document as one task. This implies the annotator receives this one multi-page document for annotation. | ||
| ## Page navigation | ||
|
|
||
| 2. Use Hot-key driven annotation using the left arrow key to move upwards and the right arrow key to move downwards in a multi-page document. | ||
| When performing multi-page annotations, you'll see a new navigation menu: | ||
|
|
||
| 3. Introduce API driven process to customize the pagination in a multi-page document. | ||
|  | ||
|
|
||
| 4. Help companies and organizations that deal with large, complex documents, including extensive research papers and healthcare, legal, or financial documents that run between dozens and hundreds of pages long. | ||
| !!! info Tip | ||
| * To navigate to a specific page, click on the page numbers in the center and enter the desired page. | ||
| * You can also use the following hot keys to navigate: `command + left`/`command + right` or `ctrl + left`/`ctrl + right`. | ||
|
|
||
|
|
||
| ## Labeling configuration | ||
|
|
||
| The simplest way to work with multi-page document annotation is to go in [project settings](/guide/setup.html#Modify-the-labeling-interface) and specify the following labeling configuration. | ||
| Enable multi-page annotation by using the `<Image>` tag and setting the `valueList` parameter instead of the `value` parameter. | ||
|
|
||
| ```html | ||
| ```xml | ||
| <View> | ||
| <Repeater on="$images" indexFlag="{{idx}}" mode="pagination"> | ||
| <Image name="page_{{idx}}" value="$images[{{idx}}].url"/> | ||
| <Header value="Utterance Review"/> | ||
| <RectangleLabels name="labels_{{idx}}" toName="page_{{idx}}"> | ||
| <Label value="Document Title" /> | ||
| <Label value="Document Date" /> | ||
| </RectangleLabels> | ||
| <Taxonomy | ||
| name="categories_{{idx}}" | ||
| toName="page_{{idx}}" | ||
| perRegion="true" | ||
| visibleWhen="region-selected" | ||
| > | ||
| <Choices name="utterance_action_{{idx}}" showInline="true" toName="user_{{idx}}"> | ||
| <Choice value="Archaea"/> | ||
| <Choice value="Bacteria"/> | ||
| <Choice value="Eukarya"> | ||
| <Choice value="Human"/> | ||
| <Choice value="Oppossum"/> | ||
| <Choice value="Extraterrestrial"/> | ||
| </Choices> | ||
| </Taxonomy> | ||
| </Repeater> | ||
| <RectangleLabels name="rectangles" toName="pdf" showInline="true"> | ||
| <Label value="Title" background="red"/> | ||
| <Label value="Summary" background="blue"/> | ||
| <Label value="Author" background="green"/> | ||
| <Label value="Table" background="gold"/> | ||
| <Label value="Citation" background="purple"/> | ||
| </RectangleLabels> | ||
| <Image valueList="$pages" name="pdf"/> | ||
| </View> | ||
| ``` | ||
|
|
||
| ## About the labeling configuration | ||
| Similar to `value`, the `valueList` parameter accepts a variable. The difference in this case is that you are referencing an array in your JSON. | ||
|
|
||
| All labeling configurations must be wrapped in [View](/tags/view.html) tags. | ||
| For example, the following references `pages` in the [example input data](#Input-data-format) below. | ||
|
|
||
| You can add a [header](/tags/header.html) tag to provide instructions to the annotator: | ||
| ```xml | ||
| <Header>Label the video:</Header> | ||
| <Image valueList="$pages" name="pdf"/> | ||
| ``` | ||
|
|
||
| Use the [Choices](/tags/choices.html) tag to create a group of choices, with radio buttons, or checkboxes. | ||
|
|
||
| ```xml | ||
| <Choices name="utterance_action_{{idx}}" showInline="true" toName="user_{{idx}}"> | ||
| ``` | ||
|
|
||
|
|
||
| ## Label Studio UI enhancements | ||
|
|
||
| The multi-page document annotation feature includes the following UI enhancements: | ||
|
|
||
| 1. Ability to add `<Repeater mode="pagination" ...>` parameter in the **Repeater** tab labeling configuration. | ||
|
|
||
| 2. A pagination box that appears on every page (in a multi-page document) that allows quick navigation to the first page, last page, or specific page efficiently. | ||
|
|
||
| 3. An option to use **Outliner** for scrolling from one page to another in a multi-page document. | ||
|
|
||
|
|
||
| ## Use cases | ||
| The multi-page document annotation feature provides the following use cases: | ||
|
|
||
| ### Draw bounding boxes | ||
|
|
||
| To draw bounding boxes over some specific elements inside a multi-page (100 pages) PDF document: | ||
|
|
||
| 1. Convert the PDF document into 100 images because as per this example, there are 100 pages in a PDF document. | ||
| 2. Store the images to create a task (with 100 images). | ||
| 3. Import data as JSON tasks with the list of 100 images. | ||
| 4. When the interface is loaded with 100 images, you can draw bounding boxes inside the images. You can also use the **Repeater** tag with pagination to navigate back and forth throughout these multi-page documents and annotate images. | ||
|
|
||
| <img src="../images/draw-bb-multipage-document-annotation.png" class="gif-border" /> | ||
|
|
||
| <i>Figure 1: Use bounding boxes. </i> | ||
|
|
||
| ### Page scroll | ||
|
|
||
| By default, the multi-page document annotation did not support pagination as shown in Figure 2. | ||
|
|
||
| <img src="../images/without-pagination-option.png" class="gif-border" /> | ||
|
|
||
| <i>Figure 2: Without pagination. </i> | ||
|
|
||
| To find a page or a specific page number, use your mouse pad to scroll down the multi-page document. | ||
|
|
||
|
|
||
| ### Page navigation control | ||
|
|
||
| You can use `mode` property to the **Repeater** tag and the multi-page document annotation with pagination capability to annotate multiple pages in a single document: | ||
|
|
||
| 1. In the Label Studio UI, navigate to **Settings** >> **Labeling Interface** >> **Code**. | ||
|
|
||
| <img src="../images/add-mode-pagination-location.png" class="gif-border" /> | ||
| !!! note Classification | ||
| This example uses bounding box labeling. However, you can modify the template to perform classification tasks instead (using the [`Choices` tag](/tags/choices)). When using the `Choices` tag, note the following: | ||
|
|
||
| * You can perform per-page classification using `perItem="true"`. | ||
| * You can also leave `perItem="false"` (the default) to perform per-document classification. | ||
| * `perRegion="true"` is supported when drawing regions on pages. | ||
|
|
||
| <i>Figure 3: Location to add mode pagination. </i> | ||
| ## Input data format | ||
|
|
||
| The following example would upload a 5-page PDF that had been pre-processed into 5 images. You would reference the pages in your labeling configuration using `valueList="$pages"`. | ||
|
|
||
| 2. Add **mode="pagination"** inside the **Repeater** tag. | ||
|
|
||
| <img src="../images/add-mode-pagination.png" class="gif-border" /> | ||
|
|
||
| <i>Figure 4: Added mode property. </i> | ||
|
|
||
| 3. Now, return to the previous screen and click on the task. | ||
|
|
||
| <img src="../images/return-back-to-the same-project.png" class="gif-border" /> | ||
|
|
||
| <i>Figure 5: Return to the previous screen. </i> | ||
|
|
||
| 4. The pagination feature is available in the UI preview window. | ||
|
|
||
| <img src="../images/with-pagination-option.png" class="gif-border" /> | ||
|
|
||
| <i>Figure 6: Pagination option in the UI preview. </i> | ||
|
|
||
| 5. The pagination option is now available on every page of the multi-page document. | ||
| <img src="../images/pagination-option-on-each-page.png" class="gif-border" /> | ||
|
|
||
| <i>Figure 7: Pagination option on every page. </i> | ||
|
|
||
|
|
||
| To find a page or a specific page number, use the navigation buttons in the pagination box. | ||
|
|
||
| 1. Click the `<<` button to navigate to the first page and click the `<` button to navigate to the previous page in a multi-page document. | ||
|
|
||
| <img src="../images/button-to-navigate-to-first-page.png" class="gif-border" /> | ||
|
|
||
| <i>Figure 11: Button to navigate to the first page. </i> | ||
|
|
||
|
|
||
| View of the first page using the navigation button. | ||
| <img src="../images/navigate-to-first-page.png" class="gif-border" /> | ||
|
|
||
| <i>Figure 12: Navigated to the first page. </i> | ||
|
|
||
|
|
||
| 2. Click the `>>` button to navigate to the last page and click the `>` button to navigate to the next page in a multi-page document. | ||
|
|
||
| <img src="../images/button-to-navigate-to-last-page.png" class="gif-border" /> | ||
|
|
||
| <i>Figure 13: Button to navigate to the last page. </i> | ||
|
|
||
| View of the last page using the navigation button. | ||
| <img src="../images/navigate-to-last-page.png" class="gif-border" /> | ||
|
|
||
| <i>Figure 14: Navigated to the first page. </i> | ||
|
|
||
|
|
||
| ### Page navigation with Outliner | ||
| ```json | ||
| { | ||
| "data": { | ||
| "pages": [ | ||
| "https://htx-pub.s3.amazonaws.com/demo/images/demo_stock_purchase_agreement/0001.jpg", | ||
| "https://htx-pub.s3.amazonaws.com/demo/images/demo_stock_purchase_agreement/0002.jpg", | ||
| "https://htx-pub.s3.amazonaws.com/demo/images/demo_stock_purchase_agreement/0003.jpg" | ||
| ] | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| To find a page or a specific page number, use the outliner feature. | ||
| ## Export format | ||
|
|
||
| Example of using outliner document title: | ||
| When exporting your annotations, the `item_index` property returns the index of the image that a region is attached to. Indexes start with 0, meaning the first image will be `0`, the second will be `1`, and so on: | ||
|
|
||
| <img src="../images/navigate-to-pages-using-outliner-document-title.png" class="gif-border" /> | ||
| <i>Figure 9: Navigate using outliner document title. </i> | ||
| ```json | ||
| [ | ||
| { | ||
| "id": 859, | ||
| "annotations": [ | ||
| { | ||
| "id": 25569355, | ||
| "result": [ | ||
| { | ||
| "id": "tdS8Of63VJ", | ||
| "type": "rectanglelabels", | ||
| "value": { | ||
| "x": 2.48868778280543, | ||
| "y": 4.807692307692308, | ||
| "width": 52.88461538461539, | ||
| "height": 5.128205128205128, | ||
| "rotation": 0, | ||
| "rectanglelabels": [ | ||
| "Title" | ||
| ] | ||
| }, | ||
| "origin": "manual", | ||
| "to_name": "pdf", | ||
| "from_name": "choices", | ||
| "item_index": 0, | ||
| "image_rotation": 0, | ||
| "original_width": 2550, | ||
| "original_height": 3300 | ||
| }, | ||
| { | ||
| "id": "RGGrHm95R7", | ||
| "type": "rectanglelabels", | ||
| "value": { | ||
| "x": 30.693815987933636, | ||
| "y": 12.179487179487179, | ||
| "width": 58.4841628959276, | ||
| "height": 1.2820512820512828, | ||
| "rotation": 0, | ||
| "rectanglelabels": [ | ||
| "Citation" | ||
| ] | ||
| }, | ||
| "origin": "manual", | ||
| "to_name": "pdf", | ||
| "from_name": "choices", | ||
| "item_index": 1, | ||
| "image_rotation": 0, | ||
| "original_width": 2550, | ||
| "original_height": 3300 | ||
| } | ||
| ] | ||
| } | ||
| ] | ||
| } | ||
| ] | ||
|
|
||
| ``` | ||
|
|
||
| ## Related tags | ||
| - [Choices](/tags/choices.html) | ||
| - [Labels](/tags/labels.html) | ||
| - [RectangleLabels](/tags/rectanglelabels.html) | ||
| - [Image](/tags/image) | ||
| - [Label](/tags/label) | ||
| - [RectangleLabels](/tags/rectanglelabels) |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters