Organizing Your Data
Organizing your data before using the ingest client will help you maximize its use. The ingest client currently supports two types of ingests: tile based and volumetric. As such the way that the data is prepared before ingest will differ depending on which one will be during the ingest process. Please make sure to read the Creating Ingest Job Configuration Files page to make sure you make the right decision on the type of ingest you would benefit most from.
We recommend that you stitch up your data before proceeding. For example if you currently have your data in the following format:
You should consider stitching it up to follow the format:
We provide a tiling option in our ingest configuration process, that will automatically grab your data slice FORMAT B and break it into tiles similar to FORMAT A of a specified size before uploading.
A typical directory should look something like this:
/MY_DATA
- tile_1
- tile_2
- tile_3
NOTE: Please try not to have any changing parts of the name that are not the z slice number change. Ingest client knows how to handle * cards if the data is in S3, but not in a local file system. This means that Data organized in this way:
/MY_DATA
- tile_time1_1
- tile_time3_2
- tile_time18_3
Will not be a valid data organization format, and the ingest client will have difficulty spatially tracking tiles.