Create a data ingress and data visualization flow for NetCDF data rendered as Image

[aashish24]

Related to #1

[BryonLewis]

• DataType or Model in the DB representing a netCDF file
  • description - description of the data in terms of data contained in, dimensions and variables as well as multi dimensional data
• A way to take a field that has at least 3 dimensions and convert it into an image. This would require specifying X/Y dimensions (typically lat/lon) and having static values for all other dimensions besides the slider. Then the user would be able to specify a sliding dimension. This would allow a user to configure lat/longitude and then utilize that to generate images. The images could be stored in S3/MinIO.
• Endpoint would return images associated with a netCDF and these images would have their own parameters indicating their options for slicing.
• Client side would take the bounds for the images and render them as a flat image. Could use some preaching logic from DIVE to allow smooth transitions between the images.

[BryonLewis]

NetCDF Data Model:

• fileItem foreign Key
• Description - JSONField that contains the output of description information for the netCDF. This includes dimensions, variables and their sizes. I may want to do some additional processing inf the future to have them so that you can select variables that have over 3 individual variables for cutting/slicing.

NetCDF Images Configuration Model:

• parameters - a combination of X,Y, Slide and list of static parameters in a JSON field.
• owner - the user who created this CDF image list
• name - A descriptive name for the NetCDF Image Configuration. I.E -Precipitation over Time
• description - more text for some descriptive text indicating what these images are graphing
• bounds - min/max bounds of all the related images. Typically this will be computed automatically from the latitude/longitude fields but if they don't exists this allows configuring an image in a spatial location and not having it represent the data underneath. I.E maybe looking at two X/Y variables that aren't spatial over time and just assigning the visualization of it to a site where the data is relevant.
• color scheme - matplotlib color scheme for the output image, defaults to virdis.

NetCDF Image Model:

• FileItem S3 URL location for listing
• parameters - X/Y,Slider, Most likely the slider time is being used for scrubbing through the images.
• bounds - I think each image should have it's own bounds for the rare instance in which the slices for an image don't

Processing/Ingesting a NetCDF file:

• On import it should create a NetCDF Data Model and calculate the description information to be stored in the JSONField

Image Generation:

• This will take the main variable for description as well as the X/Y, Slider arguments and additional arguments for other fields.

Rest Endpoints:

• GET /netCDF/information - gets the description field from the netCDF for processing data
• POST /netCDF/generate-images - {MAIN}, {X,Y,Slider,Arg1, Arg2, Arg3} - This endpoint will generate an image configuration that will be used to specify how the images are generated in the system.
  • MAIN - This the main field that is going to be graphed based on the description, it should be a field that at least has 3 dimensions
  • X and Y are the values for X and Y dimensions of the image. Most likely this will probably be longitude and latitude. The Slider is the variable that will be used as the slider
  • Slider - The field that is adjusted through time/steps for determining what is rendered. Most likely this will be a time field but it may not
  • Arg1, Arg2, Arg3 - These are static values for additional arguments in the property. I.E. if the property has more than 3 fields these fields need to have a static value to create a consistent 2D plane through the dataset.
• GET /netCDF/{id}/image-configs - This will return a list of the available image-configs
• GET /image-config/{id} - returns the image config with the name/descriptiona nd additional parameters. Also returns a list of all of the images associated with the image config.

I don't think I need any direct endpoints to get information for the direct images themselves.

[BryonLewis]

Check The latitudes to see if they need to be limited less than -90 to 90 and more like -80 to 80 Aashish believes it is -85 to 85.

[BryonLewis]

The NetCDF PR is almost complete. The only thing left is to have it sync up with the LayerCollection and LayerRepresentation system.