Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Explore grdcut to tile images #220

Open
Esteban82 opened this issue Aug 24, 2023 · 11 comments
Open

Explore grdcut to tile images #220

Esteban82 opened this issue Aug 24, 2023 · 11 comments

Comments

@Esteban82
Copy link
Member

Testing tiling images for the server:

Paul, this failed:
gmt grdcut @earth_day_30s_p -R0/15/15/30 -Gonetile.tif=gd 

grdcut [ERROR]: gmt_gdalwrite: Output driver gd not recognized
grdcut [WARNING]: The following format drivers are configured and support output:
grdcut [WARNING]:   VRT: Virtual Raster
grdcut [WARNING]:   GTiff: GeoTIFF
grdcut [WARNING]:   COG: Cloud optimized GeoTIFF generator
grdcut [WARNING]:   NITF: National Imagery Transmission Format
grdcut [WARNING]:   HFA: Erdas Imagine Images (.img)
grdcut [WARNING]:   ELAS: ELAS
grdcut [WARNING]:   AAIGrid: Arc/Info ASCII Grid
grdcut [WARNING]:   DTED: DTED Elevation Raster
grdcut [WARNING]:   PNG: Portable Network Graphics
grdcut [WARNING]:   JPEG: JPEG JFIF
grdcut [WARNING]:   MEM: In Memory Raster
grdcut [WARNING]:   GIF: Graphics Interchange Format (.gif)
grdcut [WARNING]:   FITS: Flexible Image Transport System
grdcut [WARNING]:   XPM: X11 PixMap Format
grdcut [WARNING]:   BMP: MS Windows Device Independent Bitmap
grdcut [WARNING]:   PCIDSK: PCIDSK Database File
grdcut [WARNING]:   PCRaster: PCRaster Raster File
grdcut [WARNING]:   ILWIS: ILWIS Raster Map
grdcut [WARNING]:   SGI: SGI Image File Format 1.0
grdcut [WARNING]:   SRTMHGT: SRTMHGT File Format
grdcut [WARNING]:   Leveller: Leveller heightfield
grdcut [WARNING]:   Terragen: Terragen heightfield
grdcut [WARNING]:   GMT: GMT NetCDF Grid Format
grdcut [WARNING]:   netCDF: Network Common Data Format
grdcut [WARNING]:   HDF4Image: HDF4 Dataset
grdcut [WARNING]:   ISIS3: USGS Astrogeology ISIS cube (Version 3)
grdcut [WARNING]:   ISIS2: USGS Astrogeology ISIS cube (Version 2)
grdcut [WARNING]:   PDS4: NASA Planetary Data System 4
grdcut [WARNING]:   VICAR: MIPL VICAR file
grdcut [WARNING]:   ERS: ERMapper .ers Labelled
grdcut [WARNING]:   JP2OpenJPEG: JPEG-2000 driver based on OpenJPEG library
grdcut [WARNING]:   FIT: FIT Image
grdcut [WARNING]:   GRIB: GRIdded Binary (.grb, .grb2)
grdcut [WARNING]:   RMF: Raster Matrix Format
grdcut [WARNING]:   WMS: OGC Web Map Service
grdcut [WARNING]:   RST: Idrisi Raster A.1
grdcut [WARNING]:   INGR: Intergraph Raster
grdcut [WARNING]:   GSAG: Golden Software ASCII Grid (.grd)
grdcut [WARNING]:   GSBG: Golden Software Binary Grid (.grd)
grdcut [WARNING]:   GS7BG: Golden Software 7 Binary Grid (.grd)
grdcut [WARNING]:   R: R Object Data Store
grdcut [WARNING]:   KMLSUPEROVERLAY: Kml Super Overlay
grdcut [WARNING]:   WEBP: WEBP
grdcut [WARNING]:   PDF: Geospatial PDF
grdcut [WARNING]:   Rasterlite: Rasterlite
grdcut [WARNING]:   MBTiles: MBTiles
grdcut [WARNING]:   CALS: CALS (Type 1)
grdcut [WARNING]:   WMTS: OGC Web Map Tile Service
grdcut [WARNING]:   MRF: Meta Raster Format
grdcut [WARNING]:   PNM: Portable Pixmap Format (netpbm)
grdcut [WARNING]:   PAux: PCI .aux Labelled
grdcut [WARNING]:   MFF: Vexcel MFF Raster
grdcut [WARNING]:   MFF2: Vexcel MFF2 (HKV) Raster
grdcut [WARNING]:   BT: VTP .bt (Binary Terrain) 1.3 Format
grdcut [WARNING]:   LAN: Erdas .LAN/.GIS
grdcut [WARNING]:   IDA: Image Data and Analysis
grdcut [WARNING]:   LCP: FARSITE v.4 Landscape File (.lcp)
grdcut [WARNING]:   GTX: NOAA Vertical Datum .GTX
grdcut [WARNING]:   NTv2: NTv2 Datum Grid Shift
grdcut [WARNING]:   CTable2: CTable2 Datum Grid Shift
grdcut [WARNING]:   KRO: KOLOR Raw
grdcut [WARNING]:   ROI_PAC: ROI_PAC raster
grdcut [WARNING]:   RRASTER: R Raster
grdcut [WARNING]:   BYN: Natural Resources Canada's Geoid
grdcut [WARNING]:   ARG: Azavea Raster Grid format
grdcut [WARNING]:   USGSDEM: USGS Optional ASCII DEM (and CDED)
grdcut [WARNING]:   BAG: Bathymetry Attributed Grid
grdcut [WARNING]:   NWT_GRD: Northwood Numeric Grid Format .grd/.tab
grdcut [WARNING]:   ADRG: ARC Digitized Raster Graphics
grdcut [WARNING]:   BLX: Magellan topo (.blx)
grdcut [WARNING]:   PostGISRaster: PostGIS Raster driver
grdcut [WARNING]:   SAGA: SAGA GIS Binary Grid (.sdat, .sg-grd-z)
grdcut [WARNING]:   XYZ: ASCII Gridded XYZ
grdcut [WARNING]:   HF2: HF2/HFZ heightfield raster
grdcut [WARNING]:   JPEGLS: JPEGLS
grdcut [WARNING]:   ZMap: ZMap Plus Grid
grdcut [WARNING]:   SIGDEM: Scaled Integer Gridded DEM .sigdem
grdcut [WARNING]:   GNMFile: Geographic Network generic file based model
grdcut [WARNING]:   GNMDatabase: Geographic Network generic DB based model
grdcut [WARNING]:   ESRI Shapefile: ESRI Shapefile
grdcut [WARNING]:   MapInfo File: MapInfo File
grdcut [WARNING]:   S57: IHO S-57 (ENC)
grdcut [WARNING]:   DGN: Microstation DGN
grdcut [WARNING]:   Memory: Memory
grdcut [WARNING]:   CSV: Comma Separated Value (.csv)
grdcut [WARNING]:   GML: Geography Markup Language (GML)
grdcut [WARNING]:   GPX: GPX
grdcut [WARNING]:   LIBKML: Keyhole Markup Language (LIBKML)
grdcut [WARNING]:   KML: Keyhole Markup Language (KML)
grdcut [WARNING]:   GeoJSON: GeoJSON
grdcut [WARNING]:   GeoJSONSeq: GeoJSON Sequence
grdcut [WARNING]:   Interlis 1: Interlis 1
grdcut [WARNING]:   Interlis 2: Interlis 2
grdcut [WARNING]:   OGR_GMT: GMT ASCII Vectors (.gmt)
grdcut [WARNING]:   GPKG: GeoPackage
grdcut [WARNING]:   SQLite: SQLite / Spatialite
grdcut [WARNING]:   WAsP: WAsP .map format
grdcut [WARNING]:   MSSQLSpatial: Microsoft SQL Server Spatial Database
grdcut [WARNING]:   PostgreSQL: PostgreSQL/PostGIS
grdcut [WARNING]:   MySQL: MySQL
grdcut [WARNING]:   DXF: AutoCAD DXF
grdcut [WARNING]:   FlatGeobuf: FlatGeobuf
grdcut [WARNING]:   Geoconcept: Geoconcept
grdcut [WARNING]:   GeoRSS: GeoRSS
grdcut [WARNING]:   GPSTrackMaker: GPSTrackMaker
grdcut [WARNING]:   PGDUMP: PostgreSQL SQL dump
grdcut [WARNING]:   GPSBabel: GPSBabel
grdcut [WARNING]:   CouchDB: CouchDB / GeoCouch
grdcut [WARNING]:   Cloudant: Cloudant / CouchDB
grdcut [WARNING]:   ODS: Open Document/ LibreOffice / OpenOffice Spreadsheet 
grdcut [WARNING]:   XLSX: MS Office Open XML spreadsheet
grdcut [WARNING]:   Elasticsearch: Elastic Search
grdcut [WARNING]:   Carto: Carto
grdcut [WARNING]:   AmigoCloud: AmigoCloud
grdcut [WARNING]:   Selafin: Selafin
grdcut [WARNING]:   JML: OpenJUMP JML
grdcut [WARNING]:   VDV: VDV-451/VDV-452/INTREST Data Format
grdcut [WARNING]:   GMLAS: Geography Markup Language (GML) driven by application schemas
grdcut [WARNING]:   MVT: Mapbox Vector Tiles
grdcut [WARNING]:   NGW: NextGIS Web
grdcut [WARNING]:   MapML: MapML
grdcut [WARNING]:   TIGER: U.S. Census TIGER/Line
grdcut [WARNING]:   ENVI: ENVI .hdr Labelled
grdcut [WARNING]:   EHdr: ESRI .hdr Labelled
grdcut [WARNING]:   ISCE: ISCE raster
grdcut [WARNING]:   Zarr: Zarr
@Esteban82
Copy link
Member Author

These two worked:

gmt grdcut @earth_day_30s_p.tif -R0/10/0/10 -Gonetile.tiff=GTiff
gmt grdcut @earth_day_30s_p.tif -R0/10/0/10 -Gonetile.png=PNG

onetile

It is not supported to upload the Gtiff file.

@Esteban82
Copy link
Member Author

I even could make a map with correct annotations:

gmt grdcut @earth_day_30s_p.tif -R-60/-50/0/10 -Gonetile.png=PNG
gmt grdimage onetile.png -Baf -png test

test

@PaulWessel
Copy link
Member

Good. I noticed two further points:

  1. No need to append =PNG or =GTiff, just having the right file extension png, tiff etc. is enough for us to set the right driver.
  2. The tiff with our without =GTiff was still a geotiff and identical

@Esteban82
Copy link
Member Author

This looks good. Do we have those gdal "tools" in GMT?

# 1. Create two tiles
gmt grdcut @earth_day_30s_p.tif -R-70/-60/0/10 -G1.tiff
gmt grdcut @earth_day_30s_p.tif -R-60/-50/0/10 -G2.tiff

# 2, Merge files
gdalbuildvrt mosaic.vrt ?.tiff
gdal_translate -of GTiff -co "TILED=YES" mosaic.vrt mosaic.tiff

# 3. Test merge file
gmt grdimage mosaic.tiff -Baf -png test

test

@joa-quim
Copy link
Member

Do we have those gdal "tools" in GMT?

On Windows yes. Others, depend on how they installed GDAL.

@PaulWessel
Copy link
Member

My GDAL on macOS has them. Perhaps

  1. Split those big images into managable tiles as done with for grids. E.g., write src_tiler_img.sh based on src_tiler.sh (and rename that one src_tiler_grid.sh).
  2. Given -R, determine which tiles we need (as for grids) but then call gdalbuildvrt and gdal_translate to build the combo mosaic.
  3. Plot that thing.

@joa-quim
Copy link
Member

Own builds are not guaranteed to have the gdal tools. But we have grdgdal that works as gdal_teanslate.

@PaulWessel
Copy link
Member

The installers have the GDAL tools we use so can easily add a few more if needed. As for building from source, we require GDAL I think.

@joa-quim
Copy link
Member

We indicate gdal-dev not the package with the binaries.

@anbj
Copy link

anbj commented Aug 25, 2023

We indicate gdal-dev not the package with the binaries.

But gdal-bin is listed in the optional packages.

@Esteban82
Copy link
Member Author

Here I made a more complex test leaving some empty places to see if gdal can merge the tiles. It did it well.

test

Full script

# 1. Create tiling
gmt grdcut @earth_day_30s_p.tif -R-70/-60/0/10 -G1.tiff #=PNG
gmt grdcut @earth_day_30s_p.tif -R-60/-50/0/10 -G2.tiff #=PNG
gmt grdcut @earth_day_30s_p.tif -R-70/-60/-20/-10 -G5.tiff #=PNG

# 2, Merge files
gdalbuildvrt mosaic.vrt ?.tiff
gdal_translate -of GTiff -co "TILED=YES" mosaic.vrt mosaic.tiff

# 3. Test merge
gmt grdimage mosaic.tiff -Baf -png test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@joa-quim @Esteban82 @PaulWessel @anbj