|
| 1 | +# GDAL Support |
| 2 | + |
| 3 | +<script src="https://asc-public-docs.s3.us-west-2.amazonaws.com/common/scripts/isis-demos/jquery-3.7.1.min.js"></script> |
| 4 | +<link href="../../../css/isis-demos.css" media="all" rel="stylesheet"/> |
| 5 | + |
| 6 | + |
| 7 | +<script type="text/javascript"> |
| 8 | +if (typeof window.isisDemosLoaded == 'undefined') { |
| 9 | + var isisDemosLoaded = true; |
| 10 | + $.getScript("https://asc-public-docs.s3.us-west-2.amazonaws.com/common/scripts/isis-demos/easeljs-0.8.1.min.js").done( function(s,t) { $.getScript("../../../js/isisDemos.js");}); |
| 11 | +} |
| 12 | +</script> |
| 13 | + |
| 14 | + |
| 15 | +The Geospatial Data Abstraction Library (GDAL) provides a broad range of data formats that ISIS could read or write. Currently, ISIS only supports Geospatial Tag Image File Format (GeoTiff) but has the potential to expand to other desired formats. How to access each format along with any potential quirks of processing using said formats will be detailed below. |
| 16 | + |
| 17 | +## Processing With GeoTIFFs |
| 18 | + |
| 19 | +Users wanting to process using GeoTIFFs only have to use `+GTiff` as an output attribute similar to cube [storage formats](../../concepts/isis-fundamentals/command-line-usage.md#storage-format) in ISIS found under the Command Line Usage page in the Isis Fundamentals. This will produce a `.tiff` file and a `.tiff.msk` file as your output file. For example: |
| 20 | + |
| 21 | +``` |
| 22 | +mroctx2isis from=mroraw.img to=mrotiff.tiff+GTiff |
| 23 | +``` |
| 24 | +results in |
| 25 | + |
| 26 | +``` |
| 27 | +mrotiff.tiff |
| 28 | +mrotiff.tiff.msk |
| 29 | +``` |
| 30 | +The GeoTIFFs can then be used as input like cubes to other applications. |
| 31 | + |
| 32 | +One can also combine supported output attributes with the `+GTiff` attribute. The `GTiff` attribute will work with all ISIS [pixel type attributes](../../concepts/isis-fundamentals/command-line-usage.md#pixel-type) and [label format attributes](../../concepts/isis-fundamentals/command-line-usage.md#label-format). [Pixel storage order](../../concepts/isis-fundamentals/command-line-usage.md#pixel-storage-order) does not work and will not perform any alterations to the byte ordering. |
| 33 | + |
| 34 | + |
| 35 | +When using +GTiff in ISIS, the current defaults will automatically set a DEFLATE compression with PREDICTOR=2. This type of compression is lossless and supports all bit types. Overviews are not automatically created by ISIS, but they can be added to any TIFF using the GDAL routine gdaladdo. |
| 36 | + |
| 37 | +Technically, a fully realized GeoTIFF is only enabled when the data is map projected. For images that are not yet map projected, the underlying TIFF format will still be used, but there will be no geospatial map projection in the ISIS or TIFF label. For more information on the Open Geospatial Consortium GeoTIFF specification please [see](https://www.ogc.org/publications/standard/geotiff/). |
| 38 | + |
| 39 | +### Processing With Cloud Data |
| 40 | + |
| 41 | +GDAL allows for access to online/cloud volumes. Now that ISIS supports reading/writing GeoTIFFs, this includes accessing online/cloud data. |
| 42 | + |
| 43 | +When working with online/cloud volumes, one can access these volumes by adding `/vsicurl/` in front of the URL for your data. See [this](https://gdal.org/en/stable/user/virtual_file_systems.html) GDAL page for more information on `vsicurl` An example process can look something like this: |
| 44 | +``` |
| 45 | +catlab from=/vsicurl/https://astrogeo-ard.s3-us-west-2.amazonaws.com/mars/mro/ctx/controlled/usgs/T01_000881_1752_XI_04S223W__P22_009716_1773_XI_02S223W/T01_000881_1752_XI_04S223W__P22_009716_1773_XI_02S223W-DEM.tif |
| 46 | +``` |
| 47 | +Qview can also access files in the same way, one can run |
| 48 | +``` |
| 49 | +qview /vsicurl/https://astrogeo-ard.s3-us-west-2.amazonaws.com/mars/mro/ctx/controlled/usgs/T01_000881_1752_XI_04S223W__P22_009716_1773_XI_02S223W/T01_000881_1752_XI_04S223W__P22_009716_1773_XI_02S223W-DEM.tif |
| 50 | +``` |
| 51 | +and see cloud data in qview. |
| 52 | + |
| 53 | +Many cloud volumes will have overviews, or compressed versions of the image that can be requested for display rather than the full resolution image. |
| 54 | + |
| 55 | +The best use of overviews is for responsiveness over the web. If an online dataset has overviews, users will be able to stream the overview data rather than the full resolution data. This decreases the amount of data needed to display the image. For example, if someone is displaying a 500 x 500 portion of a 1000 x 1000 image at .25 scale that has overviews, GDAL will extract the DNs from the overview with a downsampling of 8. So instead of having to transfer all 250000 DN values over the network then subsampled, only 15625 downsampled DNs will to be transferred. |
| 56 | + |
| 57 | +Below is a comparison between a GeoTIFF with overviews and a GeoTIFF with no overviews. If a GeoTIFF does not have overviews it will be subsampled based on the scale as ISIS already does with cubes. When using overviews, the image will be sampled at the closest resolution to the requested resolution. The images will look slightly different as a result when viewed in Qview. As shown in the gif below, the images look like they are changing but in reality, `B10_013341_1010_XN_79S172W_no_ovr.cal.tiff` is being subsampled while `B10_013341_1010_XN_79S172W.cal.tiff` is using overviews. |
| 58 | + |
| 59 | + |
| 60 | + |
| 61 | +For information on creating overviews, checkout the section on [Working with GDAL Products Outside of ISIS](gdal-support.md#working-with-gdal-products-outside-of-isis). |
| 62 | + |
| 63 | +### ISIS Specific GeoTIFF Data |
| 64 | + |
| 65 | +All axiliary processing data used in ISIS will be stored onto the GeoTIFF as JSON. As mentioned in ISIS [RFC 13](https://github.com/DOI-USGS/ISIS3/discussions/5698), each blob/table will be stored as a string under a keyword that represents the blob. All tables are denoted as "Table_`NAME`" and all blobs are "`OBJECT`_`NAME`". For example the ISIS `SunPosition` table can be found under `Table_SunPosition`, and the ISIS `History` blob can be found under `History_IsisCube`. |
| 66 | + |
| 67 | +Below is an explicit example of how the table looks on the TIFF vs the Cube: |
| 68 | + |
| 69 | +Table Metadata from Cube label |
| 70 | +``` |
| 71 | +Object = Table |
| 72 | + Name = SunPosition |
| 73 | + StartByte = 493963254 |
| 74 | + Bytes = 112 |
| 75 | + Records = 2 |
| 76 | + ByteOrder = Lsb |
| 77 | + CacheType = Linear |
| 78 | + SpkTableStartTime = 297088762.24158 |
| 79 | + SpkTableEndTime = 297088808.37074 |
| 80 | + SpkTableOriginalSize = 2.0 |
| 81 | + Description = "Created by spiceinit" |
| 82 | + Kernels = ($base/kernels/spk/de430.bsp, |
| 83 | + $base/kernels/spk/mar097.bsp) |
| 84 | +
|
| 85 | + Group = Field |
| 86 | + Name = J2000X |
| 87 | + Type = Double |
| 88 | + Size = 1 |
| 89 | + End_Group |
| 90 | +
|
| 91 | + Group = Field |
| 92 | + Name = J2000Y |
| 93 | + Type = Double |
| 94 | + Size = 1 |
| 95 | + End_Group |
| 96 | +
|
| 97 | + Group = Field |
| 98 | + Name = J2000Z |
| 99 | + Type = Double |
| 100 | + Size = 1 |
| 101 | + End_Group |
| 102 | +
|
| 103 | + Group = Field |
| 104 | + Name = J2000XV |
| 105 | + Type = Double |
| 106 | + Size = 1 |
| 107 | + End_Group |
| 108 | +
|
| 109 | + Group = Field |
| 110 | + Name = J2000YV |
| 111 | + Type = Double |
| 112 | + Size = 1 |
| 113 | + End_Group |
| 114 | +
|
| 115 | + Group = Field |
| 116 | + Name = J2000ZV |
| 117 | + Type = Double |
| 118 | + Size = 1 |
| 119 | + End_Group |
| 120 | +
|
| 121 | + Group = Field |
| 122 | + Name = ET |
| 123 | + Type = Double |
| 124 | + Size = 1 |
| 125 | + End_Group |
| 126 | +End_Object |
| 127 | +``` |
| 128 | + |
| 129 | +Table Metadata and data on TIFF |
| 130 | +```json |
| 131 | +{"Table_SunPosition": |
| 132 | + '{ |
| 133 | + "_container_name":"Table", |
| 134 | + "_type":"object", |
| 135 | + "Field_J2000X":{ |
| 136 | + "_container_name":"Field", |
| 137 | + "_type":"group", |
| 138 | + "Name":"J2000X", |
| 139 | + "Type":"Double", |
| 140 | + "Size":"1" |
| 141 | + }, |
| 142 | + "Field_J2000Y":{ |
| 143 | + "_container_name":"Field", |
| 144 | + "_type":"group", |
| 145 | + "Name":"J2000Y", |
| 146 | + "Type":"Double", |
| 147 | + "Size":"1" |
| 148 | + }, |
| 149 | + "Field_J2000Z":{ |
| 150 | + "_container_name":"Field", |
| 151 | + "_type":"group", |
| 152 | + "Name":"J2000Z", |
| 153 | + "Type":"Double", |
| 154 | + "Size":"1" |
| 155 | + }, |
| 156 | + "Field_J2000XV":{ |
| 157 | + "_container_name":"Field", |
| 158 | + "_type":"group", |
| 159 | + "Name":"J2000XV", |
| 160 | + "Type":"Double", |
| 161 | + "Size":"1" |
| 162 | + }, |
| 163 | + "Field_J2000YV":{ |
| 164 | + "_container_name":"Field", |
| 165 | + "_type":"group", |
| 166 | + "Name":"J2000YV", |
| 167 | + "Type":"Double", |
| 168 | + "Size":"1" |
| 169 | + }, |
| 170 | + "Field_J2000ZV":{ |
| 171 | + "_container_name":"Field", |
| 172 | + "_type":"group", |
| 173 | + "Name":"J2000ZV", |
| 174 | + "Type":"Double", |
| 175 | + "Size":"1" |
| 176 | + }, |
| 177 | + "Field_ET":{ |
| 178 | + "_container_name":"Field", |
| 179 | + "_type":"group", |
| 180 | + "Name":"ET", |
| 181 | + "Type":"Double", |
| 182 | + "Size":"1" |
| 183 | + }, |
| 184 | + "Name":"SunPosition", |
| 185 | + "StartByte":"1", |
| 186 | + "Bytes":"112", |
| 187 | + "Records":"2", |
| 188 | + "ByteOrder":"Lsb", |
| 189 | + "CacheType":"Linear", |
| 190 | + "SpkTableStartTime":"297088762.24158", |
| 191 | + "SpkTableEndTime":"297088808.37074", |
| 192 | + "SpkTableOriginalSize":"2.0", |
| 193 | + "Description":"Created by spiceinit", |
| 194 | + "Kernels":[ |
| 195 | + "$base/kernels/spk/de430.bsp", |
| 196 | + "$base/kernels/spk/mar097.bsp" |
| 197 | + ], |
| 198 | + "Data":"4cffffffd401ffffffe62effffffd3ffffffa8ffffffc11d03ffffffffffffff8525495dffffffc157fffffffb03ffffff89ffffff800c404114ffffffd1ffffffa7ffffffb6ffffffecffffffe7ffffffcaffffffbfffffffc6fffffffa48ffffffd7ffffffe1ffffffe637ffffffc0246fffffff93ffffffae39ffffffea25ffffffc074ffffffd83dfffffffa36ffffffb5ffffffb141fffffff6ffffffc264fffffff92effffffd3ffffffa8ffffffc1503cffffffb32a394a5dffffffc1fffffff5fffffff746ffffffceffffff830b4041ffffffc6066d1b4fffffffe3ffffffcaffffffbf6c4f1dffffff80ffffffe1ffffffe637ffffffc03fffffffb246ffffffde39ffffffea25ffffffc0ffffff8fffffffe85e2837ffffffb5ffffffb141" |
| 199 | + }' |
| 200 | +} |
| 201 | +``` |
| 202 | +### Working with GDAL Products Outside of ISIS |
| 203 | +GDAL also provides a suite of applications that work on either GeoTIFFs or cubes. The programs that can be run on ISIS produced images can be found [here](https://gdal.org/en/stable/programs/index.html) |
| 204 | + |
| 205 | +While there are plans to update the GeoTiff Driver in GDAL to support and maintain this ISIS JSON metadata, if an external application is used, the ISIS metadata within the GeoTIFF will likely not be recognized or lost during conversion. For example, during a conversion of an ISIS-created GeoTIFF using `gdal_translate`, the output file will not contain the JSON metadata. |
| 206 | + |
| 207 | +Some notable applications are `gdalinfo`, `gdal_translate`, and `gdaladdo`. |
| 208 | + |
| 209 | +- [`gdalinfo`](https://gdal.org/en/stable/programs/gdalinfo.html#gdalinfo) provides information on supported GDAL formats |
| 210 | +- [`gdal_translate`](https://gdal.org/en/stable/programs/gdal_translate.html#gdal-translate) can convert one supported GDAL image format into another. |
| 211 | +- [`gdaladdo`](https://gdal.org/en/stable/programs/gdaladdo.html#gdaladdo) builds image overviews. This is largely useful in a cloud computing environment when viewing images, as overviews will transfer less data when requesting DNs over the net. |
0 commit comments