v0.18.0

0.18.0 (2022-12-06)

Breaking Changes

• BREAKING CHANGES: Refactoring spectral indices management (#47)
  • Using spyndex library, allowing to use all spectral indices
    listed here
  • SAR products may now compute indices if possible (see this list)
  • Old EOReader indices are still available for legacy purposes, with some changes:
    • For Sentinel-2 data, the band NIR and NARROW_NIR may be interchanged for some index (
      see this discussion)
    • OSAVI formula has changed to stick with the original paper definition (see issue)
    • NDRE2/3 formula are fixed, now using VRE_2/3 and NDRE1 corresponds to NDREI and uses VRE_1
    • CI1 is renamed CI32 and CI2 is renamed CI21 for readability purposes
    • NDWI21 can be written NDWI2100 for homogeneity purposes
    • RDI (or DSI) uses now SWIR_1 instead of SWIR_2 (see this issue)
    • PANI equivalent is now BITM and is normalised ! (divided by 3)
    • SBI is normalized (divided by 2) to fit with BIXS definition
    • ⚠ You may need to install the last spyndex directly from GitHub latest version to have all available indices
• BREAKING CHANGES: Using pyresample to geocode Sentinel-3 data (#55)
  • Cleaner: better conversion from swath to grid
  • Faster: Up to 4 times faster
  • Allows code refactoring between OLCI and SLSTR
• BREAKING CHANGES: For SAR product types that are not available in the Data Access Portfolio, default resolution is now the pixel spacing instead of the rg x az resolution
  • Changes mainly Sentinel-1 default resolutions (except from IW mode)

Enhancements

• ENH: Adding the support of Harmonized Landsat-Sentinel constellation (#49)
• ENH: Adding the support of GEOSAT-2 constellation (#59)

Bug Fixes

• FIX: Fixing CustomProduct initialization when fields are set to None (instead of not declaring them)
• FIX: SNAP cannot handle float predictors other than 1! Set it to 1 when saving ortho SAR images to disk, in order for SNAP to be able to despeckle
  them (https://forum.step.esa.int/t/exception-found-when-reading-compressed-tif/654/7). (#62)
• FIX: Fixing mix in Sentinel-2 mapping for B8 (NIR, 10m resolution, large spectral bandwidth) and B8A (NARROW_NIR, 20m resolution, narrow spectral bandwidth)

Other

• DOC: Add FAQ entry concerning SAR constellations extent KML files failing to be read (TLDR: needs ogr2ogr in your
  PATH)
• DOC: Add Technical Note published in Remote Sensing MDPI in Readme
• DOC: Update optical band mapping graphs (fix regression to 0.15.0 supported constellation)
• DOC: Add information about DEM management in SAR notebook (#61)
• DOC: Updating indices paragraphs
• CI: Using actions/checkout@v3
• CI: Updating versions of pre-commit hooks
• LIBS: Updating requirements.txt and setup.py to add pyresample and zarr

v0.17.0

0.17.0 (2022-10-12)

Enhancements

• ENH: Adding the support of RapidEye constellation
• ENH: Handling Planet data with multiple subdatasets (#45)
• ENH: Adding the support of Landsat Level-2 products (#49)
• ENH: Adding the support of Pleides Neo SEN and PRJ products (needs GDAL 3.5+ or rasterio 1.3.0+)
• ENH: Adding the function bands.is_thermal_band
• ENH: Adding the ability for optical custom stacks to load indices
• ENH: Adding BAIM (MODIS Burned Area Index) spectral index
• ENH: Better management of raw units of the bands of optical products
• ENH: Copying files from tmp_process when changing product's output

Bug Fixes

• FIX: Stacks saved as integers on disk keep their original dtype (float32) in Python
• FIX: Stacks with bands loaded "as is" are correctly saved as integers on disk (#52)
• FIX: Using stack CRS (if projected) for DIMAP products instead of recomputing from lat/lon, solving potential discrepencies between stack and product CRS
• FIX: Workaround for JP2 bug when updating an existing raster (maybe related to this bug)
• FIX: Better management of SkySat datetime conversion from JSON to XML (deterministic way)
• FIX: Fixing computation of invalid pixels for Sentinel-2 and DIMAP products (do not remove straylight mask)
• FIX: Fixing reprojection resolution of VHR data
• FIX: Computing Brightness Temperature of Landsat TIR bands instead of leaving them as is
• FIX: Better management of Landsat Instrument values
• FIX: Better radiometry attribute (adding brightness temperature and reflectance and brightness temperature values)
• FIX: Changing Brilliance Temperature to the correct Brightness Temperature
• FIX: Fixing pandas FutureWarning The frame.append method is deprecated and will be removed from pandas in a future version.
• FIX: Fixing DeprecationWarning invalid escape sequence \.
• FIX: Manage correctly Planet dubious pixels (especially for 8 bands products)

Optimizations

• OPTIM: Reduce memory usage when updating all the bands attributes
• OPTIM: Reduce memory usage when stacking as integers

Other

• DOC: Add the need of using SNAP 8.0 up-to-date or SNAP 9.0 (#42)
• DOC: Add the STAC session in API documentation
• DOC: Add warnings for shifts when orthorectifying DIMAP SEN products (using RPCs) (#53)
• DOC: Add limitations to custom stacks
• DEPS: Dropping support of Python 3.7 (#18)
• DEPS: Update minimum version of libs (geopadans 0.11.0+, rasterio 1.3.0+...)

v0.16.1

0.16.1 (2022-08-03)

Bug Fixes

• FIX: Add the missing conversion to reflectance for Sentinel-3 OLCI
• FIX: Better condition for the conversion to reflectance for Sentinel-2 THEIA
• FIX: Add logs for SkySat data that cannot been converted to reflectance and fix the radiometry field of its band xarrays
• FIX: Add the correct nodata (when overridden by the user) to stacks saved as uint16

Optimizations

• OPTIM: Reduce memory usage during stacking

Other

• CI: Test reflectance values

v0.16.0

0.16.0 (2022-08-01)

Enhancements

• ENH: Adding the support of SuperView-1 constellation (#21)
• ENH: Adding the support of SPOT-4/5 constellations (#39)
• ENH: Allow the possibility to pass a constellation (or a constellation list) to Reader().open() to speed up the opening of a product
• ENH: Add a quicklook search for Sentinel-3 products

Bug Fixes

• FIX: Fix quicklook media type with JP2 files
• FIX: Fix Sentinel-3 SLSTR F1 bands based on F grid
• FIX: Correct the UTM projection for Sentinel-3 data
• FIX: Fix handling of zipped Sentinel-2 L2Ap
• FIX: Fix zipped Sentinel-2 with other XML files in GRANULE subdirectories

Other

• Renaming master branch to main

v0.15.1

0.15.1 (2022-06-02)

Optimizations

• OPTIM: Try to create Vision-1 footprint from the preview file instead of from the stack.
• OPTIM: Create footprints for stacked products (i.e. Maxar, SkySat, Custom...) without mask by opening only the first band of the stack
• OPTIM: Create footprints for Maxar Products with a resolution 10 times lower.
• OPTIM: Footprints have now maximum 50 vertices in order to avoid pixelized footprints

Bug Fixes

• FIX: Fixing condensed name to avoid duplicates:
  • adding the job_id for VHR products
  • adding the polarization channels for SAR products
• FIX: Remove import of pystac in stac_utils
• FIX: Fix bug for Vision-1 data looking for non-existing RPC files in case of ORTP product type
• FIX: Fix quicklook regex for Vision-1 data
• FIX: Fix regex for raw bands for extracted Sentinel-3 OLCI products
• FIX: Fix PlanetScope identifying regex to handle products with a satellite_id containing a letter
• FIX: Force metadata regex for Maxar products to look for a file with pattern {name}.XML to avoid other misplaced XML to be found in place of the true XML.
• FIX: Fix regression for Landsat-7 footprint
• FIX: Manage the case with cloud_cover = -999.0 for Maxar products (returns None)

Other

• CI: Add new optical products to be tested for end-to-end tests

v0.15.0

0.15.0 (2022-05-30)

Breaking Changes

• BREAKING CHANGES: Optical becomes Spectral when more appropriate
• BREAKING CHANGES: Platform and Sensor become Constellation when more appropriate, to fit STAC vocabulary (#29):
  • Platform enum becomes Constellation
  • prod.platform becomes prod.constellation
  • prod.sat_id becomes prod.constellation_id
• BREAKING CHANGES: File alias is removed, replaced by *_bands files and proper imports in bands.__init__
• BREAKING CHANGES: Product attribute band_names becomes bands in order to be STAC compliant (#29)
• BREAKING CHANGES: Better use of NIR and NARROW_NIR in the indices file (according to the gsd of Sentinel-2 bands composing the indices)
• BREAKING CHANGES: Correcting Landsat product types to better manage processing levels and instrument. Landsat-8/9 condensed name may change!

Enhancements

• ENH: Adding the support of SkySat (Collect) products (#20)
• ENH: Bands in mapping are now objects, instead of just IDs (#29). This allows us to:
  • Add band metadata (such as center wavelength, bandwidth...)
  • Map spectral bands between STAC spec and EOReader format (#29)
  • Add a better __repr__ functions
• ENH: Handling 8 bands PlanetScope data (#20)
• ENH: Adding the GREEN1 mapped band, corresponding to PlanetScope GREEN I and Sentinel-3 OLCI Oa05 band
• ENH: Handle some slightly broken Sentinel-2 products:
  • when the metadata files are corrupted or when the detfoo vectors are empty (#34)
  • with missing MSK prefix for QI_DATA files (i.e DETFOO instead of MSK_DETFOO)
• ENH: Handle exception for corrupted bands (in Sentinel-2 and utils.read) (#34)
• ENH: Add a STAC object that can be used to retrieve STAC Items from every Product (prod.stac.create_item()) (#29)
• ENH: Add a get_mean_viewing_angles for Optical Products to fill STAC View Extension (#29)
• ENH: Extending get_raw_band_paths to every product (#31)
• ENH: Adding a is_ortho attribute corresponding to when the product is already orthorectified/geocoded, in order to avoid computing heavy processes without wanting it (i.e. footprint...)
• ENH: Adding the instrument name of every constellation, under prod.instrument
• ENH: Handling COSMO product with only the h5 file in it (if missing XML metadata file) (#36)

Optimizations

• OPTIM: Retrieve name from filename if possible
• OPTIM: Retrieve extent from metadata when possible (for VHR data)
• OPTIM: Refactoring Landsat-XX products into LandsatProduct, this should bee invisible for user.

Bug Fixes

• FIX: Fixing the band mapping of WorldView-2/3 Multi (8 bands)
• FIX: Retrieval (if possible) of Sentinel-1 unique ID (was missing from the product name, as it is not in the product preview)
• FIX: Fixing PAZ/TDX MTD regex
• FIX: Optical products: Only set cloud cover and radiometry attributes if spectral bands are asked

Other

• INTERNAL: File spot_6 and spot_7 are removed, replaced by a unique spot file. This shouldn't affect the user.
• INTERNAL: Some refactoring in VHR files
• WARNINGS: Filter warnings from __init__
• CI: Do not process two times the zipped Sentinel-1 in end-to-end tests and manage when the runner kills SNAP
• DOC: Adding a STAC notebook
• DOC: Various updates

v0.14.0

Breaking Changes

• BREAKING CHANGES: footprint, extent, wgs84_extent and crs properties are converted back to methods in order to prevent side effects of expensive computation when displaying the object when debugging (rollback before version 0.8.0)
• BREAKING CHANGES: get_all_index becomes get_all_indices
• BREAKING CHANGES: acquisition_datetime becomes datetime and default_resolutionbecomes resolution for CustomProduct in accepted keywords, and the metadata fields have been renamed according to the CustomFields enum

Enhancements

• ENH: Adding spectral indices:
  • Shadow Index (SI)
  • Global Vegetation Moisture Index (GVMI)
  • Soil Brightness Index (SBI), Soil Cuirass Index (SCI)
  • Panchromatic mocking Index (PANI)
  • Green-to-Red ratio Index (GRI)
  • Soil Adjusted Vegetation Index (SAVI)
  • Optimized Soil Adjusted Vegetation Index (OSAVI)
  • Visible Atmospherically Resistant Index (Green) (VARI)
  • Enhanced Vegetation Index (EVI)
  • Chlorophyll Index RedEdge VRE_3/VRE_2 (CI1)
  • Chlorophyll Index RedEdge VRE_2/VRE_1 (CI2)
  • Normalized Difference Moisture Index (with SWIR_21) (NDMI21)
• ENH: Making SAR attribute snap_filename public
• ENH: Handling ICEYE pure SLC products
• ENH: Allowing the user to choose if they want the GRD or SLC image for ICEYE products
• ENH: Add the possibility to directly load the cloud cover for optical data (and add it in the band attributes) (#28)
• ENH: Add the possibility to retrieve the quicklook path (if existing) and add the plot function allowing the user to plot the quicklook (if existing) (#28)
• ENH: Add the possibility to retrieve the orbit direction (and add it in the band attributes) (#28)

Bug Fixes

• FIX: Fixing the inversion between 8 and 8A bands for Sentinel-2 and Sentinel-2 Theia products
• FIX: Loading every optical band in reflectance (fixed for Sentinel-2 THEIA, Maxar, Planet and Vision-1 data) (#30)
• FIX: Fixing ReferenceError: weakly-referenced object no longer exists when deleting an object
• FIX: Do not set sea values to nodata when orthorectifying SAR data with SNAP
• FIX: Handle Sentinel-2 data with PB < 02.07 as L2Ap products
• FIX: Fixing nodata and offset for Sentinel-2 data with PB > 04.00
• FIX: Handle new ICEYE metadata name's nomenclature
• FIX: Fixing harmless regex error when searching for B1 path for Landsat products
• FIX: Fixing platform for Sentinel-2 Theia

Other

• DOC: Creating a real base notebook and renaming the old one to optical
• DOC: Better type hints (replacing XDS_TYPE by xr.DataArray)
• CI: Using sertit.ci.reduce_verbosity instead of recreating the function

v0.13.1

0.13.1 (2022-03-08)

• FIX: Handling Sentinel-2 L2Ap data
• FIX: Do not use --no-binary fiona,rasterio directly in requirements.txt (breaks on Windows)
• FIX: Fixing stacking with string bands
• FIX: Better __repr__ function
• CI: Adding a tag for choosing the runners
• DOC: Fixing cartopy/GEOS conflicts making the documentation build to fail

v0.13.0

0.13.0 (2022-03-02)

• ENH: Adding the support of Landsat-9 sensor
• ENH: Support Sentinel-2 with missing datatake metadata file(sometimes happens with data downloaded from AWS buckets and converted to .SAFE)
• FIX: Using default SAR resolution from official Copernicus Data Access Portfolio (2014-2022) (Sentinel-2 default resolution goes to 10.0 m !)
• FIX: Use --no-binary fiona,rasterio directly in requirements.txt
• FIX: Removing useless outputComplex line in GPT graphs that is breaking SNAP on Linux
• FIX: Removing the workarounds caused by some bugs of cloudpathlib and enabling retrieval of nested SAR products (TSX, TDX, PAZ, RCM) from S3 compatible storage.
• FIX: Do not process nodata for a band already existing
• FIX: Fixing an error when reading TIR bands with Landsat-7
• FIX: Fixing an error when additive/multiplicative coefficients are set to NULL for Landsat data
• CI: Do not try to process SAR end to end if GPT cannot be found
• CI: Publishing wheel from Github instead of Gitlab
• REPO: Setting GitHub as the main repository and using new Gitlab runners

v0.12.0

0.12.0 (2022-02-09)

• ENH: Adding the support of Pleiades-Neo, Vision-1 and SAOCOM sensors
• ENH: Adding a keyword to allow passing a specific DEM path in load/stack (for VHR orthorectification and DEM bands)
• ENH: Adding the name of the DEM in DEM band (i.e. allow to compute the HILLSHADE with a DEM and the SLOPE with a DTM)
• FIX: Sentinel-2 Processing Baseline 04.00: NARROW_NIR bands are now loaded correctly
• FIX: Maxar products (with Multi band ID) are now correctly handled
• FIX: Using COPDEM-30 (GLO-30) by default for SNAP as it appears that the retrieval has been fixed.
• FIX: Fixing the default name for cleaned bands for Sentinel-3 SLSTR data (was set on CLEAN instead of NODATA)
• FIX: Fixing default band for Custom stacks
• FIX: Fixing get_existing_band_paths behavior for Custom stacks
• FIX: Remove other never covered lines of code (archived RCM products, complex ICEYE products, others...)
• FIX: Re-enabling loading str bands (regression)
• FIX: Proper check for empty fields when parsing metadata
• FIX: VHR _get_dem_path raises ValueError instead of TypeError
• FIX: Pre-process SAR bands before despeckling if not existing (was OK in most of the cases, but broke in some cases, especially with CI folder activated and S3 compatible storage)
• FIX: Remove warning invalid escape sequence \., \w, \D and \s
• FIX: Do not set long_name for RAW_CLOUDS arrays
• FIX: Providing a URL DEM on Windows throws a OSError instead of a bare Exception
• OPTIM: Do not pre-process existing Sentinel-3 geocoded bands
• OPTIM: Do not look for valid metadata further than a given nested level in product's directory (for extracted products)
• CI: Using another (faster) runner
• CI: Add on disk and end-to-end tests
• CI: Do not write tmp files when running on disk tests
• CI: Coverage:
  • Get coverage as HTML
  • Remove useless lines from coverage
  • Combine coverage of S3 and on disk tests
• DOC: Adding a DEM notebook