v1.0.6

Added

Features

• Data now directly affords a find_special_constants() method

Dataset Support

• most IHW datasets
• additional Voyager 1 and 2 datasets
• GRSFE and WFF/ATM datasets
• LOIRP dataset
• additional MSL, Apollo, and Mariner datasets
• additional Earth-based lunar data
• Viking Lander datasets
• Mars Odyssey DAN and SAM data
• MEX ASPERA, HRSC, SRC, and PFS data
• most Phoenix TEGA, ELEC, and WCL products
• see supported_datasets.md for details

Changed

• Assorted type-hinting and other in-code documentation improved.
• Behavior for character stripping in string columns of ASCII tables was
  inconsistent. To increase usability, string columns of ASCII tables now
  discard preceding and trailing spaces, commas, newlines, and double-quotes
  (regardless of strict label specification).
• Generally improved performance on table reading

Fixed

• Incorrect column bounds due to unusual prefix/suffix/infix specifications
  in some ASCII tables
• Incorrect column bounds due to container padding in some tables
• FITS HDUs with non-printable characters in header cards will now load,
  discarding any such cards from the header (unless the cards are required,
  e.g. NAXIS1). Resolves issue #52

v1.0.5

[1.0.5] - 2023-12-07

Added

Features

• doc strings for API on readthedocs
• Error tracking features that were accidentally deleted from the last version were re-added

Dataset Support

• most Voyager 1 and 2 datasets
• additional MGS datasets (MAG/ER, RSS EDS, MOC)
• Mariner 9 datasets
• Pioneer 10 and 11 GTT RDRs
• Sakigake, Suisei, and SOHO datasets
• ICE and IUE datasets
• see supported_datasets.md for details.

Fixed

• Tables with containers with REPETITIONS in them were reading the same data each repetition, this has been fixed
• Exponential notation in pvl quantity objects are now properly handled (Fixes issue #53)

Removed

• MRO MCS DDR data is not supported due to formatting issues

Full Changelog: v1.0.4...v1.0.5

v1.0.4

[1.0.4] - 2023-10-23

Added

Features

• support for GIF files referenced as PDS3 objects
• improved verbosity of some file-not-found error messages

Dataset Support

• MER 1 and 2
• Ulysses
• see supported_datasets.md for details.

Changed

• updated install specifications
• added hotpatch to address pds4-tools Python 3.12 incompatibility,
  to be removed following pds4-tools 1.4 release

Fixed

• correctly interpret PVL non-decimal integer representations as
  int(non_decimal_integer, base=int(radix)). e.g., pdr will now interpret
  the PVL statement:

   "SAMPLE_BIT_MASK          = 2#1111111111111111#"
  

  as a metadata item with key "SAMPLE_BIT_MASK" and value 65535.

  Please note that the bit masks are not applied programmatically because we don't believe
  their meanings are consistent across the corpus of planetary data. However, users are
  encouraged to explore their use within their own work and enjoy the glories of the Python & operator.

  An example for how to apply a bit mask using the & bitwise operator:

      data = pdr.read('/path/to/file.img')
      masked = data.IMAGE & data.metaget('SAMPLE_BIT_MASK')
  

  Alternatively, if you're using this value for something for which you'd prefer to have the original PVL
  value, use the bin, oct, or hex packages for base 2, 8, or 16 respectively to convert the value
  back from an integer.

  (Resolves issue #51)

• correctly interpret PVL Sequences and Sets of Unquoted Strings as
  tuple[str] or set[str] respectively

• fix some cases in which PVL End Statements were incorrectly interpreted
  as parameters with empty-string values

v1.0.3

Added

Features

• FITS files can now be opened 'directly' (without a PDS label).
  See README.md for more detailed usage.
• 1d, non-structured ARRAY objects are now compatible with dump_browse
• support for ARRAY objects with nested ARRAY or COLLECTION objects
• pytest-compatible unit test suite
  intended to complement our comprehensive regression test framework

Dataset Support

• most New Horizons datasets
• Venera 15 and 16 derived data
• Giotto PIA and VEGA 1/2 Puma_mode data
• Venus Climate Orbiter data
• Saturn Ring Plane Crossings (RPX) 1995-1996 data
• most Phoenix datasets
• Deep Impact and EPOXI data
• Mars Pathfinder data
• several MRO datasets including HiRISE EDRs, MCS, MARCI, and CTX
• see supported_datasets.md for details.

Changed

• Debug-mode error tracking is substantially more detailed
• Debug-mode error logs are now written as JSON
• Assorted backend refactoring, cleanup, error checks, and in-code documentation

Fixed

• Fixed incorrect PDS3 object -> FITS HDU mapping for several data types
• scaling/offset defined in FITS tables (BSCALE/BZERO) is now applied correctly to output DataFrames
• 8-byte integers found in some nonstandard binary files are now handled correctly
• more consistent handling of long / malformatted PVL comments
• improved handling of 'stub' primary FITS HDUs

v1.0.2

Added

• Support for Rosetta ALICE EDR, RDR, and REFDR data
• Support for several Galileo datasets
• Support for sample interleaved (BIP) images, including:
  • Support for DAWN VIR EDR and RDR cube products
  • Support for Rosetta VIRTIS EDRs
• Support for Deep Space 1, NEAR, Stardust, and Stardust-NExT datasets
• Support for VAX_REAL image products, including Pioneer Venus radar and IMIDR images
• Support for IBM_REAL and EBCIDC data types, including Pioneer Venus SEDR
• Support for LRO CRaTER EDR secondary science and housekeeping tables
• Support for several Mars Odyssey datasets
• Support for several Vega datasets
• Support for FITS tables
• Documentation on behavior of FITS files (in README)

Changed

• Tables/Series objects will now apply label provided offset and scaling factors.
  This has not yet been implemented for ARRAY objects or BIT COLUMNS
• The license has been updated to reflect the additional license from the vendoring of
  vax.py

Fixed

• Tables with nested containers now read correctly; closes issue 50)
• ^STRUCTURE pointers inside a COLUMN/FIELD will now load in the relevant format file

Removed

• No longer get a UserWarning when a data type has spaces in bit column data type

v1.0.1

Fixed

• re-enabled the ability to import pdr.Data by adding it back to __init__.py

v1.0.0

This release represents a major refactoring effort to reduce technical debt and decrease workflow complexity.
The fundamental way end users interact with the main pdr.read() interface has not changed.

Added

• error/workflow tracking for all loaders.
  • Errors can be accessed using data.loaders["OBJECT_NAME"].errors. You must pass debug=True during the initial read call (e.g. pdr.read('/path/to/file', debug=True) to access the errors.
  • Tracking files will now generate in a .tracker_logs folder in the pdr repo. They show which functions
    have been hit by a particular call of pdr and if those functions were successful.
• handling for a wider array of ISIS-style "qube" data and metadata,
  including side/back/bottom/topplanes (as long as they are along only one
  image axis)
• support for (most) THEMIS qube products
• TODO for cassini XDR image scaling functionality
• support for additional LRO datasets: CRaTER, LAMP, LEND, LOLA, Mini-RF, and Radio Science
• support for several Venusian datasets including: Magellan GVDR, Pioneer Venus,
  "Pre-Magellan" products at the GEO node, and Earth-based radar observations.
• support for several Lunar datasets including: GRAIL, Lunar Prospector, MSX,
  Kaguya/Selene, and Earth-based radar and spectroscopy data.

Changed

• reworked fundamental data loading workflow. Data class no longer contains
  all the loader functions, they've been refactored in the loaders module.
• formats.core.py now only contains special case checking functions and is renamed to formats.checkers.py.
  Other functions that were previously in it (like pointer_to_loader and generic_image_properties) have
  moved to the loaders module
• changes to various special cases based on the data loading workflow refactor
• reworked image-loading flow for better handling of various band storage
  types and pre/suffixes
• BIL images now retain original byteorder
• reworked cassini xdr special case for compatibility

Fixed

• discovered some rosetta VIRTIS and cassini UVIS product types were not
  actually reading correctly, marked them out of support

Removed

• m3 special case module (deprecated by new image-loading flow)
• messenger special case module (deprecated by new data loading workflow)
• rasterio loading options for image data
• check_special_case has been removed and the special cases have been moved to functions
  that more specifically targeted to the issues of the particular dataset rather
  than overriding the entire workflow.

v0.7.5

Added

• support for a variety of Magellan data (see supported_datasets.md
  for specifics)

Changed

• sample types are now more permissive and allow spaces
• line endings are less strict and allow between 0-2 carriage returns

Fixed

• HiRISE EDRs are now only listed as notionally supported (rather than known
  unsupported and notionally supported)

Removed

• special case for mgs-rss data with IEEE REAL sample types (now supported in core
  functionality)

v0.7.4

Added

• support for a variety of LRO Diviner, Cassini, and Huygens data products
  (see supported_datasets.md
  for specifics)
• This change log file
• Support for bit columns with ITEMS
• Browse images can now be output in grayscale
• handling for COMPRESSED_FILE objects

Changed

• pdr will now accept non-UTF-8 characters in detached PVL label / format files
• refactored special case checking for readability/maintainability
• assorted linting and in-code documentation edits
• labels will by default read in up to 1 MB of the file (previously 500 bytes). If
  you're trying to get faster performance for shorter attached labels en masse, pass
  the new --pvl_limit parameter. If you're not running large numbers of attached
  label files in sequence this is largely irrelevant.

Fixed

• label comments that are left unclosed by the data providers no longer prevent
  reading in the data
• group offset computations by offset not start byte (allows opening of Juno Jane V04
  products, closes issue 43)
• bit columns are now split based on both start_bit and number of bits rather than
  simply start_bit (this fixed a number of previously incorrectly read files,
  and mirrors handling for other column types)
• dump_browse now properly accepts the float_dtype argument; reducing
  bit depth can significantly reduce memory use when browsifying large arrays
• malformed labels with extra end-block statements no longer crash BlockParser
• JUNO JIRAM RDR special case now covers tables as well as images
• HEADER objects in compressed files now load properly
• DATA_SET_MAP_PROJECTION_CATALOGS no longer go to read_header (which does
  not work on them)

Removed

• special-case handling for COMPRESSED_FILEs
• support for MSL CCAM LIBS EDRs

v0.7.3

1. Browse product improvements:

• Tables with a single row of data will now output as a column for their .csv browse products to increase readability
• Browse images can be specified to output in formats other than .jpg

2. Newly supported:

• ARRAY pointers are now supported
• FITS headers are now supported. If a pointer with HEADER in it points to a FITS file it will return that header. If there is a pointer without HEADER in it that points to the FITS file then after loading that key, an additional key will appear in the format: {key}_HEADER that will contain the FITS header.
• .jp2 files are now supported
• Many new data types from missions such as MGS, Clementine, Rosetta, and more (see supported_datasets.md for specifics)

3. Supported_datasets.md is now alphabetically organized by mission name to more easily search.
4. Various optimizations and bug fixes (really a whole lot of them)!