HDF5 Raw Data File Support

[huyuhuster]

I would like to use Raw to process the data collected from a synchrotron radiation source, and my data is stored in HDF5 format. I noticed that the Raw source code includes methods for importing HDF5 data, but I couldn't find relevant documentation in the user guide.

I would like to confirm whether Raw supports processing experimental raw data in HDF5 format. If so, are there any specific requirements for the data structure?

thanks

[jbhopkins]

HDF5 is just a generic container, so you would need to have a specific data structure in mind for RAW to load. While RAW does support certain types of HDF5 data structures, including those it produces itself for series datasets and HDF5 packaged data from the LiX beamline at NSLS II, it's likely that anything else would require writing an additional specification. The easiest thing to try first is just to open the HDF5 file in RAW. If RAW can figure it out, then it'll load, if not it'll provide some kind of error.

As far as adding a new HDF5 file definition, RAW supports text based files to define HDF5 data structures. If you're looking at the code you can check out the loadHdf5File function in SASFileIO.py and the lix_hdf5.txt file in the definitions/hdf5 folder. While this function is intended to allow pretty generic definitions, at the moment it's only been tested with the LiX definition, so there may well be bugs or limitations. But that would be where to start in writing a new loader for HDF5 data. I can potentially help with that.

I should note that if you have a definitions file that you make, it can be added to an existing installation of RAW using the Options -> Add HDF5 Definition. However, for generating the definition working from source code is probably easiest.