README_cereal

Motivation

Cereal is a header-only C++11 serialization library that is fast, light-weight, and easy to extend.

The Cereal serialization framework in SHOGUN uses the new tag parameter framework, which allows the readable archive of SGObject data.

For SHOGUN developers

• Cereal serialization library is required for SHOGUN compilation. If no Cereal is found, SHOGUN will automatically download the library to third_party/.

• SHOGUN now supports the serialization of data into 3 formats: binary, XML, and JSON archives. The 3 pairs of save/load methods can be called by:

save_binary(filename);
save_json(filename);
save_xml(filename);
load_binary(filename);
load_json(filename);
load_xml(filename);

• All parameters saved in tag parameter list for one SGObject can be saved and load by:

SGObject obj_save;
obj_save.save_json(filename);

SGObject obj_load;
obj_load.load_json(filename);

• Customized archives can be added as shown here

For serialization framework developers

The serialization framework has two components:

• serialization interfaces implemented in SGObejct, and

• serialization (load/save) methods implemented in SGObject and non-SGObject based data structrues.

Serialization interface

• The save_binary() method in SGObject.h generates an cereal::BinaryOutputArchive object and saves SGObject to binary file by calling cereal_save() method in SGObject. load_binary() method generates an cereal::BinaryInputArchive object and loads the parameters from binary file back to SGObject by calling cereal_load() method in SGObject. The ideas are the same for JSON and XML archives.

Serialization methods in SGObject

• cereal_save() method iterates through the parameter list of SGObject registered as self::map, archives the name value pair, with name as basetag.name() and value by calling any.cereal_save().

• cereal_load() method iterates through the parameter list and reset the parameter by calling any.cereal_load()

Serialization methods in Any

• Namespace serial and object serial::DataType m_datatype in Any.h save and convert the data type of the value of parameters in Any constructors into Enum.

    enum EnumContainerType
    {
        CT_UNDEFINED,
        CT_PRIMITIVE,
        CT_SGVECTOR,
        CT_SGMATRIX
    };

    enum EnumPrimitiveType
    {
        PT_UNDEFINED,
        PT_BOOL_TYPE,
        PT_CHAR_TYPE,
        PT_INT_8,
        PT_UINT_8,
        PT_INT1_6,
        PT_UINT_16,
        PT_INT_32,
        PT_UINT_32,
        PT_INT_64,
        PT_UINT_64,
        PT_FLOAT_32,
        PT_FLOAT_64,
        PT_FLOAT_MAX,
        PT_COMPLEX_128,
    };

• Cereal_save() together with cereal_save_helper() methods cast the object storage to its input type and archives the value.

• Cereal_load() together with cereal_load_helper() methods read the saved value back to storage and reset the policy based on the data type.

Serialization methods in SGVector, SGMatirx and SGReferencedData

Both SGVector and SGMatirx are derived from SGReferencedData class.

• SGReferencedData archives whether ref_counting is on by saving true/false, and the ref_counting value if m_refcount != NULL, i.e. ref_counting is on.

• SGVector and SGMatrix archives SG_REF value by calling base class load/save methods: cereal::base_class<SGReferencedData>(this) (Introduction). For SGVector, length and vector values are archived, while for SGMatrix, row number, column number, and matrix values in T* matrix are archived. Data of complex128_t type is casted to float64_t type before archiving.