v00-99

v00-99 (v01-00 pre-release)

This is a pre-release for the first stable release of podio. We consider the file format as well as stable and also consider the base functionality to be in place. All additional functionality should be implementable on top of the existing file format as well as existing features. We will make a few smaller fixes and remove still existing deprecated functionality before we release v01-00 in the next few weeks.

Changes wrt v00-17-04

• 2024-02-06 tmadlener (PR#554)

  • Bring back vector<Data> and vector<Component> into the dictionaries to allow for better interoperability with RNTuple.
    • See #464 for some related discussion.

• 2024-02-06 tmadlener (PR#552)

  • Introduce an operator< to the interface types to make it possible to use them in STL containers that require that (e.g. std::map and std::set).

• 2024-02-06 tmadlener (PR#549)

  • Rename {ROOT,SIO}Frame{Reader,Writer} to {ROOT,SIO}{Reader,Writer} since these names are now no longer taken by the deprecated EventStore based ones and Frame based I/O is the default now.
    • Keep the old names around as deprecated aliases to not break everything immediately. But plan to remove the aliases for v1.0.

• 2024-02-02 Mateusz Jakub Fila (PR#550)

  • Fixed typos in tests and interface template

• 2024-01-30 tmadlener (PR#548)

  • Remove an unnecessary usage of the TClass machinery inside the RNTupleReader as all the necessary information is also available from metadata that we carry around in any case.

• 2024-01-30 Graeme A Stewart (PR#544)
  Add CMake targets for running black and flake8 on Python source files

• 2024-01-30 tmadlener (PR#528)

  • Use black to format all python source files
  • Add a pre-commit hook for running black in CI

• 2024-01-25 jmcarcell (PR#541)

  • Add tool to transform between TTrees and RNTuples

• 2024-01-24 jmcarcell (PR#543)

  • Change ROOTNTuple{Reader,Writer} to RNTuple{Reader,Writer}

• 2024-01-22 tmadlener (PR#516)

  • Add a new category of types that can be generated by podio: interfaces. These can be used to provide a type that can be initialized from several other datatypes and offers some common functionality. These interface types can be used in OneToOneRelations and in OneToManyRelations.
    • interfaces need to provide a list of types which they interface. Other types cannot be used with them.

• 2024-01-18 Mateusz Jakub Fila (PR#542)

  • Fixed typos in documentation and comments, updated .gitignore

• 2024-01-18 jmcarcell (PR#540)

  • Add a check for the C++ standard using ROOT_CXX_STANDARD that was introduced yesterday in ROOT (root-project/root@d487a42)

• 2024-01-16 jmcarcell (PR#539)

  • Fix crash when a writer is in the global namespace by adding a class that will manage all the writers and finish them before exiting

• 2024-01-16 jmcarcell (PR#538)

  • Use getAvailableCollections() instead of the deprecated .collections

• 2024-01-12 jmcarcell (PR#536)

  • Add a getAvailableCollections method in python that does the same thing as in C++

• 2023-12-19 tmadlener (PR#535)

  • Implement the suggestions from coverity to move in places where it is easily possible.
  • Fix a small resource leak in SIO.
  • Fix a small copy-paste error in test output (only triggered in case test fails)
  • Restore ostream state after altering it for formatting.

• 2023-12-19 tmadlener (PR#534)

  • Update the coverity workflow to use EL9

• 2023-12-18 Andre Sailer (PR#533)

  • rootUtils: include sstream, fixes build on macOS 12 / 13

• 2023-12-18 tmadlener (PR#532)

  • Remove some benchmark utilities that became unused with the removal of the TimedReader and TimedWriter classes in #485

• 2023-12-15 tmadlener (PR#531)

  • Remove the last few deprecated accessors from GenericParameters.

v00-17-04

v00-17-04

• 2023-12-14 tmadlener (PR#527)

  • Split the ClassGenerator into a base class (mixin) and two specific c++ and julia code generators that only deal with their language specific needs.
    • Instantiate and configure the correct reader in the podio_class_generator.py main script depending on the desired language.
  • Slightly cleanup the MemberVariable to declutter its __init__ method a bit.

• 2023-12-13 tmadlener (PR#530)

  • Remove the reading of the deprecated old-style format of component definitions in the YAML files.

• 2023-12-13 tmadlener (PR#485)

  • Remove the deprecated EventStore functionality as announced in #429

• 2023-12-12 tmadlener (PR#529)

  • Switch the relation range tests to use Frame based I/O.

• 2023-12-06 tmadlener (PR#526)

  • Switch the edm4hep workflows to an LCG stack with a recent enough version of CMake. Necessary after key4hep/EDM4hep#235

• 2023-12-05 jmcarcell (PR#523)

  • Remove comment with file name and line number. It's very unlikely it remains up to date when either the name or the content of the files changes

• 2023-12-04 jmcarcell (PR#521)

  • Do not import ROOT when using podio-dump --help, otherwise it can take a while depending on the system only to print the help.

• 2023-12-04 tmadlener (PR#514)

  • Introduce the MaybeSharedPtr to manage the Obj* in the user facing handle classes.
    • This splits the control block and the managed object into two distinct entities with potentially different lifetimes, which allows to fix #174 and #492.
    • This increases the size of the user facing handle classes by a factor two, since they are now effectively two pointers instead of one, even if the control block will not be initialized in case a handle is obtained from a collection.
  • Remove the ObjBase base class and make the ObjectID a member of the Obj classes.
  • Make the user facing handle class constructors from an Obj* private as users will not have access to raw Obj* in any case.
    • Introduce a static makeEmpty method for the generated classes in order to create an empty handle, which is also used internally to handle unpersisted relations.
  • Enable more existing test cases in sanitizer workflows now that it has become possible to do so.

• 2023-12-04 Ananya Gupta (PR#473)

  • Added Julia code generation support in the existing Python interface.
  • Implemented a new design structure for generated Julia code.
  • Added default parameters in constructor definitions with support for Abstract types (for builtins).
  • Created _sort_components_and_datatypes function to perform topological sort on components and datatypes.
  • Created _has_static_arrays_import to check for the need of using Static Arrays in the generated julia code.
  • Added --lang (-l) programming language argument to specify the programming language for code generation, current choices: cpp and julia, default: cpp.
  • Added --upstream-edm code generation support for julia.
  • Added tests in the unit test suite, covering the Julia code generation of the example data models.
  • Added documentation for julia code generation.
  • Added ENABLE_JULIA toggle option. By default it is OFF.

• 2023-12-01 jmcarcell (PR#520)

  • Add an error message when there is an std::bad_function_call, which currently shows
    a stacktrace and is quite uninformative.

• 2023-12-01 tmadlener (PR#519)

  • Make generated member getter functions return by value for builtin types. Keep return by const reference for all other types. Fixes #518

• 2023-12-01 tmadlener (PR#488)

  • Add python bindings for the RNTuple reader and writer
  • Make podio-dump understand RNTuple based files
  • Fix missing storage of datamodel definitions for RNTuple based files

v00-17-03

v00-17-03

• 2023-11-14 tmadlener (PR#513)
  • Introduce checks in ROOTFrameWriter::writeFrame and ROOTNTupleWriter::writeFrame that ensure consistent contents for all Frames of a given category. If inconsistent contents are found an exception is thrown. Before these changes this might lead to a crash or to unreadable files. Fixes #382
  • Refactor ROOTNTupleWriter internals to have only one map that keeps track of categories instead of two maps and a set that need to be kept consistent.

v00-17-02

v00-17-02

• 2023-11-08 jmcarcell (PR#511)

  • Decouple generation tools and files from the rest of the podio python files by creating a new folder called podio_gen. This is a transparent change for users that only use the generator script.
    • This makes the configuration / generation times negligible again, because we don't load libraries unnecessarily any longer for the generation.
  • Simplify the python bindings (podio) __init__.py and remove the test_utils from it.
  • Move the tests for writing frames in python to the tests folder, where they belong and also test the SIO python writer.

• 2023-11-06 jmcarcell (PR#510)

  • Fix legacy tests; an extra argument was being passed and default in the .cpp file example_frame.root was being used which (almost) always exists (because there is a another test creating it) so it was hard to notice.

• 2023-11-06 jmcarcell (PR#509)

  • Add an option for using clang format and set it to off by default. It is significantly slower to run cmake with this option on.

• 2023-11-02 jmcarcell (PR#508)

  • Use the cmake ExternalData module to manage test data. This lets cmake take care of downloading the tests by hash so they can be version controlled. In addition, it's possible to set up a local store using -DExternalData_OBJECT_STORES=/path/to/store and it will download the test files there if they are not there but otherwise use them from there, so building from scratch won't download the test files again.

• 2023-10-16 jmcarcell (PR#507)

  • Copy .clang-format to the dumpmodel test directory and fix some tests when the build directory is not a subdirectory of the main directory. In some tests clang-format will try to find a .clang-format looking in the directories above and if it doesn't exist it will format files differently and some tests will fail.

v00-17-01

v00-17-01

• 2023-10-12 tmadlener (PR#505)

  • Bump the pylint version for CI to 2.17.7

• 2023-10-11 tmadlener (PR#502)

  • Add a deleteBuffers function that is populated at generation time to the CollectionReadBuffers to make it possible to dispose of unconsumed buffers. This fixes the main leaks described in #500
  • Make the ROOTFrameData clean up all unconsumed buffers at desctruction.
  • Make sure to delete buffers that are no longer necessary in the CollectionData constructor. This fixes some more leaks described in #500

• 2023-10-08 Wouter Deconinck (PR#503)

  • Install podio-vis

• 2023-10-04 jmcarcell (PR#497)

  • Move podio_PYTHON_DIR to the top level CMakeLists so that it is set even when BUILD_TESTING is off

• 2023-10-02 jmcarcell (PR#496)

  • Add an operator != to fix negative comparisons since after #493 now id() returns a podioObjectID

• 2023-09-29 tmadlener (PR#493)

  • Make [Mutable]Object::id() return a podio::ObjectID instead of unsigned, since the latter has become useless with #412. Fixes #438
  • Add an operator<<(std::ostream&) for podio::ObjectID

v00-17

v00-17

Barring any unforeseen issues this will be the last pre-release series.

We plan to no longer change the file layouts, especially the default ROOT (TTree based) one. Please let us know if you have any issue that you would still like to be addressed before a version 1.0 release.

Changes w.r.t. v00-16-05

• 2023-09-22 Juraj Smiesko (PR#491)

  • podio-dump: print warning if requested entry not present in the file

• 2023-09-22 tmadlener (PR#490)

  • Fix bugs in python imports when podio is built without SIO support. Fixes #489

• 2023-09-22 tmadlener (PR#486)

  • Make sure to initialize ObjectIDs to untracked to properly track whether they have been added to a Frame or not
  • Change CollectionIDTable interfaces of name and collectionID to return optional to signal whether a collection (ID) is known to the table. This is a breaking change if you use the CollectionIDTable!
    • Avoids having to do the lookup twice to check existence and a subsequent retrieval
  • Fix bug of overly shared CollectionIDTable in ROOTNTupleReader that was uncovered by the CollectionIDTable switch to optional returns.
  • Switch tests from EventStore to Frame based I/O.
  • Fix bug in Frame based I/O that lead to crashes when trying to resolve relations to unpersisted objects.

• 2023-09-18 tmadlener (PR#484)

  • Make the podio python bindings import structure "feel more pythonic"

• 2023-09-15 Benedikt Hegner (PR#483)

  • Clarify error message in case of not implemented schema changes

• 2023-09-15 Benedikt Hegner (PR#482)

  • rename CMake macro createBuffers into create_buffers

• 2023-09-13 jmcarcell (PR#481)

  • Rename the cmake executable or target unittest to unittest_podio, to avoid possible collisions since the unittest name is relatively common

• 2023-09-13 Benedikt Hegner (PR#480)

  • Move the code generation of buffers into the 'create_buffers' macro

• 2023-09-13 Thomas Madlener (PR#472)

  • Allow comparison of data schemata across versions
  • Provide syntax to clarify user intentions in schema evolution
  • Provide schema evolution implementation based on ROOT backend
  • Include infrastructure for future support for schema evolution in other backends
  • Documentation for schema evolution functionality

• 2023-09-11 tmadlener (PR#477)

  • Use nlohmann/json_fwd.hpp in headers to reduce unnecessary template instantiations. Fixes #475

• 2023-09-08 tmadlener (PR#478)

  • Add empty method to CollectionBase
  • Add operator== to the collection iterators

• 2023-09-08 Dmitry Kalinkin (PR#465)

  • Introduce member typedefs to the user facing classes.
    • Object's collection type can now be referenced as Object::collection_type. Conversely, the object type is reachable as ObjectCollection::value_type. The mutable objects can be reached via Object::mutable_type.

• 2023-08-30 tmadlener (PR#471)

  • Initialize the branch names to be index based for reading (i.e. legacy behavior) for all releases of the v00-16 series.

• 2023-08-22 Andre Sailer (PR#469)

  • Tests: update required catch2 version to 3.4 for builds with c++20
  • CI: use clang16 on el9 (alma9), instead of clang12 on cs7, using c++20
  • CI: disable key4hep-release-based tests (tabulate available)

• 2023-08-22 Benedikt Hegner (PR#445)

  • Allow to specify units as part of the datamodel definition

• 2023-07-26 tmadlener (PR#463)

  • Make sure to only access vector member buffers of collections of datatypes with VectorMembers if they actually exist. This is necessary to make subset collections of such datatypes work in I/O. Fixes #462
  • Add a test that reproduces the original issue and is fixed by this.

• 2023-07-25 tmadlener (PR#461)

  • Make sure the ROOTFrameReader on the master branch can read v00-16-06 files
  • Add v00-16-06 to the legacy versions that are tested

• 2023-07-25 tmadlener (PR#447)

  • Add a python wrapper around the different available Frame writers.
  • Add a put method to the Frame wrapper that allows to add collections to the Frame without having to explicitly use cppyy.gbl.std.move. Fixes #432
  • Add test cases that write via python bindings and read via c++.

• 2023-07-20 tmadlener (PR#457)

  • Simplify the test setup for SIO in CMake and make it explicit on the ENABLE_SIO option rather than on the presence of targets.

• 2023-07-18 jmcarcell (PR#456)

  • Cache podio_PYTHON_DIR

• 2023-07-18 jmcarcell (PR#455)

  • Rename CMAKE_BINARY_DIR to PROJECT_BINARY_DIR

• 2023-07-18 tmadlener (PR#439)

  • Introduce the FrameCategories.h header that puts a few of the conventions and otherwise hardcoded strings into variables / functions.

• 2023-07-14 jmcarcell (PR#454)

  • Rename CMAKE_{SOURCE,BIN}_DIR to PROJECT_{SOURCE,BIN}_DIR

• 2023-07-14 tmadlener (PR#452)

  • Extend the pre-processor condition for the to_json functionality to not be visible in rootcling or the root interpreter. Fixes #435

• 2023-07-13 Benedikt Hegner (PR#450)

  • Add optional description and author fields to component definition

• 2023-07-13 tmadlener (PR#449)

  • Fix the pre-commit workflow by making it run on top of the key4hep nightlies that come with a recent enough root version to be able to work with the RNTuple addition (#395)
  • Fix a few minor complaints from newer versions of clang-format, clang-tidy and pylint
  • Enable building the RNTuple backend for more workflows (anything that comes with a new enough ROOT essentially).

• 2023-07-13 tmadlener (PR#448)

  • Remove the lcio datalayout which has been untouched (and unbuilt) for quite a few years.

• 2023-07-13 tmadlener (PR#446)

  • Make calling finish for the SIOFrameWriter non-mandatory. See #442 for other related changes.

• 2023-07-11 jmcarcell (PR#442)

  • Allow not calling finish() when using the writers

• 2023-07-11 jmcarcell (PR#395)

  • Add support for the new RNTuple format by adding a writer, reader and tests.

• 2023-06-30 Ananya Gupta (PR#437)

  • Modified parse function definition to incorporate missing description in member definition error message. Fixes #436

• 2023-06-27 Thomas Madlener (PR#413)

  • Introduce podio::SchemaEvolution that can hold schema evolution functions and that offers an evolveBuffers method that does the schema evolution on these buffers before collections are created.
  • Add hooks in podio::Frame to call this when collections are read from the FrameData.

• 2023-06-23 tmadlener (PR#434)

  • Properly handle the slightly different branch contents before v00-16-04. Fixes #433
  • Add tests that use the ROOTLegacyReader to actually read the downloaded legacy files

• 2023-06-15 tmadlener (PR#428)

  • Split the tests directory into several (more or less) topical sub-directories to declutter the main test CMakeLists.txt a bit
  • Move commonly used functionality into cmake/podioTests.cmake (e.g. setting up a test environment)
  • Move python unittests config to the python directory

• 2023-06-15 tmadlener (PR#427)

  • Delay library loading as long as possible, mainly for quicker responses for --help
  • Add a --version flag for dumping the podio version
  • Display collections and parameters in alphabetical order and automatically adjust column widths to fit contents (using the tabulate package).

• 2023-06-09 Thomas Madlener (PR#402)

  • Add public static constexpr char* type names to the collections and make the getXXXName() methods return string_views to these strings. This is a breaking change to the interface of the collections if you explicitly rely on them being std::string
    • typeName: the full type name of the collection (returned also by getTypeName)
    • valueTypeName: the (immutable) type name of the objects of the collection (returned by getValueTypeName)
    • dataTypeName: the type name of the data PODs (returned by getDataTypeName)
  • Make unittest environment properly use PODIO_SIOBLOCK_PATH
  • USE_EXTERNAL_CATCH2 now can also be set to AUTO to look for a suitable version of Catch2 before falling back and fetching and building it's own version instead of a hard fail.

• 2023-...

v00-16-07

v00-16-07

• 2023-08-30 Thomas Madlener (PR#470)
  • Backport of #439
  • (Partial) backport of #469 to fix CI for this branch. Removed the RNTuple setup since that is not available here yet.
  • Backport the AUTO configuration option for the cmake USE_EXTERNAL_CATCH2 config parameter

v00-16-06

v00-16-06

• 2023-07-14 Thomas Madlener (PR#453)

  • Backport of #452

• 2023-07-13 Thomas Madlener (PR#451)

  • Backport some bug fixes to make a v00-16-06 patch release
    • #422
    • #423 (fixes only, no additional tests)
    • #434 (fixes only, not the additional tests)

v00-16-05

v00-16-05

• 2023-05-23 tmadlener (PR#420)
  • Fix a version check inside the ROOTReader to avoid segmentation violations

v00-16-04

v00-16-04

WARNING: The ROOTReader (deprecated) is broken in this release, please use v00-16-05 that has a fix for the issue.

• 2023-05-23 tmadlener (PR#417)

  • Fix an issue with reading multiple files via the ROOTFrameReader (#411)
    • Add documentation for API of opening file(s)
    • Add tests for reading multiple files

• 2023-05-22 tmadlener (PR#418)

  • Bring back the public templated getMap functionality for podio::GenericParameters as they are already used in DD4hep (see AIDASoft/DD4hep#1112).
    • Mark the existing getXYZMap as deprecated but keep them for a brief transition period.
    • These have been removed in #415.

• 2023-05-19 jmcarcell (PR#416)

  • Remove selection rules for classes that don't exist anymore

• 2023-05-15 jmcarcell (PR#415)

  • Remove the deprecated getters and setters from the generic parameters

• 2023-05-15 jmcarcell (PR#410)

  • Remove the square that is run when cmake runs

• 2023-05-09 tmadlener (PR#414)

  • Fix off-by-one error in UserDataCollection::print that caused the first element to be printed twice.

• 2023-05-09 Thomas Madlener (PR#394)

  • Introduce a CollectionBufferFactory that can create the necessary buffers from a collection type, a schema version and a subset collection flag.
    • Use this factory throughout all existing Readers
    • Remove createBuffers and createSchemaEvolvableBuffers from podio::CollectionBase interface
  • Make the minimum allowed schema_version 1 in the yaml definition files. Default to 1 if no schema_version is provided
  • Add a schemaVersion to the DatamodelDefinition.h header that is generated and that can be accessed via {{ package_name }}::meta::schemaVersion. Use this to propagate schema information to the necessary places.
  • Make SIOBlocks write the current schema version, such that on reading they can generate the appropriate buffers for the version on file.

• 2023-04-22 Christopher Dilks (PR#408)

  • fix type inconsistency between Collection::size() and index for const object accessors

• 2023-04-21 jmcarcell (PR#387)

  • Make sure that the dump model round trip tests work without ENABLE_SIO
  • Actually test the extension model dumping

• 2023-04-12 Thomas Madlener (PR#400)

  • Fix a bug in SIOFrameData::getAvailableCollections to also work with Frames where some of the collections have not been written and that could lead to a seg fault.
  • Add a test for this in c++ (previously only covered in python unittests of Frame).

• 2023-04-05 Thomas Madlener (PR#399)

  • Add PODIO_ENABLE_SIO=1 to the public target_compile_definitions for podioSioIO so that all dependent targets automatically get it as well. This should make it easier to use SIO dependent features in dependencies.
  • Consistently use a scope for target_link_libraries in tests.

• 2023-04-03 Paul Gessinger-Befurt (PR#398)

  • Do not reject building if ROOT was built with C++20 (instead of C++17).

• 2023-04-03 Thomas Madlener (PR#397)

  • Remove the GENERATED property from generated files in CMake to avoid inconsistent removal of headers and source files with the clean target. Fixes #396

• 2023-03-15 Benedikt Hegner (PR#341)

  • Adding infrastructure for schema evolution
  • Added explicit version tracking to the metadata
  • Data model comparison tool w/ simple heuristics to identify potential omissions / mistakes (e.g. checking for the limits of the ROOT backend)
  • Changed handling of backwards compatibility for the collection info metadata