Paraview and VisIt readers read the same dataset differently (and both incorrectly)

[BenWibking]

Describe the bug
I opened the example-2d HDF5 dataset from openpmd-example-datasets (https://github.com/openPMD/openPMD-example-datasets/blob/draft/example-2d.tar.gz) in both Paraview (using the openpmd-api reader with a *.pmd file) and with the OpenPMD Visit reader.

Wrong in VisIt:

• The data layout of the field data appears to be wrong in VisIt.
• The particles do not render at all in VisIt.
• The axes are wrong (should be x-z instead of x-y)

Wrong in Paraview:

• The relative alignment of the field and particle data is wrong:
  • The field is plotted in the x-y plane (should be x-z)

To Reproduce
Open the example-2d dataset in both Paraview and VisIt using their respective OpenPMD readers.

Expected behavior
They should be the same!

Software Environment

• version of Paraview: 5.13.1
• version of VisIt: 3.4.2

Additional context
Each plugin does it own data layout transformation (incorrectly): #1708

[BenWibking]

I've uploaded the Visit session and Paraview state files for reproducibility:
paraview_state_openpmd_example_2d.pvsm.zip
visit_session_openpmd_example_2d.zip

[BenWibking]

@ax3l Do you know what is going on here?

[ax3l]

Hi @BenWibking , thank you for the report!

For the different readers, we track the issues that appear upstream:
VisIt: https://github.com/visit-dav/visit/issues?q=is%3Aissue%20state%3Aopen%20openPMD

ParaView: https://gitlab.kitware.com/paraview/paraview/-/issues/?sort=created_date&state=all&search=openPMD&first_page_size=20

For ParaView, I did fix the transpose for 3D data sets a while ago, but maybe there is a bug left for 2D that gets them wrong... I can test this with the latest release, thanks! I will also check if the 2D example from old WARP is actually correctly annotated or if the dataOrder and axisLabels are misformed.

For VisIt, the current reader is directly written in HDF5 / C and might not support the openPMD 1 dataOrder attribute correctly. You are working on an implementation in openPMD-api, do I recall correctly?

[BenWibking]

Hi @BenWibking , thank you for the report!

For the different readers, we track the issues that appear upstream: VisIt: https://github.com/visit-dav/visit/issues?q=is%3Aissue%20state%3Aopen%20openPMD

ParaView: https://gitlab.kitware.com/paraview/paraview/-/issues/?sort=created_date&state=all&search=openPMD&first_page_size=20

Ok, let me know if you'd prefer that I open issues upstream for each of these. The inconsistency between the two is a sort of meta-issue that I didn't know where to best report.

For ParaView, I did fix the transpose for 3D data sets a while ago, but maybe there is a bug left for 2D that gets them wrong... I can test this with the latest release, thanks! I will also check if the 2D example from old WARP is actually correctly annotated or if the dataOrder and axisLabels are misformed.

For VisIt, the current reader is directly written in HDF5 / C and might not support the openPMD 1 dataOrder attribute correctly. You are working on an implementation in openPMD-api, do I recall correctly?

Yes, I'm working on that in this repo (https://github.com/BenWibking/openpmd-visit-reader). That motivated this investigation, since I was unable to get my reader working for non-native data layout orderings (and then I discovered this inconsistency with the existing readers).