Issue 7153 - 3DGS IO #7191

lumurillo · 2025-03-08T04:52:33Z

Fixes #7153

Bug fix (non-breaking change which fixes an issue): Fixes #
New feature (non-breaking change which adds functionality). Resolves #
Breaking change (fix or feature that would cause existing functionality to not work as expected) Resolves #

Motivation and Context

Provide IO capabilities with 3DGS support for files like PLY and SPLAT.

Checklist:

I have run python util/check_style.py --apply to apply Open3D code style
to my code.
This PR changes Open3D behavior or adds new functionality.
- Both C++ (Doxygen) and Python (Sphinx / Google style) documentation is
  updated accordingly.
- I have added or updated C++ and / or Python unit tests OR included test
  results (e.g. screenshots or numbers) here.
I will follow up and update the code if CI fails.
For fork PRs, I have selected Allow edits from maintainers.

Notes

Some of the test may fail because we need that the sample files are uploaded to the corresponding repo.

* Include additional 3DGS properties for reading operation * Add validation methods for 3DGS mandatory properties * Read and Write including additional vertex * Add the validation check to the write method * Add a unit test for a valid ReadCloutPoint operation * PLY read to point cloud * PLY write correction * Inline documentation * Add constant defines for magic numbers and attribute names * Add unit tests for reading and writing 3DGS formatted point clouds * Add 3DGS attribute verification to the read operation --------- Co-authored-by: Juan Cruz <[email protected]> Co-authored-by: Gomez Rodriguez, Diego <[email protected]>

* Add splat writer * Add splat reader * Splat reader added to PointCloudIO.h * Fix CMakeList.txt * Add test for reading PointCloud from SPLAT format * Fixing Style --------- Co-authored-by: ikun03 <[email protected]>

update-docs · 2025-03-08T04:52:37Z

Thanks for submitting this pull request! The maintainers of this repository would appreciate if you could update the CHANGELOG.md based on your changes.

review-notebook-app · 2025-03-08T04:52:38Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

ssheorey

Thanks @lumurillo and team for this very useful contribution!

Naming: Can we replace the use of 3DGS to GaussianSplat everywhere?
This also take care of variants of 3DGS such as 2D Gaussian Splatting. Also splat seems to be the more common / short name for this representation in other software.

Btw: a good way to remember the f_rest size is that it's n^2-1 for order n :-)

PS: Review is WIP: Will review test and docs later.

cpp/open3d/t/geometry/PointCloud.h

cpp/open3d/t/io/file_format/FilePLY.cpp

cpp/open3d/t/io/file_format/FileSPLAT.cpp

ssheorey · 2025-03-09T07:29:54Z

cpp/open3d/t/io/file_format/FileSPLAT.cpp

+                    rot.cwiseMin(255.0).cwiseMax(0.0).cast<int>();
+
+            for (int idx = 0; idx < 4; ++idx) {
+                splat_write_byte(splat_file, int_to_byte(int_rot[idx]));


The bounds checking inside int_to_byte is unnecessary, since we have clamped the values above. Simpler static_cast should be faster.

ssheorey · 2025-03-09T07:34:07Z

cpp/tests/t/io/PointCloudIO.cpp

@@ -309,16 +359,11 @@ TEST(TPointCloudIO, WritePTSColorConversion1) {

 // Check PTS color boolean to uint8 conversion.
 TEST(TPointCloudIO, WritePTSColorConversion2) {


Is this .pts file IO test deletion accidental?

cpp/tests/t/io/PointCloudIO.cpp

ssheorey

I think we need to modify the unit tests a bit:

ply:

Start with a ply file as a string [Done].
Create tensors for each of the attributes (manually fill them in). See example here.
Write ply string to disk as ply.
Read it back from disk.
Test - all attributes are present, are the right shape and data type as the tensors in step 2.
Test all attribute values match (See example here).

splat:

Start with .splat file as a binary blob - you can get this by converting the .ply file above to .splat with the reference convert.py script (antimatter github). You can convert it to a C array with xxd -i infile.splat outfile.h. If the C array is too big, let me know and we can add it directly to the git repo (with some cmake additions). This step directly proves that antimatter reference splats can be read by Open3D.
Start with the same tensor attributes as in the ply test.
Write it to disk as splat.
Read it back from disk.
Test - all attributes are present, are the right shape and data type as the tensors.
Test all attribute values match (See example here).

This simplifies the tests by using round trip test, no data file is needed for splat,

cpp/tests/t/io/PointCloudIO.cpp

ssheorey · 2025-03-10T21:43:16Z

docs/jupyter/io/files/point_cloud_sampled_only_table.splat

We don't add data to the repo (as much as possible), so we can delete it from here. Instead, I'll add it to the Open3D downloads site.

ssheorey

Thanks @joserochh!

cpp/open3d/t/io/file_format/FileSPLAT.cpp

joserochh · 2025-03-21T19:33:53Z

I think we need to modify the unit tests a bit:

ply:

Start with a ply file as a string [Done].

Create tensors for each of the attributes (manually fill them in). See example here.

Write ply string to disk as ply.

Read it back from disk.

Test - all attributes are present, are the right shape and data type as the tensors in step 2.

Test all attribute values match (See example here).

splat:

Start with .splat file as a binary blob - you can get this by converting the .ply file above to .splat with the reference convert.py script (antimatter github). You can convert it to a C array with xxd -i infile.splat outfile.h. If the C array is too big, let me know and we can add it directly to the git repo (with some cmake additions). This step directly proves that antimatter reference splats can be read by Open3D.

Start with the same tensor attributes as in the ply test.

Write it to disk as splat.

Read it back from disk.

Test - all attributes are present, are the right shape and data type as the tensors.

Test all attribute values match (See example here).

This simplifies the tests by using round trip test, no data file is needed for splat,

I have modified splat unit tests.

lumurillo and others added 3 commits March 7, 2025 12:45

Add first draft of 3DGS documentation

7fd485f

Add splat writer

b5071b9

* Add splat writer * Add splat reader * Splat reader added to PointCloudIO.h * Fix CMakeList.txt * Add test for reading PointCloud from SPLAT format * Fixing Style --------- Co-authored-by: ikun03 <[email protected]>

lumurillo changed the title ~~Issue 7153 dev~~ Issue 7153 - 3DGS IO Mar 8, 2025

ssheorey requested review from benjaminum, ssheorey and aditya18007 March 8, 2025 06:20

ssheorey requested changes Mar 9, 2025

View reviewed changes

ssheorey requested changes Mar 10, 2025

View reviewed changes

ssheorey added the hackathon label Mar 10, 2025

Applying fixes from code review

6f37deb

ssheorey reviewed Mar 20, 2025

View reviewed changes

cpp/open3d/t/io/file_format/FileSPLAT.cpp Show resolved Hide resolved

Applying further fixes from code review

386b8bc

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue 7153 - 3DGS IO #7191

Issue 7153 - 3DGS IO #7191

lumurillo commented Mar 8, 2025

update-docs bot commented Mar 8, 2025

review-notebook-app bot commented Mar 8, 2025

ssheorey left a comment •

edited

Loading

ssheorey Mar 9, 2025

joserochh Mar 19, 2025

ssheorey Mar 9, 2025

ssheorey left a comment

ssheorey Mar 10, 2025

ssheorey left a comment

joserochh commented Mar 21, 2025

		@@ -309,16 +359,11 @@ TEST(TPointCloudIO, WritePTSColorConversion1) {

		// Check PTS color boolean to uint8 conversion.
		TEST(TPointCloudIO, WritePTSColorConversion2) {

Issue 7153 - 3DGS IO #7191

Are you sure you want to change the base?

Issue 7153 - 3DGS IO #7191

Conversation

lumurillo commented Mar 8, 2025

Motivation and Context

Checklist:

Notes

update-docs bot commented Mar 8, 2025

review-notebook-app bot commented Mar 8, 2025

ssheorey left a comment • edited Loading

Choose a reason for hiding this comment

ssheorey Mar 9, 2025

Choose a reason for hiding this comment

joserochh Mar 19, 2025

Choose a reason for hiding this comment

ssheorey Mar 9, 2025

Choose a reason for hiding this comment

ssheorey left a comment

Choose a reason for hiding this comment

ssheorey Mar 10, 2025

Choose a reason for hiding this comment

ssheorey left a comment

Choose a reason for hiding this comment

joserochh commented Mar 21, 2025

ssheorey left a comment •

edited

Loading