PointCloud interoperability between cupoch and Open3D

[dlzou]

Firstly, thanks for the effort put into this great library!

I have some Python code that uses cupoch for ICP registration. Afterward, it calls get_information_matrix from Open3D API. Currently, the code does this to convert from cupoch.geometry.PointCloud to open3d.geometry.PointCloud:

import cupoch as cph
import open3d as o3d

...

cloud_A = cph.geometry.PointCloud()
cloud_B = cph.geometry.PointCloud()
cloud_A.points = cph.utility.Vector3fVector(pts_A)
cloud_B.points = cph.utility.Vector3fVector(pts_B)

tfB2A = cph.registration.registration_icp(
    cloud_B,
    cloud_A,
    ...
)

# Conversion
cloud_A_cpu = o3d.geometry.PointCloud()
cloud_B_cpu = o3d.geometry.PointCloud()
pts_A_cpu = np.asarray(cloud_A.points.cpu()).astype(float, copy=False)
cloud_A_cpu.points = o3d.utility.Vector3dVector(pts_A_cpu)
pts_B_cpu = np.asarray(cloud_B.points.cpu()).astype(float, copy=False)
cloud_B_cpu.points = o3d.utility.Vector3dVector(pts_B_cpu)

information_icp = (
    o3d.pipelines.registration.get_information_matrix_from_point_clouds(
        cloud_B_cpu,
        cloud_A_cpu,
        INFORMATION_MATRIX_CORRESPONDENCE_DISTANCE,
        tfB2A,
    )
)

From timing the section commented as "Conversion" I suspect it's doing some unnecessary copying. Is there a faster way to do this conversion?

[eclipse0922]

To facilitate interoperability between Cupoch and Open3D, it's essential to understand their memory management and data structures:

Cupoch: Designed for GPU acceleration, Cupoch primarily operates on GPU (device) memory.

Open3D: Offers two types of geometries:

Legacy Geometry: Utilizes CPU (host) memory.
Tensor-based Geometry: Employs open3d.core.Tensor, which can reside in either CPU or GPU memory, depending on the specified device.
When transferring data between Cupoch and Open3D, consider the following:

Device Compatibility: If both data structures are on the same device (both on CPU or both on GPU), you can achieve zero-copy data sharing using DLPack. Open3D supports conversion to and from PyTorch tensors without memory copy through DLPack, enabling efficient data sharing between different array libraries.
OPEN3D

Device Mismatch: If one data structure is on the CPU and the other on the GPU, data transfer will necessitate copying between host and device memory.

By aligning the memory locations of your data structures and utilizing DLPack, you can optimize interoperability between Cupoch and Open3D.

https://www.open3d.org/html/python_api/open3d.core.Tensor.html#open3d.core.Tensor.from_dlpack
https://www.open3d.org/html/python_api/open3d.core.Tensor.html#open3d.core.Tensor.to_dlpack

https://github.com/neka-nat/cupoch/blob/main/examples/python/basic/from_torch_tensor.py
https://github.com/neka-nat/cupoch/blob/main/examples/python/basic/to_torch_tensor.py

import cupoch as cph
import open3d as o3d
import numpy as np

# declare a cupoch point cloud object sample postion 1, 2, 3
cupoch_pcd = cph.geometry.PointCloud()
cupoch_pcd.points = cph.utility.Vector3fVector([[1, 2, 3]])

# convert cupoch to dlpack
pt_dl = cupoch_pcd.to_points_dlpack()

# Import DLPack tensor into Open3D tensor
o3d_pcd = o3d.t.geometry.PointCloud(o3d.core.Tensor.from_dlpack(pt_dl))

# prtint values
print("converted to open3d")
print(o3d_pcd.point)
print(o3d_pcd.point.positions)

# convert open3d to dlpack
dl_o3d = o3d.core.Tensor.to_dlpack(o3d_pcd.point.positions)

# Import DLPack tensor into cupoch tensor
cupoch_pcd2 = cph.geometry.PointCloud()
cupoch_pcd2.from_points_dlpack(dl_o3d)

print("converted back to cupoch")
print(np.asarray(cupoch_pcd2.points.cpu()))

result

converted to open3d
TensorMap(primary_key="positions") with 1 attribute:
  - positions: shape={1, 3}, dtype=Float32, device=CUDA:0 (primary)
  (Use . to access attributes, e.g., tensor_map.positions)
[[1 2 3]]
Tensor[shape={1, 3}, stride={3, 1}, Float32, CPU:0, 0xf7d4560]
Tensor[shape={1, 3}, stride={3, 1}, Float32, CUDA:0, 0x78118da00000]
converted back to cupoch
[[1. 2. 3.]]