-
Notifications
You must be signed in to change notification settings - Fork 96
Description
Some(not all. ) of this code are created by AI, So it may need some small change to finished
some part of my envirments(I have to use cuda 12.8 cause that damn Blackwell in RTX50+):
torch 2.8.0+cu128+cp311(in nightly version)
torchaudio 2.7.0+cu128+cp311(in nightly version)
torchvision 0.22.0+cu128+cp311
open3d 0.15.1+
...
- Here's some fixed:
1. in windows only: cannt find vdbfusion
I've tried to rebuild the code from vdbfusion.VDBVolume(source: https://github.com/PRBonn/vdbfusion/blob/main/src/vdbfusion/pybind/vdb_volume.py) by using open3d.TSDF. So we need to change the source code in this project in extract_mesh.py.
find this line in the code:vdb_volume = vdbfusion.VDBVolume(voxel_size=0.01, sdf_trunc=0.04, space_carving=False) # For Scene, replace this line to
voxel_size = 0.01
sdf_trunc = 0.04
volume = open3d.pipelines.integration.ScalableTSDFVolume(
voxel_length=voxel_size,
sdf_trunc=sdf_trunc,
color_type=open3d.pipelines.integration.TSDFVolumeColorType.RGB8
)
then find this partfor camera in tqdm(cameras[::3]) :.... geo_mesh.export(os.path.join(work_dir, 'fused_mesh.ply'))replace them all into code
for camera in tqdm(cameras[::3]):
# Record original parameters before downsampling
original_width = camera.image_width
original_height = camera.image_height
original_fx = camera.intrinsics[0, 0].item()
original_fy = camera.intrinsics[1, 1].item()
original_cx = camera.intrinsics[0, 2].item()
original_cy = camera.intrinsics[1, 2].item()
camera.downsample_scale(args.resolution)
camera = camera.to("cuda")
# Get color information during rendering
with torch.no_grad():
render_pkg = renderer.render(camera, pcd)
rendering = render_pkg["render"] # [3, H, W]
# Extract color image and convert format
color = rendering.permute(1, 2, 0).cpu().numpy() # [H, W, 3]
color_image = (color * 255).astype(np.uint8)
color_image = np.ascontiguousarray(color_image)
color_image = open3d.geometry.Image(color_image)
# Extract depth map and convert format
depth = render_pkg["rendered_median_depth"][0].cpu().numpy()
depth_image = open3d.geometry.Image((depth * 1000).astype(np.uint16))
# Correct pose matrix (remove inverse)
cam_pose = camera.extrinsics.cpu().numpy() # Use extrinsic matrix directly
# Build correct intrinsic parameters
scaled_fx = original_fx / args.resolution
scaled_fy = original_fy / args.resolution
scaled_cx = original_cx / args.resolution
scaled_cy = original_cy / args.resolution
intrinsic = open3d.camera.PinholeCameraIntrinsic(
width=int(original_width / args.resolution),
height=int(original_height / args.resolution),
fx=scaled_fx,
fy=scaled_fy,
cx=scaled_cx,
cy=scaled_cy
)
# Create colored TSDF volume
volume = open3d.pipelines.integration.ScalableTSDFVolume(
voxel_length=0.01,
sdf_trunc=0.04,
color_type=open3d.pipelines.integration.TSDFVolumeColorType.RGB8
)
# Integrate color and depth data
rgbd_image = open3d.geometry.RGBDImage.create_from_color_and_depth(
color_image,
depth_image,
depth_scale=1000.0,
depth_trunc=10.0,
convert_rgb_to_intensity=False
)
volume.integrate(rgbd_image, intrinsic, cam_pose)
# Export final colored mesh
mesh = volume.extract_triangle_mesh()
open3d.io.write_triangle_mesh(
str(Path(work_dir) / 'colored_mesh.ply'),
mesh,
write_vertex_colors=True
)
2. some bug from source code
find some default data error
in project ./datasets/__init__.py find trans: np.array = np.array([0.0, 0.0, 0.0]) replace into trans: np.ndarray = field(default_factory=lambda: np.zeros(3, dtype=np.float32))
then find principal_point_ndc: np.array = np.array([0.5, 0.5]) replace them into principal_point_ndc: np.ndarray = field(default_factory=lambda: np.array([0.5, 0.5], dtype=np.float32))
I can't find the logs(forgot to save that) but this ensures type safety and prevents hard-to-debug issues with shared mutable state between different class instances.
3. root problem
The new version of gaussian provide a new structure(or the mistakes from your programmer) cause the code args.source_path = os.path.join(os.path.dirname(model_path), "cameras.json") can't find the root. So we have to change the root in to args.source_path = os.path.join(model_path, "cameras.json")
The structure from latest gaussian version provide like this:
as you can see, the camera is in the model's path not in the parent path. So we don't need to use os.path.dirname.
and now this project can work.

