Skip to content

I've fixed some problem and this project can work with Win11 and cuda12+ now #96

@Cooldereb

Description

@Cooldereb

Some(not all. ) of this code are created by AI, So it may need some small change to finished
some part of my envirments(I have to use cuda 12.8 cause that damn Blackwell in RTX50+):
torch 2.8.0+cu128+cp311(in nightly version)
torchaudio 2.7.0+cu128+cp311(in nightly version)
torchvision 0.22.0+cu128+cp311
open3d 0.15.1+
...

- Here's some fixed:

1. in windows only: cannt find vdbfusion
I've tried to rebuild the code from vdbfusion.VDBVolume(source: https://github.com/PRBonn/vdbfusion/blob/main/src/vdbfusion/pybind/vdb_volume.py) by using open3d.TSDF. So we need to change the source code in this project in extract_mesh.py.
find this line in the code:vdb_volume = vdbfusion.VDBVolume(voxel_size=0.01, sdf_trunc=0.04, space_carving=False) # For Scene, replace this line to

voxel_size = 0.01
   sdf_trunc = 0.04
   volume = open3d.pipelines.integration.ScalableTSDFVolume(
       voxel_length=voxel_size,
       sdf_trunc=sdf_trunc,
       color_type=open3d.pipelines.integration.TSDFVolumeColorType.RGB8
   )

then find this partfor camera in tqdm(cameras[::3]) :.... geo_mesh.export(os.path.join(work_dir, 'fused_mesh.ply'))replace them all into code

for camera in tqdm(cameras[::3]):
        # Record original parameters before downsampling
        original_width = camera.image_width
        original_height = camera.image_height
        original_fx = camera.intrinsics[0, 0].item()
        original_fy = camera.intrinsics[1, 1].item()
        original_cx = camera.intrinsics[0, 2].item()
        original_cy = camera.intrinsics[1, 2].item()

        camera.downsample_scale(args.resolution)
        camera = camera.to("cuda")

        # Get color information during rendering
        with torch.no_grad():
            render_pkg = renderer.render(camera, pcd)
        rendering = render_pkg["render"]  # [3, H, W]

        # Extract color image and convert format
        color = rendering.permute(1, 2, 0).cpu().numpy()  # [H, W, 3]
        color_image = (color * 255).astype(np.uint8)
        color_image = np.ascontiguousarray(color_image)
        color_image = open3d.geometry.Image(color_image)

        # Extract depth map and convert format
        depth = render_pkg["rendered_median_depth"][0].cpu().numpy()
        depth_image = open3d.geometry.Image((depth * 1000).astype(np.uint16))

        # Correct pose matrix (remove inverse)
        cam_pose = camera.extrinsics.cpu().numpy()  # Use extrinsic matrix directly

        # Build correct intrinsic parameters
        scaled_fx = original_fx / args.resolution
        scaled_fy = original_fy / args.resolution
        scaled_cx = original_cx / args.resolution
        scaled_cy = original_cy / args.resolution

        intrinsic = open3d.camera.PinholeCameraIntrinsic(
            width=int(original_width / args.resolution),
            height=int(original_height / args.resolution),
            fx=scaled_fx,
            fy=scaled_fy,
            cx=scaled_cx,
            cy=scaled_cy
        )

        # Create colored TSDF volume
        volume = open3d.pipelines.integration.ScalableTSDFVolume(
            voxel_length=0.01,
            sdf_trunc=0.04,
            color_type=open3d.pipelines.integration.TSDFVolumeColorType.RGB8
        )

        # Integrate color and depth data
        rgbd_image = open3d.geometry.RGBDImage.create_from_color_and_depth(
            color_image,
            depth_image,
            depth_scale=1000.0,
            depth_trunc=10.0,
            convert_rgb_to_intensity=False
        )
        volume.integrate(rgbd_image, intrinsic, cam_pose)

        # Export final colored mesh
    mesh = volume.extract_triangle_mesh()
    open3d.io.write_triangle_mesh(
        str(Path(work_dir) / 'colored_mesh.ply'),
        mesh,
        write_vertex_colors=True
    )

2. some bug from source code
find some default data error
in project ./datasets/__init__.py find trans: np.array = np.array([0.0, 0.0, 0.0]) replace into trans: np.ndarray = field(default_factory=lambda: np.zeros(3, dtype=np.float32))
then find principal_point_ndc: np.array = np.array([0.5, 0.5]) replace them into principal_point_ndc: np.ndarray = field(default_factory=lambda: np.array([0.5, 0.5], dtype=np.float32))
I can't find the logs(forgot to save that) but this ensures type safety and prevents hard-to-debug issues with shared mutable state between different class instances.
3. root problem
The new version of gaussian provide a new structure(or the mistakes from your programmer) cause the code args.source_path = os.path.join(os.path.dirname(model_path), "cameras.json") can't find the root. So we have to change the root in to args.source_path = os.path.join(model_path, "cameras.json")
The structure from latest gaussian version provide like this:

Image

as you can see, the camera is in the model's path not in the parent path. So we don't need to use os.path.dirname.

and now this project can work.

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions