Details about sampling from 3d objects

[Quest4Science]

Thank you for your excellent work and for sharing your implementation.

I am reading your description:

"To sample 3D point triplets within each part, we convert the surface mesh from PartNet into a tetrahedral mesh using Tetgen [10] and then take the vertices of tet mesh as sampled interior points. All shapes are normalized to [−1, 1]^3 during training, and we uniformly sample 100,000 points per shape as input to the network."

I would like to clarify the precise sampling process described here. Specifically:

            Do the 100,000 sampled points per shape consist only of the tetrahedral mesh vertices generated by Tetgen, or do you also sample additional points?
            
            Are further points sampled uniformly inside the tetrahedral volume (i.e., within the interior of each tetrahedron), or do you include surface points from the original mesh as well?  In other words, besides the vertices of the tetrahedral mesh, are the remaining sampled points drawn from the interior of the object (tetrahedral cells), the surface of the original mesh, or a combination of both?

              If possible, could you briefly explain how the final set of 100,000 points is constructed? This would help me ensure I am correctly reproducing your data preparation pipeline.

Thank you so much for your time and clarification!

[Quest4Science]

Could you please clarify how your pipeline handles non-watertight (open) meshes during tetrahedralization?

Do you perform any mesh repair or hole filling before running TetGen?

If TetGen fails on meshes with holes, do you have a fallback strategy (e.g., using only surface points for sampling)?

Or does your implementation rely on TetGen's internal handling of such cases?

[Colin97]

def compute(vertices: np.ndarray, faces: np.ndarray, size: int = 128,
            fix: bool = False, level: float = 0.015, return_mesh: bool = False):
    r""" Converts a input mesh to signed distance field (SDF).

    Args:
      vertices (np.ndarray): The vertices of the input mesh, the vertices MUST be
          in range [-1, 1].
      faces (np.ndarray): The faces of the input mesh.
      size (int): The resolution of the resulting SDF.
      fix (bool): If the input mesh is not watertight, set :attr:`fix` as True.
      level (float): The value used to extract level sets when :attr:`fix` is True,
          with a default value of 0.015 (as a reference 2/128 = 0.015625). And the
          recommended default value is 2/size.
      return_mesh (bool): If True, also return the fixed mesh.
    """
    # compute sdf
    sdf = mesh2sdf.core.compute(vertices, faces, size)
    if not fix:
        return (sdf, trimesh.Trimesh(vertices, faces)) if return_mesh else sdf

    # NOTE: the negative value is not reliable if the mesh is not watertight
    udf = np.abs(sdf)
    vertices, faces, _, _ = skimage.measure.marching_cubes(udf, level)
    components = trimesh.Trimesh(vertices, faces).split(only_watertight=False)
    new_mesh = [] #trimesh.Trimesh()
    if len(components) > 100000:
        raise NotImplementedError
    for i, c in enumerate(components):
        c.fix_normals()
        new_mesh.append(c) #trimesh.util.concatenate(new_mesh, c)
    new_mesh = trimesh.util.concatenate(new_mesh)

    # generate tet mesh
    tet = tetgen.TetGen(new_mesh.vertices, new_mesh.faces)
    tet.tetrahedralize(plc=True, nobisect=1., quality=True, fixedvolume=True, maxvolume=math.sqrt(2) / 12 * (2 / size) ** 3)
    tmp_vtk = tempfile.NamedTemporaryFile(suffix='.vtk', delete=True)
    tet.grid.save(tmp_vtk.name)

    reader = vtk.vtkUnstructuredGridReader()
    reader.SetFileName(tmp_vtk.name)
    reader.Update()
    
    # extract tet vertices (surface & internal)
    unstructuredGrid = reader.GetOutput()
    points = unstructuredGrid.GetPoints()
    vtk_points = points.GetData()
    tet_points = np.array(numpy_support.vtk_to_numpy(vtk_points)) * (2.0 / size) - 1.0
    
    # extract surface mesh from tet mesh
    surface_filter = vtk.vtkDataSetSurfaceFilter()
    surface_filter.SetInputConnection(reader.GetOutputPort())
    surface_filter.Update()
    polydata = surface_filter.GetOutput()
    writer = vtk.vtkOBJWriter()
    tmp_obj = tempfile.NamedTemporaryFile(suffix='.obj', delete=True)
    writer.SetFileName(tmp_obj.name)
    writer.SetInputData(polydata)
    writer.Update()

    # recompute sdf value
    new_mesh = trimesh.load(tmp_obj.name, force='mesh')
    new_mesh.vertices = new_mesh.vertices * (2.0 / size) - 1.0  # normalize it to [-1, 1]
    sdf = mesh2sdf.core.compute(new_mesh.vertices, new_mesh.faces, size)
    return (sdf, new_mesh, tet_points) if return_mesh else sdf

Here is the code snippet for processing the PartNet dataset and generating interior point triplets. Hope it helps. Yes, we perform some preprocessing before tetrahedralization (see the code for details). In our implementation, the triplet points are sampled from the tetrahedral mesh vertices; we did not sample additional points from the interior of each tetrahedron.