tetranerf.md

Tetra-NeRF

Tetra-NeRF: Representing Neural Radiance Fields Using Tetrahedra

:color: primary
:outline:
Paper Website

:color: primary
:outline:
Code

SfM input pointcloud is triangulated and resulting tetrahedra is used as the radiance field representation

Installation

First, make sure to install the following:

CUDA (>=11.3)
PyTorch (>=1.12.1)
Nerfstudio (>=0.2.0)
OptiX (>=7.2, preferably 7.6)
CGAL
CMake (>=3.22.1)

Follow the installation section in the tetra-nerf repository

Finally, you can install Tetra-NeRF by running:

pip install git+https://github.com/jkulhanek/tetra-nerf

Running Tetra-NeRF on custom data

Details for running Tetra-NeRF can be found here.

python -m tetranerf.scripts.process_images --path <data folder>
python -m tetranerf.scripts.triangulate --pointcloud <data folder>/sparse.ply --output <data folder>/sparse.th
ns-train tetra-nerf --pipeline.model.tetrahedra-path <data folder>/sparse.th minimal-parser --data <data folder>

Three following variants of Tetra-NeRF are provided:

Method	Description	Memory	Quality
tetra-nerf-original	Official implementation from the paper	~18GB*	Good
tetra-nerf	Different sampler - faster and better	~16GB*	Best

*Depends on the size of the input pointcloud, estimate is given for a larger scene (1M points)

Method

The input to Tetra-NeRF is a point cloud which is triangulated to get a set of tetrahedra used to represent the radiance field. Rays are sampled, and the field is queried. The barycentric interpolation is used to interpolate tetrahedra vertices, and the resulting features are passed to a shallow MLP to get the density and colours for volumetric rendering.

Existing checkpoints and predictions

For an easier comparisons with Tetra-NeRF, published checkpoints and predictions can be used:

Dataset	Checkpoints	Predictions	Input tetrahedra
Mip-NeRF 360 (public scenes)	download	download	download
Blender	download	download	download
Tanks and Temples	download	download	download