NN Extraction For Conversion and use in Other Formats

[Davicam2]

Outstanding project scientists!

This project achieves wonderful results Orders of magnitudes faster than anything else out there.

I would like to take the trained model, and apply its use in other engines/projects, but am not sure how to extract the NN. the .msgpack as far as i can tell is built for the test bed, and i cant seem to find the trained NN being stored anywhere.

Please let me know if this is a future development or if i can manipulate the test with python to extract a trained model!

[collinabidi]

^ I'm interested in this as well.

I assume that we will have to use the tiny-cuda-nn package like this, but we don't have a .json.

[Tom94]

Hi there, neither instant-ngp now tiny-cuda-nn directly support exporting models in portable formats.

That said, you can replicate the computation graph from NerfNetwork in your favorite machine learning framework and obtain the raw trained parameters either by

• reading the MessagePack file (which is essentially a binary json-equivalent that has implementations in all kinds of languages) and extracting the snapshot.params_binary field (can be fp16 or fp32, depending on config / machine), or
• by calling m_trainer->params() within Testbed to get pointer to GPU memory holding the trained parameters in fp32 format.

NerfNetwork::set_params and callees showcase how the parameters are laid out. Note that the underlying MLP implementations CutlassMLP and FullyFusedMLP apply different amounts of padding to the last layer (8 vs. 16).

Cheers!