How to save rendered mesh model?

[xuduo18311199384]

Hi. I really appreciate your nice work. But when i use save_to_file() to save a .ply, i find .ply file do not have RGB color. How can i save rendered mesh model as "rendered model view" in the following picture?

[xuduo18311199384]

@RaduAlexandru

[RaduAlexandru]

Hi @xuduo18311199384

The color renders in the teaser figure are done using sphere tracing which you can find here

I don't have an easy way to transfer the color to a texture.
You could try unwrap the mesh to a uv map and optimize a texture so that it matches all the input images.
This would be similar to Deferred Neural rendering
There is a bit of functionality for it here but I currently can't offer much support in that aspect.

[xuduo18311199384]

Thanks for your reply! @RaduAlexandru
How i can save a colored model in .ply with run_net_sphere_traced()?

[Zvyozdo4ka]

Hi. I really appreciate your nice work. But when i use save_to_file() to save a .ply, i find .ply file do not have RGB color. How can i save rendered mesh model as "rendered model view" in the following picture?

where did you get input data for this case?

could you share please, how did you save file? did you run create_my_meshes.py?

[xuduo18311199384]

Hi. I really appreciate your nice work. But when i use save_to_file() to save a .ply, i find .ply file do not have RGB color. How can i save rendered mesh model as "rendered model view" in the following picture?

where did you get input data for this case?

could you share please, how did you save file? did you run create_my_meshes.py?

from permuto_sdf_py.utils.sdf_utils import extract_mesh_from_sdf_model
extracted_mesh=extract_mesh_from_sdf_model(model_sdf, 700, min_val=-0.5, max_val=0.5)
extracted_mesh.save_to_file(os.path.join("/workspace/", phase.name+".ply") )

[Zvyozdo4ka]

from permuto_sdf_py.utils.sdf_utils import extract_mesh_from_sdf_model extracted_mesh=extract_mesh_from_sdf_model(model_sdf, 700, min_val=-0.5, max_val=0.5) extracted_mesh.save_to_file(os.path.join("/workspace/", phase.name+".ply") )

I appreciate it. As model sdf should i give "input sdf_model.pt"?

[Zvyozdo4ka]

where are the trained results?
workspace/permuto_sdf/checkpoints/permuto_sdf_dtu_scan55_default/ ?

[xuduo18311199384]

from permuto_sdf_py.utils.sdf_utils import extract_mesh_from_sdf_model extracted_mesh=extract_mesh_from_sdf_model(model_sdf, 700, min_val=-0.5, max_val=0.5) extracted_mesh.save_to_file(os.path.join("/workspace/", phase.name+".ply") )

I appreciate it. As model sdf should i give "input sdf_model.pt"?

model_sdf=SDF(in_channels=3, boundary_primitive=aabb, geom_feat_size_out=hyperparams.sdf_geom_feat_size, nr_iters_for_c2f=hyperparams.sdf_nr_iters_for_c2f).to("cuda")

[xuduo18311199384]

where are the trained results? workspace/permuto_sdf/checkpoints/permuto_sdf_dtu_scan55_default/ ?

I use my custom datasets to train and do not know dtu datasets' result.

[Zvyozdo4ka]

What should be inside cameras_large.npz and cameras_sphere.npz?

in paper who provides DTU data there is no mention about cameras_large.npz and cameras_sphere.npz?

They even don't mention in permuto paper.

I think all of you have some magic abilities to read mind of each other to get necessary information

[Zvyozdo4ka]

where are the trained results? workspace/permuto_sdf/checkpoints/permuto_sdf_dtu_scan55_default/ ?

I use my custom datasets to train and do not know dtu datasets' result.

Could share your experience more detailed. I did not understand anything. How did you get cameras_large.npz and cameras_sphere.npz?

And what is this?

model_sdf=SDF(in_channels=3, boundary_primitive=aabb, geom_feat_size_out=hyperparams.sdf_geom_feat_size, nr_iters_for_c2f=hyperparams.sdf_nr_iters_for_c2f).to("cuda")

[RaduAlexandru]

Hi @Zvyozdo4ka and @xuduo18311199384 ,

1. The cameras_large.npz and cameras_sphere.npz are part of the DTU dataset which is linked in the readme: link
2. In order to save image like the head image please check this script which is used to generate images. It boils down to creating a frame object which contains your camera intrinsics and extrinsics and then rendering from that camera pose using run_net_in_chunks.