glTF parsing is *frightfully* slow

[SeanCurtis-TRI]

Problem statement

Importing the glTF file can dominate performance.

Quantify the problem

I'm working on a scenario trying to get the rendering as fast as possible.

I did some profiling, and found (for a given set of rendering configurations), that the majority of the server's time was spent in importing the glTF file. I took a bunch of measurements of the time the server spent handling a request. We'll see a bunch of tables as follows:

X seconds - Request handling over N measurements.
  X seconds - Blender operations over N measurements.
    X seconds - Scene init (reset, script, glTF load) over N measurements.
      X seconds - Scene reset over N measurements.
      X seconds - Load blend over N measurements.
      X seconds - Blender script over N measurements.
      X seconds - Load glTF over N measurements.
    X seconds - Scene prep over N measurements.
    X seconds - Render over N measurements.

• The times are average times. There are N instances taken (to be interpreted as N rendering requests).
• Measurements are hierarchical. For a reported time, any times indented below it represent a portion of that reported time.

In my particular scenario, I had a robot operating in an environment. For visualization reasons, the environment geometry is included in SceneGraph geometry. However, the environment geometry does not move. When it comes to rendering in drake-blender we have two choices:

1. Simply pass the SceneGraph geometry through to the server via the .gltf file, and make the .blend file bare bones.
2. Filter out all the environment geometries from SceneGraph and only include the robot model in the glTF file. The .blend file includes a blender version of all of the static environment geometries.

Probing those two cases, and measuring the server behavior we get the following:

11.864986 seconds - Request handling over 4 measurements.
  11.590297 seconds - Blender operations over 4 measurements.
    8.446637 seconds - Scene init (reset, script, glTF load) over 4 measurements.
      0.032361 seconds - Scene reset over 4 measurements.
      0.098421 seconds - Load blend over 4 measurements.
      0.011363 seconds - Blender script over 4 measurements.
      8.304475 seconds - Load glTF over 4 measurements.
    0.001303 seconds - Scene prep over 4 measurements.
    3.142343 seconds - Render over 4 measurements.

Table 1: Bare .blend file with all geometries transported via glTF -- glTF file approximately 256 Mb.

5.413307 seconds - Request handling over 57 measurements.
  5.357982 seconds - Blender operations over 57 measurements.
    2.173868 seconds - Scene init (reset, script, glTF load) over 57 measurements.
      0.037553 seconds - Scene reset over 57 measurements.
      0.234062 seconds - Load blend over 57 measurements.
      0.011327 seconds - Blender script over 57 measurements.
      1.890909 seconds - Load glTF over 57 measurements.
    0.000771 seconds - Scene prep over 57 measurements.
    3.183332 seconds - Render over 57 measurements.

Table 2: Populated .blend file with only robot geometries transported via glTF -- glTF file approximately 45 Mb.

Observations:

• Blender loads .blend files very quickly. 10 ms for empty file, 230 ms for massive file.
• Rendering time is indistinguishable.
• glTF import is crazy slow: 1.89 s for robot only, 8.3 s for robot and environment.
• I haven't explored the glTF file to determine if there is any part of it that is particularly expensive (i.e., number of nodes, number of triangles, amount of texture data, etc.) It's conceivable that not every byte comes with the same cost.
• I have explored using a .glb instead of a .gltf file. Unsurprisingly, loading a .glb file is faster, but the benefit is a constant 1 s chopped off the cost of the parse. So, 2 seconds become one second and 8 seconds become 7 s.
• The 3 second rendering time is a highly optimized rendering time -- before tweaking rendering, the render time was 23 seconds.

Possible solutions

Server caching

The basic idea is that the server maintains state from rendering to rendering and only updates poses.

Challenges:

• The server would need to be informed when a new scene comes in, and when a scene is an update of a previous load.
  • Possibly make use of the scene id we currently broadcast?
• In order to just update poses, it would have to extract node poses from the .gltf file.
  • It's not clear how fast a json parser would make that data available (compared to whatever it is blender is doing). Still to be tested.
  • Mapping may be a problem. Blender has very strict name uniqueness requirements. Every name is unique globally. SceneGraph has much weaker name requirements and glTF has none at all. The act of importing a glTF file can change the name silently. So, we'd need some form of robust data that would allow us to successfully identify nodes across glTF files w.r.t. arbitrary .blend files.

Multi-glTF API

More closely model the RenderEngine API in terms of "register geometry", "pose geometry", "remove geometry", and "render".

A geometry being registered would still be a glTF file. But it would also be accompanied by a unique client-owned identifier. The server would then map the identifier to whatever artifacts the glTF creates in the server. Thus when pose and geometry commands come with id information, the server can use its own maps to do whatever it needs to do.

Challenges:

• This is a completely different client-server API.
• It goes from processing a glTF to a host of glTFs (although, it is possible to embed the unique identifier within a glTF, so that we could put many geometries and their identifiers in a single glTF).