ugrid reader performance

[cwsmith]

Serial ugrid mesh files often contain millions of mesh elements. If possible we should improve the performance of the ugrid reader. Attached is the stdout/stderr from the execution of from_ugrid on a 100M element mixed mesh and the corresponding gprof profile. Note, under gprof performance was ~4x slower than without it.

log.txt
profile.txt

[KennethEJansen]

Running this case again outside of gprof since that seems to slow things down by 4x here are some rough timings reading coordinates 2 minutes reading tetrahedra 25 minutes reading pyr and prism 2 minutes reading hexes 20 minutes mesh verified in 860.947017 seconds planned RIB factor 8 in 608.445305 seconds At this point it failed even though the machine had about 240G available. The estimate of 2x memory for splitting to 8 seems to be incorrect. We were able to get the reader to read and export a single part. Running split on the single part (1-8) ran out of memory. Splitting to 4 also failed. Splitting to 2 then 2 to 4 then 4 to 8 and 8 to 16 succeeded. Clearly, the dominate time is in reading connectivity. The mesh size will be about 1/2 a billion if tetrahedronized so it is not small but these element read times seem excessive. I suppose the verify time is also rather large and I think it might be advised to defer this to after a split so that this could be made more manageable (understanding this comes at some risk if for some reason the mesh as read is not valid but one could always reinstate the verify after reading in that case).

…

---------------------------------------------------------------- Kenneth E. Jansen, Professor Ann and H.J. Smead Department of Aerospace Engineering Sciences ECAE 161 OFFICE (303) 492-4359 429 UCB FAX (303) 492-4990 University of Colorado at Boulder jansenke@colorado.edu Boulder, CO, 80309-0429 http://www.colorado.edu/aerospace ----------------------------------------------------------------

On Feb 16, 2017, at 10:15 AM, Cameron Smith ***@***.***> wrote: Serial ugrid mesh files often contain millions of mesh elements. If possible we should improve the performance of the ugrid reader. Attached is the stdout/stderr from the execution of from_ugrid on a 100M element mixed mesh and the corresponding gprof profile. Note, under gprof performance was ~4x slower than without it. log.txt profile.txt — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[cwsmith]

Is the 100M element ugrid mesh available for download somewhere?