Could PyTorch DataLoader flags improve worker-to-GPU throughput?

[bellegee2-create]

I would like to say Thank You for IW3. It is fantastic.
I'm using iw3 in a chunked/batched pipeline to create depth maps only.
Frames are pre-extracted as rgb48le TIF files by ffmpeg, then processed by calling iw3_main(args) in a loop with --batch-size 64 and --max-workers 16. Tiff Input is read from NVME.

My GPU telemetry shows a repeating pattern: 100% utilization during inference, then a 2-3 second dip to 60-70% between batches while workers prepare the next batch.
CPU is only 38-40% utilized during this — workers aren't saturated, but the GPU still starves between batches.
Would adding PyTorch pin_memory=True and prefetch_factor=2 to iw3's internal DataLoader help close this gap if you don't already?

This should benefit any iw3 input source. Pinned memory would allow direct DMA transfer to GPU without CPU-side copying, and pre-fetching would keep the next batch staged in RAM while the current one is processing.

[nagadomi]

iw3 does not use the PyTorch DataLoader for conversion processing.
When processing image folders, there is no guarantee that all images have the same size, so minibatches cannot be used. There is also no guarantee that the files are a sequence in filename order, so sequence based processing such as EMA is disabled.

Considering both the intended use and performance, it is better to process the video directly.

[bellegee2-create]

Thank you for the response. The IW3 Processing I use produces such marvelous depth maps that in 1080p the leaves have veins in the finished 3D. As part of my processing, I use a producer/consumer logic where the producer spawns the consumer, which initializes pytorch/iw3 and the model and waits. the producer then uses ffmpeg to take my input mkv and creates a sequential named, and equal sized TIFF file for each frame and writes it to a NVME drive. After a reasonable amount of tiffs exists, the consumer awakes and loops thru the folder of TIFF files with IW3 to create sequential depth-only png files. So basically things are reduced to single frames, sequentially processed. the EMA buffers are consistent until consumer exhausts the folder of TIFFs and rotates to the next folder, ping ponging between two folders that the producer fills with new TIFFs when consumed. IW3 is running and sustaining the model in the GPU, while consumer pushes the individual frames through.When the folder is empty, consumer will reinitalize the loop again for the next folder of TIFFs. IW3 must do some reinitialization of sorts, but i am unclear exactly, but the model seems to hold in memory so things stay fast. I do think the EMA buffers re-initialize. I doubt many are using IW3 the way I am. and I understand now this idea isn't pursuable. What I was looking for was a way that the 16 workers could utilize the prefetch and or pin the tiffs input in memory to keep the GPU fed more consistently. I have done what i can to eliminate I/O bottle necks so i was hoping the workers could utilize some GPU/pytorch tricks to speed things along. . If my processing explanation is understandable, another ask would be is there a way to have an --ema_buffer_persist flag that would allow them to stay in memory between iterations of the calls to iw3. I currently work around that by increasing the quantity of TIFF files between iterations that reduce the ema startup costs....

…

On Thu, May 21, 2026 at 8:04 AM nagadomi ***@***.***> wrote: iw3 does not use the PyTorch DataLoader for conversion processing. When processing image folders, there is no guarantee that all images have the same size, so minibatches cannot be used. There is also no guarantee that the files are a sequence in filename order, so sequence based processing such as EMA is disabled. Considering both the intended use and performance, it is better to process the video directly. — Reply to this email directly, view it on GitHub <#689 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/CBXQCRHIFWN5ODBSFKGEDEL433WDPAVCNFSM6AAAAACZHQYCMGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTOMBQGQ2DCOI> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[nagadomi]

The image loader already prefetches images in a separate thread. Image writing is submitted to a thread pool. In other words, the I/O overhead is already hidden. This applies when a folder is specified as the input.

If you are specifying individual image files as input, then of course this does not happen.

--ema-normalize is also not enabled when the input is not a video, for the reasons explained above. There is currently no export process that handles an image sequence as video.

[nagadomi]

I do not have motivation to improve the export feature, and I only have negative opinions about expanding it, so I will move it to maintenance mode.

I will add this to the README next month.

This is not because of this post.
I should have clearly stated from the beginning that I was not motivated to develop the export feature further.
I apologize for that.

[bellegee2-create]

No issues. the program is excellent. if you ever consider putting the rma persistence flag in your "tickle" file so if you are ever in the code and its easy...

…

On Fri, May 22, 2026 at 8:58 AM nagadomi ***@***.***> wrote: I do not have motivation to improve the export feature, and I only have negative opinions about expanding it, so I will move it to maintenance mode. I will add this to the README next month. This is not because of this post. I should have clearly stated from the beginning that I was not motivated to develop the export feature further. I apologize for that. — Reply to this email directly, view it on GitHub <#689 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/CBXQCRB7GREGGWN53JNCMPT44BFG3AVCNFSM6AAAAACZHQYCMGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTOMBRHE2DEMQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.***>