Improve playback performance

[ggarra13]

Playback perfornance on my 4 year-old PC is not as fast as tlRender on 4K movies. I tried it with Netflix's sample called SolLevante and it started slow and then literally froze.

[tedwaine]

Hi, thanks for this. It looks like there are a couple of encodings of SolLevante in the package, which one are you viewing?

Ted

[ggarra13]

SolLevante_HDR10_r2020_ST2084_UHD_24fps_1000nit.mov

[tedwaine]

Hi - I have a couple of questions please. What is your system spec (including GPU) & which OS are you running? Note that if you have NVidia hardware but not the appropriate NVidia graphics drivers xSTUDIO will run but it will perform badly as it relies heavily on hardware acceleration. We have not yet tested with AMD GPU cards.

If you loop on a small frame range in the source (use the 'I' and 'O' keys to set the loop range) - say 60 frames - do you get smooth playback? You should see a CPU load of about 80-100% in this case (once the frames in the loop range are decoded and cached) if things are working correctly.

[tedwaine]

If it's not the graphics acceleration that is a problem, you can change the number of decode threads in the ffmpeg reader (line 21 of ffmpeg_decoder.cpp) - this is currently hardcoded to 8 and that may not be enough to play this source, which is quite demanding. Soon we will put in some logic to increase the decode threads based on the given system. However, based on you reporting that xSTUDIO froze this doesn't sound like it's the problem.

[ggarra13]

Ubuntu 22.04 LTS
Intel Core i7 - 4790K CPU @ 4.00GHz x 8
NVIDIA GM206 [GeForce GTX 960]
8GB Memory

NVIDIA Driver Version: 525.60.11

[ggarra13]

I can't set an In and Out point as it gives me no chance. It freezes right at the beginning in the title sequence of Netflix Animation or whatever it says. It crashes afterwards.
I know it is a demanding clip --that's the point of my test. On my machine tlRender applications ( tlplay and mrViewer2 -mrv2 ) play back the sequence with audio at the beginning then after the video cache runs out they continue playing it realtime with no audio.

[ggarra13]

One thing that I should mention also is the audio. According to mrViewer 1, which allows changing audio tracks, the clip has two audio tracks one stereo and one 5:1. It isn't clear in tlRender or in xstudio what audio track it picks to play. Still, I think video is the main problem.

[darbyjohnston]

It isn't clear in tlRender or in xstudio what audio track it picks to play

The tlRender FFmpeg code uses the first track marked as default (AV_DISPOSITION_DEFAULT), or if there is no default track then the first track is used. This is for both video and audio tracks. It would also be nice to add an option to manually specify which track to use. There has been some discussion about adding this to OpenTimelineIO:
AcademySoftwareFoundation/OpenTimelineIO#145

[tedwaine]

Try this: run xstudio with no media at first. Click on the settings button (cog icon top left of viewer). Change your cache size to, say, 4GB. Then load the media. xSTUDIO does not have sophisticated (active?) memory management yet, and the default cache size is 8GB so perhaps the issue is that it is consuming all your RAM and going to swap. Granted, it shouldn't crash in that scenario either.
When I played that source I did get audio so xSTUDIO must be picking the stereo audio track (it can't handle 5.1 yet) but I'm not sure if it is applying correct logic as per tlRender.

[ggarra13]

Yes. It was the memory cache. Setting it to 4GB made it not crash or hang. It played really slow and with a lot of stuttering.

[tedwaine]

Thanks for this feedback. There is one playback optimisation we're working on at the moment that might make a big difference and give good performance with this encoding, I will get back to you when that's merged into the repo. We will also add a smarter cache size limit based on given system resources.

[alatdneg]

I've dropped the default image cache to 1Gig, this is more reasonable until we implement something a bit smarter.