VOD transcoding: Video is smooth, audio may have clicks at segments?

[stevyn81]

Hello,

I have been enjoying setting up and testing the impressive go-transcode software. I am testing the VOD feature with a .MP4 file transcoding down to a smaller size and the video looks smooth and great, but with headphones on and listening very carefully I can sometimes hear a click. Monitoring with the browser "inspect" I see the click matches with the segment transition.

I am hesitant to post this as perhaps it is my computer having trouble. I have tested with the video-keyframes: false or true and seems to happen either way.

If I stream the .mp4 as a normal static file, no transcoding, the video + audio is smooth.

Searching online I find a few others researching the audio click issues.. example: https://stackoverflow.com/questions/58898638/precise-method-of-segmenting-transcoding-videoaudio-via-ffmpeg-into-an-on

Can anyone confirm this audio issue or is it just me? Thanks much for your work!

[m1k1o]

From: https://stackoverflow.com/questions/58898638/precise-method-of-segmenting-transcoding-videoaudio-via-ffmpeg-into-an-on

I think this may be an issue with splitting using time in ffmpeg, where possibly the audio stream cannot be accurately sliced at the exact frame the video is.

This points again to the old known issue that was believed to be fixed but resurfaces again #38

So far, I've been unable to figure out a solution to this problem.

[stevyn81]

Hello, I have been looking at the transcoder code in a "plex like alternative" project called Kyoo, and it appears they are transcoding the audio separately from the video and then serve up both. https://github.com/zoriya/Kyoo/tree/master/transcoder

[m1k1o]

@stevyn81 thank you for the suggestion. I am taking a look at their implementation, and I see that they pregenerate 100 segments in advance, what is almost 20x more than what we do here by default. Meaning, even if the error would be there it would be less visible.

But the idea of having audio separate to video is a good one, as it allows having multiple audio tracks and switch between them seamlessly. I think we should move to that concept eventually.

[m1k1o]

Further tests revealed, that alone having audio and video separate does not fix the problem. The author states, however, that they were able to fix this by producing one more extra segment at the begining of the sequence and discarding it. This offset should fix the issue, I'll perform more tests.

[dmalapsh]

I've probably found a solution to this problem. The fact is that the aac codec consists of frames. If it 's the end time .ts does not match the end of the audio frame, then the remainder will be filled with an empty signal. Here is a translation of an article detailing a similar task: https://codemore-ru.translate.goog/2021/05/01/video-streaming.html?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=ru&_x_tr_pto=wapp

[m1k1o]

@dmalapsh thank you for the link. That explains in detail why starting the transcoding one segment earlier in ffmpeg should fix the issue.