Migration to H264/MP4 for Hardware Acceleration

[lemeow125]

Hello!

I hope all's well on your end.

I have tried adjusting ffmpeg parameters in recording.conf to faciliate this but recording fails upon starting with the recorder object ending unexpectedly.

It's to my understanding that there might be a reason as to why a WebM/VP8 implementation was adopted in favor over H264/MP4, perhaps due to the virtual display libraries used? I'm not too sure on this myself honestly.

Is it possible to migrate from WebM/VP8 to a more traditional H264/MP4 format? I am keen on doing just that for better hardware acceleration support much like with Jellyfin and Frigate.

I am making do right now running the recording server on another device with the following ffmpeg config below, limiting CPU usage to 4 threads as to not max out processor usage.

[ffmpeg]
common = ffmpeg -loglevel level+warning -n -hwaccel vaapi -hwaccel_output_format vaapi
# outputaudio = -c:a libopus
outputvideo = -c:v libvpx -deadline:v realtime -crf 10 -b:v 1M -threads 4
extensionaudio = .ogg
extensionvideo = .webm

Server Specifications

• S1 (Running Nextcloud AIO)
  • CPU: Intel i5-10210U (8) @ 4.200GHz
  • GPU: Intel CometLake-U GT2 [UHD Graphics]
  • Memory: 64GB
• S2 (Recording server offloaded here)
  • CPU: AMD Ryzen 3 PRO 5350GE with Radeon Graphics (8) @ 3.600GHz
  • GPU: AMD ATI Radeon Vega Series / Radeon Vega Mobile Series
  • Memory: 64GB

Thank you for looking into this! Would appreciate any leads if any are possible.

[nickvergessen]

It's to my understanding that there might be a reason as to why a WebM/VP8 implementation was adopted in favor over H264/MP4, perhaps due to the virtual display libraries used?

As far as I recall, it is the format that is playing best on as many supported devices as possible.

[biguenique]

As far as I recall, it is the format that is playing best on as many supported devices as possible.

Also, VP8, Opus and the WebM container were explicitly developed as open source (that might explain a wider adoption). As far as I understand, codecs such as H264 and AAC rely on proprietary patented technology, so using them (and distributing them) for business purpose may somewhat put you in a grey area, licensing-wise. Correct me if I'm wrong.

The rationale, from the docs:

By default VP8 is used as the video codec. VP8 is an open and royalty-free video compression format widely supported. [...]
Similarly, Opus, another open codec, is used for audio.

[biguenique]

Tweaking (AIO)
@lemeow125 I've been trying to tweak the settings in /conf/recording.conf on the nextcloud-aio-talk-recording container (Nextcloud AIO). The problem is that the conf file is overwritten by the container's /start.sh, which is read-only. I'm stuck there for now. That makes it pretty obvious that I'm not a Docker expert at this point.

Standalone server?
My next step would be to set up a standalone recording server and tweaking the ffmpeg settings there. I'm just lacking time for trying this right now. Does your S2 server have a standalone recording server already set up?

But I surely wish the AIO container would provide an easier way to modify recording.conf. At the moment, recording a Talk session with the default settings on my AIO setup quickly overloads the full CPU power of its host, a VPS with 6 dedicated cores.

The former Talked solution was apparently able to achieve recording with much less resources:

Talked works by launching a Firefox instance in a virtual framebuffer, joining the Talk call, and then recording whatever is on screen using FFmpeg.
The default configuration has been designed to run on a 2 core / 2 GB ram VPS.

The Nextcloud Talk recording server is basically using the same stack as Talked was, so I suppose there would be a way to run it with much less core power.

Also, adding an option to record only the audio in Talk would enable call summaries while requiring much less resources, making it available on much humbler instances. But that's another issue, isn't?

[individ2016]

U can just mount /conf folder as any path to host system and add readonly rights to user 122. Then u can edit conf file as u wish.

I've done recording with hardware encoding with this params on my N100:

[ffmpeg]
common = ffmpeg -init_hw_device vaapi=intel:/dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device intel -filter_hw_device intel
outputvideo = -vf format=nv12,hwupload -c:v h264_vaapi -b:v 2M -flags +low_delay -pix_fmt vaapi
extensionvideo = .mp4

And one more.. to use vaapi u should build ur container with additional libs:

`FROM nextcloud/aio-talk-recording

USER root

RUN apk update &&
apk add --no-cache intel-media-driver

USER 122`

But there is a problem with loading file on server after recording that i didnt solve yet.

2025-06-01 11:09:17.598050+00:00INFO:nextcloud.talk.recording.Service.recorder-https://ncservice.******.name/-6v2evwvb:frame= 2708 fps= 30 q=-0.0 size= 6144kB time=00:01:30.45 bitrate= 556.4kbits/s speed= 1x 2025-06-01 11:09:18.089346+00:00INFO:nextcloud.talk.recording.Service.recorder-https://ncservice.******.name/-6v2evwvb:frame= 2723 fps= 30 q=-0.0 size= 6144kB time=00:01:30.93 bitrate= 553.5kbits/s speed= 1x 2025-06-01 11:09:18.143120+00:00INFO:nextcloud.talk.recording.Server:Stop recording: https://ncservice.******.name/ 6v2evwvb 2025-06-01 11:09:18.143747+00:00DEBUG:nextcloud.talk.recording.Service-https://ncservice.******.name/-6v2evwvb:Stopping recorder 2025-06-01 11:09:18.144708+00:00INFO:werkzeug:172.16.6.1 - - [01/Jun/2025 14:09:18] "POST /api/v1/room/6v2evwvb HTTP/1.1" 200 - 2025-06-01 11:09:18.170467+00:00INFO:nextcloud.talk.recording.Service.recorder-https://ncservice.******.name/-6v2evwvb:frame= 2738 fps= 30 q=-0.0 size= 6144kB time=00:01:31.47 bitrate= 550.2kbits/s speed= 1x 2025-06-01 11:09:18.170655+00:00INFO:nextcloud.talk.recording.Service.recorder-https://ncservice.******.name/-6v2evwvb:[out#0/mp4 @ 0x7fa7d3714180] video:5446kB audio:760kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.868606% 2025-06-01 11:09:18.170951+00:00INFO:nextcloud.talk.recording.Service.recorder-https://ncservice.******.name/-6v2evwvb:frame= 2743 fps= 30 q=-0.0 Lsize= 6322kB time=00:01:31.53 bitrate= 565.7kbits/s speed= 1x 2025-06-01 11:09:18.192235+00:00INFO:nextcloud.talk.recording.Service.recorder-https://ncservice.******.name/-6v2evwvb:Exiting normally, received signal 15. 2025-06-01 11:09:18.197301+00:00DEBUG:nextcloud.talk.recording.Service-https://ncservice.******.name/-6v2evwvb:Disconnecting from signaling server 2025-06-01 11:09:18.197392+00:00DEBUG:nextcloud.talk.recording.Service-https://ncservice.******.name/-6v2evwvb.BiDiLogsHelper:14:07:45 D 1 datachannel is open 2025-06-01 11:09:18.197533+00:00DEBUG:nextcloud.talk.recording.Service-https://ncservice.******.name/-6v2evwvb.BiDiLogsHelper:14:08:33 D Peer created 1sNDeNQez1rbMie4rBL5OLi2kB07e3Sa6B_2JFj7HEt8MXhfQkJKTlZDdXpNWWFwNWQtYXN2blZEOGlsVy12cGo4eGtQZWx5NmFZU3BSMk1lcWg2V1RNVnB8MDg1NDY3ODQ3MQ== [object Object] 2025-06-01 11:09:18.197575+00:00DEBUG:selenium.webdriver.remote.remote_connection:POST http://localhost:34139/session/953c722b-10b7-4f8b-b75f-32308d62abd7/execute/sync {'script': '\n OCA.Talk.signalingKill()\n ', 'args': []} 2025-06-01 11:09:18.207833+00:00DEBUG:urllib3.connectionpool:http://localhost:34139 "POST /session/953c722b-10b7-4f8b-b75f-32308d62abd7/execute/sync HTTP/1.1" 200 0 2025-06-01 11:09:18.208470+00:00DEBUG:selenium.webdriver.remote.remote_connection:Remote response: status=200 | data={"value":null} | headers=HTTPHeaderDict({'content-type': 'application/json; charset=utf-8', 'cache-control': 'no-cache', 'content-length': '14', 'date': 'Sun, 01 Jun 2025 11:09:18 GMT'}) 2025-06-01 11:09:18.208536+00:00DEBUG:selenium.webdriver.remote.remote_connection:Finished Request 2025-06-01 11:09:18.211686+00:00DEBUG:nextcloud.talk.recording.Service-https://ncservice.******.name/-6v2evwvb.BiDiLogsHelper:14:09:18 D Close [object CloseEvent] 2025-06-01 11:09:18.709680+00:00DEBUG:selenium.webdriver.remote.remote_connection:DELETE http://localhost:34139/session/953c722b-10b7-4f8b-b75f-32308d62abd7 {} 2025-06-01 11:09:18.719990+00:00DEBUG:nextcloud.talk.recording.Service-https://ncservice.******.name/-6v2evwvb.BiDiLogsHelper:BiDi WebSocket closed 2025-06-01 11:09:19.454893+00:00DEBUG:urllib3.connectionpool:http://localhost:34139 "DELETE /session/953c722b-10b7-4f8b-b75f-32308d62abd7 HTTP/1.1" 200 0 2025-06-01 11:09:19.454934+00:00DEBUG:selenium.webdriver.remote.remote_connection:Remote response: status=200 | data={"value":null} | headers=HTTPHeaderDict({'content-type': 'application/json; charset=utf-8', 'cache-control': 'no-cache', 'content-length': '14', 'date': 'Sun, 01 Jun 2025 11:09:18 GMT'}) 2025-06-01 11:09:19.455007+00:00DEBUG:selenium.webdriver.remote.remote_connection:Finished Request 2025-06-01 11:09:19.471340+00:00DEBUG:nextcloud.talk.recording.Service-https://ncservice.******.name/-6v2evwvb:Unloading audio module 2025-06-01 11:09:19.473341+00:00DEBUG:nextcloud.talk.recording.Service-https://ncservice.******.name/-6v2evwvb:Stopping display 2025-06-01 11:09:19.476135+00:00DEBUG:pyvirtualdisplay.abstractdisplay:stdout=b'' 2025-06-01 11:09:19.476192+00:00DEBUG:pyvirtualdisplay.abstractdisplay:stderr=b'_XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.\n' 2025-06-01 11:09:19.477482+00:00DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): ncservice.******.name:443 2025-06-01 11:09:19.727780+00:00DEBUG:urllib3.connectionpool:https://ncservice.******.name:443 "POST /ocs/v2.php/apps/spreed/api/v1/recording/backend HTTP/1.1" 200 116 2025-06-01 11:09:19.729312+00:00INFO:nextcloud.talk.recording.BackendNotifier:Upload recording /tmp/httpsncservice******name/6v2evwvb/Recording 2025-06-01 14-07-45.mp4 to https://ncservice.******.name/ in 6v2evwvb as Individ 2025-06-01 11:09:19.730544+00:00DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): ncservice.******.name:443 2025-06-01 11:09:19.890317+00:00DEBUG:urllib3.connectionpool:https://ncservice.******.name:443 "POST /ocs/v2.php/apps/spreed/api/v1/recording/6v2evwvb/store HTTP/1.1" 400 180 2025-06-01 11:09:19.892934+00:00ERROR:nextcloud.talk.recording.BackendNotifier:Failed to send message to backend, 3 retries left! 2025-06-01 11:09:19.892983+00:00Traceback (most recent call last): 2025-06-01 11:09:19.892991+00:00File "/usr/local/lib/python3.13/site-packages/nextcloud/talk/recording/BackendNotifier.py", line 72, in doRequest 2025-06-01 11:09:19.892998+00:00response.raise_for_status() 2025-06-01 11:09:19.893007+00:00~~~~~~~~~~~~~~~~~~~~~~~~~^^ 2025-06-01 11:09:19.893036+00:00File "/usr/local/lib/python3.13/site-packages/requests/models.py", line 1024, in raise_for_status 2025-06-01 11:09:19.893047+00:00raise HTTPError(http_error_msg, response=self) 2025-06-01 11:09:19.893056+00:00requests.exceptions.HTTPError: 400 Client Error: Bad request for url: https://ncservice.******.name/ocs/v2.php/apps/spreed/api/v1/recording/6v2evwvb/store

There is more problem with hardware accel.. So that x11 totally on cpu, but i donno if i can use something like this:

common = ffmpeg -device /dev/dri/card0 -f kmsgrab -i -
outputvideo = -vf 'hwmap=derive_device=vaapi,scale_vaapi=w=1920:h=1080:format=nv12' -c:v h264_vaapi

bcoz of error in result:

ffmpeg version 6.1.2 Copyright (c) 2000-2024 the FFmpeg developers
built with gcc 14.2.0 (Alpine 14.2.0)
configuration: --prefix=/usr --disable-librtmp --disable-lzma --disable-static --disable-stripping --enable-avfilter --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libmp3lame --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librist --enable-libsoxr --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-lto=auto --enable-lv2 --enable-openssl --enable-pic --enable-postproc --enable-pthreads --enable-shared --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --optflags=-O3 --enable-libjxl --enable-libsvtav1 --enable-libvpl
libavutil 58. 29.100 / 58. 29.100
libavcodec 60. 31.102 / 60. 31.102
libavformat 60. 16.100 / 60. 16.100
libavdevice 60. 3.100 / 60. 3.100
libavfilter 9. 12.100 / 9. 12.100
libswscale 7. 5.100 / 7. 5.100
libswresample 4. 12.100 / 4. 12.100
libpostproc 57. 3.100 / 57. 3.100
[kmsgrab @ 0x7f775843e600] No usable planes found.
[in#0 @ 0x7f7753f7f940] Error opening input: Invalid argument
Error opening input file -.
Error opening input files: Invalid argument

Help me if someone know, how to deal with it

[dhatkovsky]

has anyone tried qsv? i have succeeded in vaapi and it dropped load ca. 4 times but i own n305 and it can do more with qsv. i have built ffmpeg for qsv and can convert manually inside the recording container but it does not work for the nextcloud itself. also -report option seems doesn't work...

[serfriz]

I have the same question. Has anyone been able to make it work with QuickSync hardware transcoding? I'd like to offload my CPU, it really gets hit hard with recordings.