VPU not used with cog

[gnulux]

​Hi,

I'm trying to play a mp4 file from cog which plays slow whereas it runs well with gst-play .
I'm wondering if cog uses the VPU as expected.
I'm running on IMX8, on Yocto with Webkit compiled with gst_gl . I tried with --media-content-types-requiring-hardware-support but nothing better

GST_DEBUG="3,webkit*:6" GST_DEBUG_FILE=/tmp/gst.log GST_DEBUG_DUMP_DOT_DIR=/tmp/gst cog -P wl --media-content-types-requiring-hardware-support='video/webm; codecs="vp":video/mp4; codecs="avc":video/* codecs="av1*"' https://people.igalia.com/clopez/wkbug/video/big_buck_bunny.mp4
debug.zip

What did i missed​ ?

[philn]

The pipeline dump shows that avdec_h264 is used for decoding, that's not hw-accelerated. Can you dump pipelines for gst-play too and check which decoder is used?

[philn]

which version of wpewebkit is this btw?

i'd advise to try 2.42, which enables a new video sink by default. Leveraging dmabufs. I don't think it's been tested much on iMX though...

[gnulux]

Hi,
wpewebkit version is 2.34.6-r0 (available for Yocto hardknott build)
here are gst-play dump with GST_DEBUG=4
gst-play.log

[philn]

avdec_h264 also used by gst-play it seems...

[gnulux]

This is weird because i see the difference on my proc when playing the video from cog, wpeWebProcess consumes 120%
which is not the case with gst-play .
here i provide more logs from gst-play :
gst-play_5.log.gz

[gnulux]

Ok , surely something with the linux user account.
The video play well on cog with root account
any idea ? perhaps root reserved memory zone or something else?

[philn]

Yeah maybe the iMX device driver requires root...

[aperezdc]

Maybe it would be possible to set the permissions on the device node (either using udev rules, or by hand using chmod u+rw /dev/foo for testing).

[gnulux]

I moved forward on this topic ,
Now gst-play big_buck_bunny.mp4 works well from non root user. But it still does not work from cog .
following this log it seems that the hardware is now enabled for video/x-h264.
webkitregistryscanner GStreamerRegistryScanner.cpp:179:hasElementForMediaType: Lookup result for video parser matching caps video/x-h264 : isSupported=true, isUsingHardware=false
Is it possible to force using hardware ?
is it possible by setting parameter --media-content-types-requiring-hardware-support= ?

Cdlt

[gnulux]

At the end root account use V4L2DEC

`====== AIUR: 4.6.1 build on May 11 2021 03:19:55. ======
Core: MPEG4PARSER_06.17.18 build on Apr 14 2021 02:39:48
file: /usr/lib/imx-mm/parser/lib_mp4_parser_arm_elinux.so.3.2

Track 00 [audio_0] Enabled
Duration: 0:01:00.139682000
Language: eng
Mime:
audio/mpeg, mpegversion=(int)4, channels=(int)2, rate=(int)22050, bitrate=(int)64000, stream-format=(string)raw, codec_data=(buffer)1390 

---

====== BEEP: 4.6.1 build on May 11 2021 03:19:55. ======
Core: AAC decoder Wrapper build on Jul 28 2020 10:45:34
file: /usr/lib/imx-mm/audio-codec/wrap/lib_aacd_wrap_arm_elinux.so.3
CODEC: BLN_MAD-MMCODECS_AACD_ARM_03.09.00_ARMV8 build on Sep 20 2017 15:02:50.

Track 01 [video_0] Enabled
Duration: 0:01:00.095000000
Language: eng
Mime:
video/x-h264, parsed=(boolean)true, alignment=(string)au, stream-format=(string)avc, width=(int)640, height=(int)360, framerate=(fraction)24/1, codec_data=(buffer)0142e01eff0100152742e01ea9181405ff2e00d418041adb0ad7bdf01001000428de09c8 

---

Track 02 [subtitle]: Disabled
Codec: 0, SubCodec: 0

---

Track 03 [subtitle]: Disabled
Codec: 0, SubCodec: 0

---

====== V4L2DEC: 1.18.0 build on Jun 3 2021 08:59:40. ======`

[philn]

Is perf with v4l2dec in wpewebkit ok or not?

[gnulux]

Still ok with root but not with normal account. Le ven. 27 oct. 2023, 21:21, Philippe Normand ***@***.***> a écrit :

…

Is perf with v4l2dec in wpewebkit ok or not? — Reply to this email directly, view it on GitHub <#631 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABLDWDF5DB4ICB3WNGW65IDYBQCSDAVCNFSM6AAAAAA6RQDYTOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBTGM4TMMJTGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[philn]

I don't think we can do much about that. Maybe ask on NXP/Freescale forums :)

[gnulux]

Yes understand, Thks again Le ven. 27 oct. 2023, 22:17, Philippe Normand ***@***.***> a écrit :

…

I don't think we can do much about that. Maybe ask on NXP/Freescale forums :) — Reply to this email directly, view it on GitHub <#631 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABLDWDHCNCQAUCF6KL2TMUTYBQJHDAVCNFSM6AAAAAA6RQDYTOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBTGQ2TQMBSGI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[gnulux]

Hi Adrian , Of course, I tried to do that on /dev/video0 /dev/video1 . But still the same. ls -al /dev/v4l/by-path/ total 0 drwxr-xr-x 2 root root 140 Oct 31 13:04 . drwxr-xr-x 3 root root 60 Oct 31 13:04 .. lrwxrwxrwx 1 root root 12 Oct 31 13:04 platform-32c00000.bus:camera-video-index0 -> ../../video3 lrwxrwxrwx 1 root root 12 Oct 31 13:04 platform-32c00000.bus:camera-video-index1 -> ../../video4 lrwxrwxrwx 1 root root 12 Oct 31 13:04 platform-32e00000.isi:m2m_device-video-index0 -> ../../video2 lrwxrwxrwx 1 root root 12 Oct 31 13:04 platform-vpu_v4l2-video-index0 -> ../../video0 lrwxrwxrwx 1 root root 12 Oct 31 13:04 platform-vpu_v4l2-video-index1 -> ../../video1 rgds Le mar. 31 oct. 2023 à 14:42, Adrian Perez ***@***.***> a écrit :

…

Maybe it would be possible to set the permissions on the device node (either using udev rules, or by hand using chmod u+rw /dev/foo for testing). — Reply to this email directly, view it on GitHub <#639 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABLDWDBIKK4M2D4J7QFSBNLYCD54BAVCNFSM6AAAAAA6XW75ICVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TIMZVGQ4DE> . You are receiving this because you authored the thread.Message ID: ***@***.***>