Linux v4l2 Camera troubles in X11 w/ SDL_PIXELFORMAT_YUY2

[Simulacrum0]

SDL3 has an excellent Camera API, but for the sample and our own implementation of using it, it can see any of our 4 test cameras on Ubuntu 23.10 using X11 (Nvidia GPU driver) via the "v4l2" driver. Have tried 1 camera at a time or many. All report name strings, all formats, and 'CAMERA_APPROVAL' and CAMERA_ADDED events as expected.

If i SDL_OpenCameraDevice w/ SDL_PIXELFORMAT_YUY2, which is the ONLY format provided by all cameras (Logitech, CreativeLabs, Generic ), then "AcquireFrame" only returns a 'WHITE' surface and it hangs in WaitForThread inside of SDL_Camera Thread inside a call to v4l2, which gdb does not name.

If i open the device w/ SDL_PIXELFORMAT_RGBA8888, it consistently acquires frame and closes without hanging.

Have tried builds with SDL_TIMER_INIT on/off, with OpenCameraDevice after CAMERA_ADDED event or at SDL startup, and have polled at 1fps, 5fps, 30fps and unlimited loop of polling.

Am posting this in case helpful.
#love_SDL3!

[Simulacrum0]

Also found that SDL_CameraSpec seems to match the interval numerator/denominator (refresh-rate) 'near' the requested values but it does not report the actual value chosen ( it returns whatever you put into OpenCameraDevice as a request...from denominators of 2 to 144, its just copied over? ). Posting if helpful?

[icculus]

It shouldn't just be copied over, it should be finding the closest values the camera reported as supported. So it's probably an SDL bug. I'll dig in.

[Simulacrum0]

Thanks Icculus!
I can test or help if useful.

#SDL3_Rocks!

[Kaktus514]

If i SDL_OpenCameraDevice w/ SDL_PIXELFORMAT_YUY2, which is the ONLY format provided by all cameras (Logitech, CreativeLabs, Generic ), then "AcquireFrame" only returns a 'WHITE' surface

When passing NULL to SDL_OpenCameraDevice I also experience a similar issue but only if I use SDL_BlitSurface to draw to the "window surface". If I instead convert the surface to a texture and use the render api then it displays correctly so I wonder if this is not just an effect of SDL_BlitSurface not supporting the YUY2 format?

[slouken]

I believe that's correct. Are you checking SDL_BlitSurface() for errors?

[Green-Sky]

it hangs in WaitForThread inside of SDL_Camera Thread inside a call to v4l2, which gdb does not name.

I ran into the same issue when I was reading the frames slower than they where provided (according to camera spec). It went away when acquiring and releasing frames until there are non left directly before closing the camera, so I suspect it locks up waiting for all existing frames to be read.

[Green-Sky]

I suspect the YUY2 frame is not fully created, because I sometimes (every ~100th) hit a heap-buffer-overlow when blitting from the aquired frame:

### found cameras:
  - Camera #0: Chicony USB2.0 Camera: Chicony
    - 1280x720@10 SDL_PIXELFORMAT_YUY2
    - 640x480@30 SDL_PIXELFORMAT_YUY2
    - 640x360@30 SDL_PIXELFORMAT_YUY2
    - 352x288@30 SDL_PIXELFORMAT_YUY2
    - 320x240@30 SDL_PIXELFORMAT_YUY2
    - 176x144@30 SDL_PIXELFORMAT_YUY2
    - 160x120@30 SDL_PIXELFORMAT_YUY2
camera interval: 100ms
camera format: SDL_PIXELFORMAT_YUY2
video frame was 1280x720 1367496316ns
video frame was 1280x720 1467500316ns
video frame was 1280x720 1567504316ns
video frame was 1280x720 1667506316ns
video frame was 1280x720 1763510316ns
video frame was 1280x720 1863513316ns
video frame was 1280x720 1963517316ns
=================================================================
==1484947==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7fcf019d1000 at pc 0x000001d9d8c9 bp 0x7fcf003bdbc0 sp 0x7fcf003bdbb8
READ of size 4 at 0x7fcf019d1000 thread T2
    #0 0x1d9d8c8 in ReadFloatPixel /home/green/workspace/tox/tomato/build/_deps/sdl3-src/src/video/SDL_blit_slow.c:383
    #1 0x1db350f in SDL_Blit_Slow_Float /home/green/workspace/tox/tomato/build/_deps/sdl3-src/src/video/SDL_blit_slow.c:841
    #2 0x1ae06f0 in SDL_SoftBlit /home/green/workspace/tox/tomato/build/_deps/sdl3-src/src/video/SDL_blit.c:86
    #3 0x17f1503 in SDL_BlitSurface_REAL /home/green/workspace/tox/tomato/build/_deps/sdl3-src/src/video/SDL_surface.c:845
    #4 0xfe2bdc in operator() /home/green/workspace/tox/tomato/src/content/sdl_video_frame_stream2.cpp:81


0x7fcf019d1000 is located 6144 bytes to the left of 1228872-byte region [0x7fcf019d2800,0x7fcf01afe848)
allocated by thread T0 here:
    #0 0x7fcf04cbc3ff in __interceptor_malloc (/nix/store/a3zlvnswi1p8cg7i9w4lpnvaankc7dxx-gcc-12.3.0-lib/lib/libasan.so.8+0xbc3ff)
    #1 0x17aa1c2 in SDL_malloc_REAL /home/green/workspace/tox/tomato/build/_deps/sdl3-src/src/stdlib/SDL_malloc.c:5287

Thread T2 created by T0 here:
    #0 0x7fcf04c4d136 in __interceptor_pthread_create (/nix/store/a3zlvnswi1p8cg7i9w4lpnvaankc7dxx-gcc-12.3.0-lib/lib/libasan.so.8+0x4d136)
    #1 0x7fcf044e0698 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/nix/store/a3zlvnswi1p8cg7i9w4lpnvaankc7dxx-gcc-12.3.0-lib/lib/libstdc++.so.6+0xe0698)

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/green/workspace/tox/tomato/build/_deps/sdl3-src/src/video/SDL_blit_slow.c:383 in ReadFloatPixel

I also know from OBS, that the camera has some funky jpeg stream that supports higher fps at 720p.

(whole application compiled with asan and sdl staticly linked, log shortened)

[Green-Sky]

I turned on DEBUG_CAMERA:

startup

INFO: CAMERA: V4L2 camera path='/dev/video0' bus_info='usb-0000:09:00.3-4' name='Chicony USB2.0 Camera: Chicony '
INFO: CAMERA:   - Has format 'SDL_PIXELFORMAT_UNKNOWN' [COMPRESSED]
INFO: CAMERA:   - Has format 'SDL_PIXELFORMAT_YUY2'
INFO: CAMERA:     * Has discrete size 1280x720
INFO: CAMERA:       * Has discrete frame interval (1 / 10), fps=10.000000
INFO: CAMERA:       * Has discrete frame interval (1 / 10), fps=10.000000
INFO: CAMERA:     * Has discrete size 640x480
INFO: CAMERA:       * Has discrete frame interval (1 / 30), fps=30.000000
INFO: CAMERA:     * Has discrete size 640x360
INFO: CAMERA:       * Has discrete frame interval (1 / 30), fps=30.000000
INFO: CAMERA:     * Has discrete size 352x288
INFO: CAMERA:       * Has discrete frame interval (1 / 30), fps=30.000000
INFO: CAMERA:     * Has discrete size 320x240
INFO: CAMERA:       * Has discrete frame interval (1 / 30), fps=30.000000
INFO: CAMERA:     * Has discrete size 176x144
INFO: CAMERA:       * Has discrete frame interval (1 / 30), fps=30.000000
INFO: CAMERA:     * Has discrete size 160x120
INFO: CAMERA:       * Has discrete frame interval (1 / 30), fps=30.000000
INFO: CAMERA:     * Has discrete size 1280x720
INFO: CAMERA:       * Has discrete frame interval (1 / 10), fps=10.000000
INFO: CAMERA:       * Has discrete frame interval (1 / 10), fps=10.000000
INFO: CAMERA: (total specs: 10)
INFO: CAMERA: Adding device 'Chicony USB2.0 Camera: Chicony ' (unknown position) with 7 specs:
INFO: CAMERA:   - fmt=SDL_PIXELFORMAT_YUY2, w=1280, h=720, numerator=1, denominator=10
INFO: CAMERA:   - fmt=SDL_PIXELFORMAT_YUY2, w=640, h=480, numerator=1, denominator=30
INFO: CAMERA:   - fmt=SDL_PIXELFORMAT_YUY2, w=640, h=360, numerator=1, denominator=30
INFO: CAMERA:   - fmt=SDL_PIXELFORMAT_YUY2, w=352, h=288, numerator=1, denominator=30
INFO: CAMERA:   - fmt=SDL_PIXELFORMAT_YUY2, w=320, h=240, numerator=1, denominator=30
INFO: CAMERA:   - fmt=SDL_PIXELFORMAT_YUY2, w=176, h=144, numerator=1, denominator=30
INFO: CAMERA:   - fmt=SDL_PIXELFORMAT_YUY2, w=160, h=120, numerator=1, denominator=30
INFO: CAMERA: App wanted [(-1x-1) fmt=(null) interval=-1/-1], chose [(1280x720) fmt=SDL_PIXELFORMAT_YUY2 interval=1/10]
INFO: CAMERA: set SDL format SDL_PIXELFORMAT_YUY2
INFO: CAMERA: set format V4L2_format=1448695129  YUYV

a frame

INFO: CAMERA: debug mmap: image 3/8  data[0]=0x7f56f01a2000
INFO: CAMERA: New frame available! pixels=0x7f56f01a2000 pitch=2560
INFO: CAMERA: Frame is going through without conversion!
video frame was 1280x720 4260183097ns

[BlockBuilder57]

Running into the same issues on Arch, on an AMD RX 5700. Both a capture card and the Droidcam source have the same behavior. The example program does manage to open the camera, but never can acquire a frame, it just stalls.

[Green-Sky]

@BlockBuilder57 for me this was solved by #10881 . However, the issue described here is somewhat different and might still apply. How recent is your SDL3 ?

[BlockBuilder57]

Latest sdl3 package on the AUR, so SDL3-preview-3.1.6 @ fe3566c. Definitely includes #10881, so it seems this is a completely different issue. I've opened #11473 if you want to take a look.