VIDIO_ENUM_FRAMESIZES ioctl fails because of strange implementation (AEGHB-1247)

[ghpille]

Checklist

• Checked the issue tracker for similar issues to ensure this is not a duplicate
• Read the documentation to confirm the issue is not addressed there and your configuration is set correctly
• Tested with the latest version to ensure the issue hasn't been fixed

How often does this bug occurs?

always

Expected behavior

I'm trying to list the known framesizes for each known pixelformat of a MIPI CSI connected camera on a esp32p4.
I'm using a ioctl VIDIOC_ENUM_FRAMESIZES with the fd of the camera and each known pixelformat. Expecting something like

Capture: 0 JPEG, JFIF JPEG
 Sizes for 3, JPEG
 w x h: 320 x 240
 w x h: 640 x 480

Actual behavior (suspected bug)

Not a single framesize is found.

When I look at esp-video-components-master/esp_video/src/esp_video.c, the ioctl is handled by esp_video_enum_framesizes which uses the v4l2_frmsizeenum frmsize parameter's type field to call esp_video_get_stream.
Normally, you don't give a type, you only fill in the pixelformat when calling a VIDIOC_ENUM_FRAMESIZES. The type should be returned, specifying whether the framesize is discrete, stepwise or continuous.
Stranger still, esp_video_get_stream expects a v4l2_buf_type, so passing a pixelformat to the enum functions makes no sense. In the test code, I try passing a buffer type too, but still no framesizes.

capture_stream_main.c

Error logs or terminal output

I (1551) ov5647: Detected Camera sensor PID=0x5647
Trying to list formats for /dev/video0
Capture: 0 BA81, RAW8 BGGR
 Sizes for 5, BA81
VIDIOC_ENUM_FRAMESIZES error: Invalid argument
Capture: 1 RGBP, RGB 5-6-5
 Sizes for 5, RGBP
VIDIOC_ENUM_FRAMESIZES error: Invalid argument
Capture: 2 RGB3, RGB 8-8-8
 Sizes for 5, RGB3
VIDIOC_ENUM_FRAMESIZES error: Invalid argument
Capture: 3 YU12, YUV 4:2:0
 Sizes for 5, YU12

Steps to reproduce the behavior

run the attached code on a esp32p4 with camera attached

Project release version

master

System architecture

other (details in Additional context)

Operating system

Linux

Operating system version

ESP-IDF v6.0

Shell

other (details in Additional context)

Additional context

No response

[donghengqaz]

Hi @ghpille , this is the usage limitation, currently the camera sensor driver doesn't support to resize the output picture, so the ioctl commands of "get/set/enum" are also not supported, but you can select the camera sensor output format resolution in menuconfig. Beside this, we plan to add a crop module to the ISP to support resize the output picture in the SoC side instead of in the camera sensor side.

[ghpille]

Hallo, @donghengqaz, I don't know what is meant by "usage limitation". If the format resolution is set in menuconfig, is it not the driver that passes this information to the camera? Then it is either not passed to the camera or some other part of the code does the transfer. And if you say "currently", does that mean that in the future, such functionality may be added to the driver?

[donghengqaz]

Hi @ghpille ,

I don't know what is meant by "usage limitation". If the format resolution is set in menuconfig, is it not the driver that passes this information to the camera? Then it is either not passed to the camera or some other part of the code does the transfer

You are right, the camera sensor driver sets the resolution based your menuconfig option, but the setting behavior is not to call functions like "set_width/set_height/set_resolution", but it directly write the registers with fixed value such as here. So although application pass the target resolution to the camera sensor driver, these parameters can't be handle correctly.

And if you say "currently", does that mean that in the future, such functionality may be added to the driver?

No, there is no such plan, but I will talk with the camera sensor driver maintainer about this. And we will add a crop module to implement this feature in the ESP32-P4 ECO5.

[ghpille]

I forgot to thank whoever wrote those deceitful functions for wasting my time. It was very clever of that person to return an error (EINVAL) blaming the user for passing erroneous parameters to the call. You don't expect the driver itself to be at fault, normally driver programmers are known for supplying very high standards of code. Espressif must be having other priorities, I am not impressed by its repositories here on Github. Perhaps I am losing my touch, I've been programming for 48 years now.
@donghengqaz, I had a look at the code you referred to ("here"), I couldn't help but noticing how close this resembles the Linux driver's code. Do you know why Espressif didn't use more of the V4L2 driver code?

[donghengqaz]

Hi @ghpille ,

Do you know why Espressif didn't use more of the V4L2 driver code?

There are 2 root causes:

1. For camera sensor without internal ISP, the V4L2 driver code especially the resolution modification depend on the hardware, for example 720P matches a pair of ISP configuration parameters and 1080 has an another pair of ISP configuration parameters, this may cause issues when users set a customized resolution, such as 320 * 320. In some hardware platforms, they have a crop module in ISP pipeline, this can receive the 1080P images from camera sensor, and crop it into the customized resolution, such as 320 * 320, they don't need different ISP configuration parameters, the ESP32-P4 ECO5 also features this module.

2. For camera sensor with internal ISP, the camera sensor driver developers have not enough help from the camera sensor vendors, they can't make sure just modification resolution is enough. Or some other registers may also need to be rewritten? Or which value is better ? and so on. So they choose to not add these part of code. They would like that customers can directly to use the code in there products.