Description
Is your feature request related to a problem? Please describe.
While trying to figure out why we find it so hard to configure cameras and video viewers in ways that balance image quality, latency, and bandwidth I've come across three situations that lead to the overall system behaving differently from what one might expect based on how the camera and video streams are configured. The camera server code has information that can explain a lot of these surprises. This proposal suggests putting that information into NetworkTables.
- The camera mode is supposed to be set to one of the available modes, but if it is set to something else, the code or the camera, will choose one of the actually available ones. However, what is currently reported on NT for the camera mode is the requested mode, not the actual one. Having the actual mode would both be helpful in catching the mistake and understanding why what you're seeing doesn't correspond to what you asked for.
- Similarly, a camera may be configured for an allowed mode but be unable to actually provide the frame rate for that mode based on other camera settings and/or the complexity of the scene being viewed. Currently, the only way I know to even estimate the achieved frame rate is using the camera stream viewers in SmartDashboard and Shuffleboard, but those rates may be affected by frame rate settings in the URL (or default settings established programatically) as well as network and dashboard computer performance.
- Related to the above, the frame rate, compression and resolution for a stream created with http://host:1181/?action=stream would normally be those configured for the camera, but if defaults have been set programmatically there is currently no easy way to discover this if you're just working at the client end (dashboard).
Describe the solution you'd like
- Change the current "mode" entry for cameras to "requested mode" and "actual mode".
- For each stream served by MjpegServer, add network table entries for the camera frame input rate (which I believe can be easily observed by calculating the frame rate for all frames that the current code retrieves using GetNextFrame) and the frame output rate (which is already calculated to decide whether to drop a retrieved frame). Finaly, it would be helpful to have an NT entry for the output bandwidth but I'm not sure this is knowable at this point in the code. The reason for this is that some of the "modern" dashboards whose use is being encouraged do not (yet?) have frame rate and bandwidth displays for the camera stream viewers.
- For each stream served by MjpegServer, add an NT entry (or entries) for the configured frame rate, resolution and compression settings actually being used.
Other considerations
For the actual vs. requested mode issue, on wpilibpi it is possible to find the actual mode using v4l2-ctl in an ssh session, but I have not been able to find out how to do it on the Roborio. Even on the Pi it is not convenient and certainly not obvious how to do it.