Camera feeds for Android, Windows and iOS and OGV support for MovieWriter

[penninghlhd]

Camera feeds

• I have added camera feed support for Android and Windows.
• Also improved camera support for MacOS, which should work on iOS as well.
• Instead of one camera feed per device, CameraServer now creates a feed per device/format/size. This way you can easily query CameraServer for the most appropriate feed based on its orientation, data format and size.
• Added example project using shaders to render camera feed.

MovieWriter

• Added support for OGV audio/video recording to MovieWriter.
• Tested on MacBook M3 with good results (30 fps with 720p).

---

• Production edit: This may supersede [3.x] Implement CameraServer on Windows #49763.

[AThousandShips]

Your branch contains unrelated commits from the 4.3 branch, please remove them

Also these changes should be separated into two parts, one for the camera feature and one for the movie writer part

You've also included unrelated files

[penninghlhd]

I do not know how to remove commits from a branch or two split commits into two separate PRs. I understand that this is better for the review/merge process, but I do not really have much time to get into this.

I think the changes are useful, at least for the further development of the CameraServer/Feed functionality. The same applies to the OGV support for MovieWriter. So if anyone else could pick this up, it would be great.

[fire]

To split commits into two separate PRs I would make two duplicate pull requests and then delete the parts that don't fit.

[keendra]

To split commits into two separate PRs I would make two duplicate pull requests and then delete the parts that don't fit.

Has this been done, can we follow the progress somewhere? Need the camera support for Android :)

[penninghlhd]

As I said before, I do not have the time to fix the conflicts myself at the moment. Also the conflicts arise largely from the recently added support for Linux which also implied some interface changes.

I propose someone in charge of arcitecture for the Camera feature should redefine the interface in order to resolve the conflicts properly.

In my opinion the main issues to be addressed should be:

1. Do we define a feed for each combination of supported dimension and data format and let the user choose to most appropriate feed (as I implemented in this PR), or do we define a feed per camera and make the preferred dimension and data format configurable via CameraTexture?
2. Do we use (predefined/custom) shaders to render the different image formats (YUV, N21, etc) by the GPU via CanvasTexture channels (as I implmented in this PR), or do we use CPU based conversion to RGB format supported by the default renderer.

[BastiaanOlij]

Was only just been made aware of this PR, shame to see it lying dormant because at first glance this implements a lot of missing logic.

Agree with some of the remarks up above that this PR should be split into multiple PRs, camera and movie logic should be separated, and it would be good if the camera logic can be split into a PR for the core changes, and then PRs for each added platform.

Also lots of nitpicking issues, formatting, documentation changes seem incomplete, there are unnecesairy breaking changes in reordering constants (enums) which do look better code wise but will be a blocker for merging.

[keendra]

Was only just been made aware of this PR, shame to see it lying dormant because at first glance this implements a lot of missing logic.

Agree with some of the remarks up above that this PR should be split into multiple PRs, camera and movie logic should be separated, and it would be good if the camera logic can be split into a PR for the core changes, and then PRs for each added platform.

Also lots of nitpicking issues, formatting, documentation changes seem incomplete, there are unnecesairy breaking changes in reordering constants (enums) which do look better code wise but will be a blocker for merging.

Exactly, it would be great to see this picked up, fixed and pushed over the finish line. I'm worried it's going to be left to die like the pr to add support for android camera in 2021.

[RubenGarcia]

I would also like to see this move forward, e.g. this is probably also needed for Vuforia integration to get access to the camera feeds.

[RubenGarcia]

Branches
https://github.com/shiena/godot/tree/feature/support-android-camera
https://github.com/shiena/godot/tree/feature/support-windows-camera
seem to have the latest code.
Let´s wait for @shiena to create PRs for these.

[shiena]

@RubenGarcia
There's a pull request (#105476 ) out to support Windows cameras.
Following that, I'm currently working on adding support for Android cameras.

[RubenGarcia]

@shiena For completion, I also found
https://github.com/Lamelynx/GodotGetImagePlugin-Android
for android cameras, but it would be great to have a unified architecture, so I look forward to your PRs. I´m particularly interested in android and iOS support.

iOS support is added by a plugin
https://github.com/godot-sdk-integrations/godot-ios-plugins
but the maintainer is not available for the last 4 months, so work has moved to
https://github.com/zt-pawer/SwiftGodotIosPlugins
but camera support is only being considered as a possible future addition.

[shiena]

@RubenGarcia
This extension works perfectly, but I only got 1-2 fps on Android.
https://github.com/j20001970/godot-cameraserver-extension

[RubenGarcia]

@shiena, for reference, how many fps do you get in your android branch?

[akien-mga]

Hello! Thanks for contributing.

This PR seems to be interesting, but has multiple issues that would need to be resolved so it can be properly reviewed and possibly merged.

• It seems to be based off on old version of the 4.3 branch (see the 4.3.1.rc1 commit). This should be redone on top of the upstream master branch.
• This PR does too much at once. Adding support for camera feeds for Android, Windows and iOS is orthogonal to adding support for the OGV format in MovieWriter. Those should be split two separate PRs (at least one for camera feeds and one for OGV).
• There are various other form issues here like adding your own VS Code config to Git, which shouldn't be done (and the .gitignore you modified was meant specifically to prevent doing that). I won't review in-depth yet to point out such issues because the first 2 points need to be addressed first.

Edit: I see some of this was already pointed out earlier, and @penninghlhd mentioned they wouldn't have time to work on it further.

So someone should volunteer and pick up that work in new PRs. I'll mark this one as draft and salvageable.