Skip to content

Add a built-in virtual controller - #116716

Open
Kazox61 wants to merge 1 commit into
godotengine:masterfrom
Kazox61:virtual-controller
Open

Add a built-in virtual controller#116716
Kazox61 wants to merge 1 commit into
godotengine:masterfrom
Kazox61:virtual-controller

Conversation

@Kazox61

@Kazox61 Kazox61 commented Feb 24, 2026

Copy link
Copy Markdown
Contributor

Closes godotengine/godot-proposals#11193

The Virtual Controller creates Joypad-Motion and -Button events. This enables testing games on mobile devices if you setup Joypad controls in the Inputmap.

2026-02-24.14-45-00.mp4

TODO:

@Kazox61
Kazox61 requested review from a team as code owners February 24, 2026 14:09
Comment thread scene/gui/virtual_joystick.cpp Outdated
ADD_SIGNAL(MethodInfo("released", PropertyInfo(Variant::VECTOR2, "input_vector")));
ADD_SIGNAL(MethodInfo("flicked", PropertyInfo(Variant::VECTOR2, "input_vector")));
ADD_SIGNAL(MethodInfo("flick_canceled"));
ADD_SIGNAL(MethodInfo("motion", PropertyInfo(Variant::VECTOR2, "input_vector")));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"motion" is a bit of a misleading name IMO, it sounds like a property

Comment thread scene/gui/virtual_controller.h
Comment thread scene/gui/virtual_controller.h Outdated
Comment thread scene/gui/virtual_controller.cpp Outdated
@AThousandShips AThousandShips added this to the 4.x milestone Feb 24, 2026
@Ivorforce
Ivorforce requested review from a team February 24, 2026 14:30
@Nintorch
Nintorch self-requested a review February 24, 2026 15:20
@Nintorch

Copy link
Copy Markdown
Member

Hello! I haven't tested this PR, but so far I think it looks nicely done, good job! :)
The original proposal also mentions having a project setting to toggle the visibility of the virtual joystick, do you think it's a good idea? May I ask if it's possible to make this functionality?

@Kazox61

Kazox61 commented Feb 24, 2026

Copy link
Copy Markdown
Contributor Author

Project Settings or directly from the input singleton? Also currently I m not sure which Node I can use to add the VietualController as Child.

@Nintorch

Copy link
Copy Markdown
Member

Project Settings or directly from the input singleton?

I think having both would be nice, see also #115119 for an example of creating a project setting + Input class property, and #115119 (comment)

@JekSun97

Copy link
Copy Markdown
Contributor

Closes also Add analog joysticks to window Embedding game for android

@Kazox61

Kazox61 commented Feb 25, 2026

Copy link
Copy Markdown
Contributor Author

Should the Virtual Controller only show up when there is a Touchscreen available?

@Kazox61
Kazox61 requested a review from a team as a code owner February 25, 2026 09:16
@Kazox61

Kazox61 commented Feb 25, 2026

Copy link
Copy Markdown
Contributor Author

For pressing multiple buttons at the same time, #110893 is needed to get merged

@Nintorch

Copy link
Copy Markdown
Member

I'm not sure Input class can depend on classes from the scene/ folder.
I think I have an idea on how to fix it, I will try to do that when I have enough free time! :)
Basically, I think we can add the VirtualController node under scene tree root, and make the node itself react to changes in the Input class property.
I'm currently not sure if it's a good approach, but if it works and other maintainers are not against it, we can do that.

@Zireael07

Copy link
Copy Markdown
Contributor

Cameras have nothing to do with input though

@Nintorch Nintorch left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created a commit in one of my fork's branch to decouple Input from VirtualController and scene/ files: Nintorch@a931913
In that same commit I also deregistered VirtualController so its nodes can't be created in scenes directly by the user, I'm not currently sure it makes sense to allow that since the current functionality of enabling it through the Input property and a project setting should be enough, unless we want to allow the users to create 2 or more VirtualControllers or allow them to be freely modified. :) (EDIT: I just noticed that you're planning to make VirtualController customizable, in this case it does make sense for its nodes to be created directly by the user, so you can ignore that change by me in my commit! :D)
I also tested the functionality of VirtualController on Windows today. I think it mostly works as expected, except for 2 notes here:

  1. I think when the VirtualController is visible, Godot should register it as a connected joypad, so projects like https://gist.github.com/anthonyec/5342fce79b2b7b22ada748df0ad7f7c0 and joypad functions like Input.get_joy_axis() or Input.is_joy_button_pressed() can also be used with it.
  2. VirtualController is not suited for smaller output sizes at the moment. This can be a problem if it's used for pixel art games.
VirtualController with a regular size Image
VirtualController with a smaller size Image

Otherwise I think it looks nicely done! :)

Comment thread core/input/input.cpp
@Kazox61

Kazox61 commented Mar 1, 2026

Copy link
Copy Markdown
Contributor Author

@Nintorch I added the changes from your branch. I also registered the virtual controller, so the gamepad debugger works.

I noticed that the Dpad Up, Down and Right Button doesn't work. I couldn't find the reason? Did you have the same problem?

2026-03-01.15-56-53.mp4

@Kazox61
Kazox61 force-pushed the virtual-controller branch from 990cc55 to 897ae64 Compare March 1, 2026 15:03
@Nintorch

Nintorch commented Mar 1, 2026

Copy link
Copy Markdown
Member

I did have the same problem with dpad, yes. I'm not exactly sure what's causing it, but I think UI navigation might be messing with it.

I think I found another small bug, VirtualController at the root doesn't respect Camera2D image

This should be an easy fix though, we just first need to create a CanvasLayer at the scene root and then add VirtualController as its child.

Another small suggestion I have is that we may need to separate Enable Virtual Controller project setting into 2:

  1. Enable Virtual Controller: If true, a VirtualController node is created at the scene root, otherwise it's not created by default;
  2. Show Virtual Controller On Startup: If true, the VirtualController created by the project setting above will be shown on the game's startup, otherwise it will be hidden by default. (In this case, we might also need to rename Input.virtual_controller_enabled to Input.virtual_controller_visible and make it not do anything if Enable Virtual Controller is disabled, since the VirtualController isn't created)

The reason for that is that we probably don't want to create this node at the scene root for all Godot games, even the ones that won't use it, and we don't want joypad 0 to always be reserved for VirtualController. What do you think? :)

@Kazox61

Kazox61 commented Mar 1, 2026

Copy link
Copy Markdown
Contributor Author

Calinou suggested using CanvasLayer as a base here: godotengine/godot-proposals#11193 (comment)
Not sure it this has any negative side effects.

I also agree that the VirtualController shouldn't be instantiated for every project. That's why my initial commit created and destroyed always the virtual controller instead of hiding and showing it when enabling.

@Kazox61

Kazox61 commented Mar 1, 2026

Copy link
Copy Markdown
Contributor Author

@Nintorch what would you suggest to fix the problem with smaller output sizes?

@Nintorch

Nintorch commented Mar 2, 2026

Copy link
Copy Markdown
Member

I think having icons by default would be nice, but I feel like we may need to wait for an answer from the usability team to see what they think :)

@Alex2782

Alex2782 commented Apr 8, 2026

Copy link
Copy Markdown
Member

BaseButton: multitouch support #110893
Even though this PR cannot yet completely replace the TouchScreenButton, it would probably be a prerequisite for the built-in virtual controller.


🐧 Linux / Minimal template (target=template_release, everything disabled) fails.

Can it be reproduced on other platforms as well? (without cache_path)

Building with flags: platform=linuxbsd target=template_release dev_mode=yes module_text_server_fb_enabled=yes modules_enabled_by_default=no module_text_server_fb_enabled=no disable_3d=yes disable_advanced_gui=yes disable_physics_2d=yes disable_physics_3d=yes deprecated=no minizip=no brotli=no accesskit=yes cache_path=/home/runner/work/godot/godot/.scons_cache/ redirect_build_objects=no

/usr/bin/ld: core/libcore.linuxbsd.template_release.x86_64.a(input.linuxbsd.template_release.x86_64.o): in function `Input::set_virtual_controller_enabled(bool)':
input.cpp:(.text+0x32c): undefined reference to `VirtualController::VirtualController()'
collect2: error: ld returned 1 exit status
scons: *** [bin/godot.linuxbsd.template_release.x86_64] Error 1
scons: building terminated because of errors.
INFO: Time elapsed: 00:01:41.49
Error: Process completed with exit code 2.

@m4gr3d
m4gr3d requested review from KoBeWi, YeldhamDev and kitbdev April 9, 2026 15:24
@kitbdev

kitbdev commented Apr 9, 2026

Copy link
Copy Markdown
Contributor

Input is in core, so it should never include anything in "scene/".
This is probably why CI is failing, also see #53295.

@m4gr3d m4gr3d left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tested the feature; this is a great first step but there are still quite a bit of polish to be done before we consider it done and ready:

  • Multiple buttons cannot be pressed at the same time - fixed by #118608
  • Buttons should highlight on hover / press to show they are being interacted with
  • The virtual controller needs a configuration screen to allow developers to configure to match their game needs:
    • Need an option to hide / show each joystick and buttons
    • Need an option to update the buttons / joysticks contrast in order to make them more visible on light background
    • Need an option to configure the joystick (e.g: inverted / regular), or just expose the virtual joystick configuration options
    • Need an option to map an action to a button press

I think this would be great to land in Godot 4.8 so I've updated the milestone accordingly. @Kazox61 Are you able to drive this PR forward to target a 4.8 release?

@Kazox61

Kazox61 commented Apr 16, 2026

Copy link
Copy Markdown
Contributor Author

Yes I would like to continue. Currently I don't who should be the owner/parent of the VirtualController, since Input is not allowed.

Comment thread main/main.cpp Outdated
@m4gr3d

m4gr3d commented Apr 16, 2026

Copy link
Copy Markdown
Contributor

Yes I would like to continue. Currently I don't who should be the owner/parent of the VirtualController, since Input is not allowed.

@kitbdev Do you have any suggestions?

@kitbdev

kitbdev commented Apr 16, 2026

Copy link
Copy Markdown
Contributor

I really don't know much about this feature, but here's my suggestions.
To show UI from a Project Setting, the functionality should be moved to somewhere else like SceneTree, Window, or Viewport. I don't see any code reason it needs to be in Input, so it should be easy to move. If the controller should only show in the main window, then it probably has to be SceneTree. The closest thing to this situation is probably #79599 .

I don't know if it needs a property in addition to the project setting, but if you really want a property on the Input singleton, it could emit an unexposed signal (prefixed with _) that SceneTree connects to.

Edit: It's been mentioned in the proposal that SceneTree isn't a good fit and I agree, but I don't think we have anywhere else for it. Maybe a new singleton?

@kisg

kisg commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Our iOS virtual controller (which enables the system-level virtual controller on iOS) PR was just closed, referring to this PR: #97530

I am adding this comment here to have the discussion in a single place.

I am not sure if this PR is a complete replacement of our PR. While this PR adds a virtual controller implemented inside Godot, to me it seems that it is not the same as the built-in iOS virtual controller.

We will review this PR in more detail to see if it solves all the use cases we have, but there might be valid use cases supporting the built-in iOS virtual controller, either by adding it to the engine or adding any required GDExtension interface, so the iOS Virtual Controller can be added as an extension.

Actually, adding the virtual controller implementations as GDExtensions might be a better way forward: we could only add a small virtual GDExtension class to the engine that implements the necessary hooks inside the system, and there could be different GDExtension-based implementations for it, with different UI styles, or even using a platform's built-in version, if there is one... etc.

What do you think?

@Kazox61
Kazox61 force-pushed the virtual-controller branch from f86bae9 to c8f77b5 Compare June 18, 2026 16:24
Comment thread scene/gui/virtual_controller.cpp Outdated
@Kazox61
Kazox61 force-pushed the virtual-controller branch from c8f77b5 to b3770f1 Compare June 19, 2026 04:21
@Kazox61

Kazox61 commented Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

@m4gr3d I just want to give you my thoughts on the things you suggested.

Need an option to update the buttons / joysticks contrast in order to make them more visible on light background

You can configure the Button and Joystick Style when you are adding a custom theme in the project settings.

Need an option to map an action to a button press

You can use the Input Maps JoypadButtons and JoypadAxes to configure the Action

The idea of the VirtualController was to easily test a game with Gamepad support on Mobile for example the official demo projects

@migueldeicaza

Copy link
Copy Markdown
Contributor

We have been using Migeran's virtual controller in Xogot to onboard mobile developers to target iOS - as it provides a really nice user experience, and when we saw that that PR got closed for this, we took this version for a spin - happy to align ourselves with the official version. While Migeran's work was initially iOS-focused, they refactored it to be suitable as a foundation to plug other backends (we have also a version for the Web exporter when used on Mobile devices that also fits there).

I find that this contribution, while comprehensive is not on par to the one in #97530 on the iOS platform:

  • The other version provides native controllers on iOS, and integrates directly with the plugin/unplugging of a physical controller as needed.
  • This also happens to conflict with the Emulate Mouse from Touch setting (which is what folks are using to get the "MouseLook" behavior).
  • The OS managed one gets to respond immediately and is not tied to the Godot performance on a given frame.

Initially, we were using the Virtual Joystick add-on, but we ran into the latency issues when the game was busy - and using the OS provided one solved those problems in practice. This is a little harder to test now, as I upgraded to a much beefier iPad and iPhone, but folks with older phones would get the slower experience.

I also find that the default user interface needs some polish, and it should default to a minimalistic set, because the current set is both overwhelming and blocks much of the screen, if this is the path to follow, it would be better to use some sort of progressive disclosure system rather than putting everyone on the path of emulating the most advanced controller on a touch screen.

My personal opinion is that these virtual controllers are tools that you use as a quick bring up for your games, but generally, before you have the time to invest and polish a natural set of idioms.

Android also recently introduced a virtual controller that fits naturally into folded devices. I think it would be preferable to delegate this to the native platform support where possible.

So I think that the decision to summarily close the other PR on behalf of this one did not have the full discussion that it should have had.

In the end, it is not really a problem for us, we can ship both - but it would certainly regress users that have chosen a more robust system when they use a stock Godot template.

@Calinou

Calinou commented Jun 23, 2026

Copy link
Copy Markdown
Member

The OS managed one gets to respond immediately and is not tied to the Godot performance on a given frame.

This is something I've never thought about before when it comes to OS-provided virtual controllers. If the virtual controller updates at 120 Hz on 120 Hz displays, this can significantly improve the gameplay experience for 30/40/60 FPS games.

It's good to hear Android is introducing a virtual controller as well, although it'll be a while before you can rely on it as a baseline. We still have to support Android 7 for now after all.

@migueldeicaza

Copy link
Copy Markdown
Contributor

These new Android APIs are distributed as Jetpacks, so you can reference and use them and deploy to older systems, without needing the OS to be updated. There is a separate question as to how far that Android Virtual Controller will go - and whether you need a fallback or not.

@Kazox61

Kazox61 commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

I agree the best would be to use the native solutions from the platforms. For not-supported platforms we could provide a default virtual controller built with Control Nodes like in this PR.

I tried this in this Branch and used the Migeran's virtual controller as base.

@Kazox61

Kazox61 commented Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

How do we want to continue here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add analog joysticks to window Embedding game for android Add a built-in virtual controller for easier game development on mobile devices