Skip to content

Commit 35ea5a4

Browse files
oPinonoPinon
committed
Added support for hand tracking and PSVR2 controllers on Apple Vision Pro, in XR.
Refactored and augmented the VisionOSXRInterface to have three independent components: - CompositorServices rendering (from previous PR) - Hand tracking (using ARKit) - Controller tracking (using ARKit and GCController) The APIs in Godot are the same as on other XR platforms: XRController3D and XRHandModifier3D.
1 parent 944a3c6 commit 35ea5a4

13 files changed

Lines changed: 1215 additions & 80 deletions

File tree

‎doc/classes/ProjectSettings.xml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3680,6 +3680,12 @@
36803680
<member name="xr/shaders/enabled" type="bool" setter="" getter="" default="false">
36813681
If [code]true[/code], Godot will compile shaders required for XR.
36823682
</member>
3683+
<member name="xr/visionos/enable_controller_tracking" type="bool" setter="" getter="" default="false">
3684+
Enables and initializes visionOS controller tracking.
3685+
</member>
3686+
<member name="xr/visionos/enable_hand_tracking" type="bool" setter="" getter="" default="false">
3687+
Enables and initializes visionOS hand tracking.
3688+
</member>
36833689
</members>
36843690
<signals>
36853691
<signal name="settings_changed">

‎main/main.cpp‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2896,6 +2896,10 @@ Error Main::setup(const char *execpath, int argc, char *argv[], bool p_second_ph
28962896
GLOBAL_DEF_BASIC("xr/openxr/binding_modifiers/analog_threshold", false);
28972897
GLOBAL_DEF_RST_BASIC("xr/openxr/binding_modifiers/dpad_binding", false);
28982898

2899+
// visionOS settings
2900+
GLOBAL_DEF_BASIC("xr/visionos/enable_hand_tracking", false);
2901+
GLOBAL_DEF_BASIC("xr/visionos/enable_controller_tracking", false);
2902+
28992903
#ifdef TOOLS_ENABLED
29002904
// Disabled for now, using XR inside of the editor we'll be working on during the coming months.
29012905

‎misc/dist/apple_embedded_xcode/godot_apple_embedded/godot_apple_embedded-Info.plist‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,5 +75,7 @@
7575
$application_scene_manifest_immersive_configuration
7676
</dict>
7777
</dict>
78+
$hand_tracking_usage_description
79+
$accessory_tracking_usage_description
7880
</dict>
7981
</plist>

‎modules/visionos_xr/doc_classes/VisionOSXRInterface.xml‎

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,20 @@
11
<?xml version="1.0" encoding="UTF-8" ?>
22
<class name="VisionOSXRInterface" inherits="XRInterface" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="../../../doc/class.xsd">
33
<brief_description>
4-
visionOS XR implementation for rendering in immersive mode.
4+
visionOS XR implementation for rendering in immersive mode, hand tracking and controller tracking.
55
</brief_description>
66
<description>
7-
This is a visionOS XR implementation for rendering in immersive mode. It uses the Compositor Services framework to render an Immersive scene. It supports the full and mixed immersive modes mode. To use this module, you must set the [member EditorExportPlatformVisionOS.application/app_role] export setting to [code]Immersive[/code]. You can choose the immersion style using the [member EditorExportPlatformVisionOS.application/immersion_style] export setting. You must use the Metal rendering driver, and only the Mobile renderer is supported for now. You can initialize this interface in the [method Node._ready] method of any of your nodes as follows:
7+
This is a visionOS XR implementation. It has three modules:
8+
- the CompositorServices module, to render with Godot in immersive mode
9+
- the hand tracking module, using Apple's ARKit
10+
- the spatial controller module, using Apple's ARKit and GCController.
11+
Those modules can be enabled/disabled independently.
12+
13+
The rendering module uses the CompositorServices framework to render an Immersive scene. It supports the full and mixed immersion modes.
14+
To use this module, you must set the [member EditorExportPlatformVisionOS.application/app_role] export setting to [code]Immersive[/code].
15+
You can choose the immersion style using the [member EditorExportPlatformVisionOS.application/immersion_style] export setting.
16+
You must use the Metal rendering driver, and only the Mobile renderer is supported for now.
17+
You can initialize this interface in the [method Node._ready] method of any of your nodes as follows:
818
[codeblock]
919
func _ready() -> void:
1020
var interface = XRServer.find_interface("visionOS")
@@ -15,6 +25,12 @@
1525
viewport.use_hdr_2d = true
1626
[/codeblock]
1727
Do not initialize the XR interface in the [method Node._process] method, as this will initialize it at some unspecified point in the middle of the frame rendering, possibly causing incorrect visionOS API usage.
28+
29+
Enable the hand tracking module with the [member ProjectSettings.xr/visionos/enable_hand_tracking] project setting.
30+
It will populate the [code]"/user/hand_tracker/left"[/code] and [code]"/user/hand_tracker/right"[/code] XRHandTrackers.
31+
32+
Enable the spatial controller module using the [member ProjectSettings.xr/visionos/enable_controller_tracking] project setting.
33+
It will populate the [code]"left_hand"[/code] and [code]"right_hand"[/code] reserved XRController trackers.
1834
</description>
1935
<tutorials>
2036
</tutorials>

‎modules/visionos_xr/visionos_controller_tracking.h‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
/**************************************************************************/
2+
/* visionos_controller_tracking.h */
3+
/**************************************************************************/
4+
/* This file is part of: */
5+
/* GODOT ENGINE */
6+
/* https://godotengine.org */
7+
/**************************************************************************/
8+
/* Copyright (c) 2014-present Godot Engine contributors (see AUTHORS.md). */
9+
/* Copyright (c) 2007-2014 Juan Linietsky, Ariel Manzur. */
10+
/* */
11+
/* Permission is hereby granted, free of charge, to any person obtaining */
12+
/* a copy of this software and associated documentation files (the */
13+
/* "Software"), to deal in the Software without restriction, including */
14+
/* without limitation the rights to use, copy, modify, merge, publish, */
15+
/* distribute, sublicense, and/or sell copies of the Software, and to */
16+
/* permit persons to whom the Software is furnished to do so, subject to */
17+
/* the following conditions: */
18+
/* */
19+
/* The above copyright notice and this permission notice shall be */
20+
/* included in all copies or substantial portions of the Software. */
21+
/* */
22+
/* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, */
23+
/* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF */
24+
/* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. */
25+
/* IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY */
26+
/* CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, */
27+
/* TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE */
28+
/* SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */
29+
/**************************************************************************/
30+
31+
#pragma once
32+
33+
#ifdef VISIONOS_ENABLED
34+
35+
#include "visionos_definitions.h"
36+
37+
#ifdef __OBJC__
38+
#define Key GodotKey
39+
#import <CoreHaptics/CoreHaptics.h>
40+
#import <GameController/GameController.h>
41+
#undef Key
42+
#else // __OBJC__
43+
typedef struct GCController *GCController_t;
44+
typedef struct CHHapticEngine *CHHapticEngine_t;
45+
#endif // __OBJC__
46+
47+
class VisionOSXRInterface;
48+
49+
// Controller tracking using ARKit and GCController
50+
struct VisionOSControllerTracking {
51+
bool enabled = false;
52+
VisionOSAuthorizationStatus authorization = VisionOSAuthorizationStatus::NOT_DETERMINED;
53+
54+
bool active() const { return enabled && authorization == VisionOSAuthorizationStatus::ALLOWED; }
55+
56+
// ARKit state
57+
ar_accessory_tracking_provider_t accessory_tracking_provider = nullptr;
58+
ar_accessories_t accessories = nullptr;
59+
ar_accessory_anchor_t left_controller_anchor = nullptr;
60+
ar_accessory_anchor_t right_controller_anchor = nullptr;
61+
62+
// Controller state
63+
Ref<XRControllerTracker> left_controller_tracker;
64+
Ref<XRControllerTracker> right_controller_tracker;
65+
GCController *left_gc_controller = nullptr;
66+
GCController *right_gc_controller = nullptr;
67+
CHHapticEngine *left_haptic_engine = nullptr;
68+
CHHapticEngine *right_haptic_engine = nullptr;
69+
70+
// Notification observers
71+
id controller_observer = nullptr;
72+
id controller_disconnect_observer = nullptr;
73+
74+
void initialize(XRServer *p_xr_server, VisionOSXRInterface *p_xr_interface);
75+
void uninitialize(XRServer *p_xr_server);
76+
77+
void init_for_controller(GCController *p_controller);
78+
void setup_controller_notifications();
79+
void cleanup_controller_notifications();
80+
void handle_controller_disconnect(GCController *p_controller);
81+
void update_accessories_list();
82+
void update_controller_trackers_from_arkit(CFTimeInterval p_trackable_anchor_time);
83+
void update_controller_from_anchor(Ref<XRControllerTracker> p_controller_tracker, ar_accessory_anchor_t p_controller_anchor, GCController *p_gc_controller);
84+
85+
// Called directly from the same function on the XRInterface
86+
void trigger_haptic_pulse(const String &p_action_name, const StringName &p_tracker_name, double p_frequency, double p_amplitude, double p_duration_sec, double p_delay_sec);
87+
88+
VisionOSXRInterface *xr_interface = nullptr; // assigned on initialization
89+
};
90+
91+
#endif // VISIONOS_ENABLED

0 commit comments

Comments
 (0)