-
Notifications
You must be signed in to change notification settings - Fork 3
Action Based Vr
Modern VR uses a system by which the physical button presses are detached from the actions within the application. This allows different controllers to be mapped differently, or for the end users to create new mappings for new controllers they may own but which the application developer had not created a mapping for.
Action manifests contain all the inputs your application will require and their types
See https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#semantic-path-reserved for details on action paths.
//Notes that ActionHandles and ActionSets are assumed to be in User code as useful constants, they aren't part of Tamarin
Action openHandMenu = Action.builder()
.actionHandle(ActionHandles.OPEN_HAND_MENU)
.translatedName("Open Hand Menu")
.actionType(ActionType.BOOLEAN)
.withSuggestedBinding(OculusTouchController.PROFILE, OculusTouchController.pathBuilder().leftHand().thumbStickClick())
.withSuggestedBinding(OculusTouchController.PROFILE, OculusTouchController.pathBuilder().rightHand().thumbStickClick())
// other controllers suggested bindings
.build();
ActionManifest manifest = ActionManifest.builder()
.withActionSet(ActionSet
.builder()
.name(ActionSets.MAIN)
.translatedName("Main Actions")
.priority(1)
.withAction(openHandMenu)
// other actions
.build()
).build();
getStateManager().attach(new OpenXrActionState(manifest, ActionSets.MAIN));
Action sets are groups of actions, that can be turned on and off as a group each tick. This may provide a way to group actions that make sense in one section of your application but not another, or you may find it easier to just have one action set that is always active. In any case OpenVr must be informed of the active action set(s),
The active action sets can be set when first registering the action manifest
getStateManager().attach(new OpenXrActionState(......, "main" )
Or later
actionBasedOpenVrState.setActiveActionSet("main");
Actions have these types (see class ActionType)
This is a request for the controller to vibrate, it can be further configured (frequency, intensity, duration) within code
Floats contain analogue data, e.g. the fractional pulling of a trigger
These contain 2D analogue data, e.g. the position of a thumb stick
Boolean actions are simple digital button presses, they are either on or off
The pose is the hands bulk location and rotation. Typically two types of pose are exposed:
- Aim poses - This pose creates a coordinate system aligned with how you might hold a gun. If you are unsure which pose to use, use this one
- Grip poses - This pose creates a coordinate system aligned with how you are holding the controller, good for a sword fighting game. 90% of Tamarin functionality works with this pose but some things (like pick lines) might be in an annoying place)
The skeleton bones are actually part of the Pose and do not need to be registered with the manifest.
The skeleton is the fine movement of the hands; indicating individual bone position. The provided bones will always be the same bones that are detailed at https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#convention-of-hand-joints. The bone names are not explicitly defined in OpenXR but the expected names for bone armatures are in the Tamarin class BoneMappings. They can also be changed from the default values there.
Note that the bones are "surprising", they are not connected to each other parent child style but are all independent. If you want to create your own hand models it may be best to start with the bones provided in the https://github.com/oneMillionWorlds/Tamarin/tree/main/blenderFiles
Tamarin has the bindings for using thumb sticks like dpads (e.g. thumb stick left -> a button click) but this is not yet that well supported by runtimes. A more well supported appproach is to the use Tamarin's SyntheticDPad. This class allows you to update it with the Vector2f of a thumb stick and have it calculate a DPad like response
SyntheticDPad movementDpad = new SyntheticDPad();
@Override
public void update(float tpf){
movementDpad.updateRawAction(openXrActionState.getVector2fActionState(ActionHandles.MOVEMENT_DPAD));
BooleanActionState leftAction = movementDpad.east();
//use leftAction as if it was a normal Action
}
Default bindings files should be included that map actions for a particular controller. These are the calls .withSuggestedBinding(OculusTouchController.PROFILE, OculusTouchController.pathBuilder().leftHand().thumbStickClick()) from the earlier example
Vibrations can be triggered as follows
//Note that ActionHandles is assumed to be in User code as a useful constant used when registering the manifest, is isn't part of Tamarin
getStateManager().getState(OpenXrActionState.ID, OpenXrActionState.class).triggerHapticAction(ActionHandles.HAPTIC, duration, frequency, amplitude)
It is also possible to restrict the output to a particular hand by passing either "/user/hand/right" or "/user/hand/left" as an additional parameter
Booleans are mapped to digital actions within java and can be accessed as follows //Note that ActionHandles is assumed to be in User code as a useful constant used when registering the manifest, is isn't part of Tamarin getStateManager().getState(OpenXrActionState.ID, OpenXrActionState.class).getBooleanActionState(Actions.FIRE_WEAPON)
It is also possible to restrict the input to a particular hand by passing either HandSide.LEFT.restrictToInputString or HandSide.LEFT.restrictToInputString as an additional parameter
Other actions are similar
Poses (hand position and rotation) can be obtained as follows //Note that ActionHandles is assumed to be in User code as a useful constant used when registering the manifest, is isn't part of Tamarin getStateManager().getState(OpenXrActionState.ID, OpenXrActionState.class).getPose(ActionHandles.POSE)
Skeletons are used for both animating the hand model and getting reference points for user finger positions (e.g. pushing buttons).
Tamarin provides mechanisms to get the raw bone positions (relative to the Pose's position)
//Note that ActionHandles is assumed to be in User code as a useful constant used when registering the manifest, is isn't part of Tamarin
getStateManager().getState(OpenXrActionState.ID, OpenXrActionState.class).getSkeleton(ActionHandles.POSE, HandSide.LEFT)
Note that the skeleton is in the frame of reference of the pose (i.e. the bones are relative to the pose position and rotation)
There is a utility method OpenXrActionState.updateHandSkeletonPositions that can control an armature for a hand model but it may be easier to use the Tamarin bound hand functionality