-
Notifications
You must be signed in to change notification settings - Fork 3
Action Based Vr
Modern VR uses a system by which the physical button presses are detached from the actions within the application. This allows different controllers to be mapped differently, or for the end users to create new mappings for new controllers they may own but which the application developer had not created a mapping for.
Action manifests must be at an actual file location (that must be referred to using absolute paths), it cannot be included within the jar. This is because the action manifest may need to be referred to by the openVR system (e.g. SteamVR) even when the application is not running.
Action manifests contain all the inputs your application will require and their types
See https://github.com/ValveSoftware/openvr/wiki/Action-manifest for streams documentation on action manifests.
{
"default_bindings": [
{
"controller_type": "oculus_touch",
"binding_url": "oculusTouchDefaults.json"
}
],
"actions": [
{
"name": "/actions/main/in/grip",
"requirement": "mandatory",
"type": "vector1"
},
{
"name": "/actions/main/out/haptic",
"type": "vibration",
"requirement": "optional"
}
]
,"action_sets": [
{
"name": "/actions/main",
"usage": "leftright"
}
],
"localization" : [
{
"language_tag": "en_us",
"/actions/main" : "My Game Actions",
"/actions/main/in/grip" : "Hand grip"
}
]
}
Action sets are groups of actions, defined by a prefix, that can be turned on and off as a group each tick (and can also be turned on and off per hand). This may provide a way to easily allow a user to change their dominant hand (e.g. have a "/actions/weapon" action set and a "/actions/navigation" action set and allow the user in game to decide which action set is active for which hand) or you may find it easier to just have one action set that is always active. In any case OpenVr must be informed of the active action set(s),
The active action sets can be set when first registering the action manifest
actionBasedOpenVrState.registerActionManifest(......, "/actions/main" )
Or later
actionBasedOpenVrState.setActiveActionSet("/actions/main");
Actions have these types
This is what it sounds like, a request for the controller to vibrate, it can be further configured (frequency, intensity, duration) within code
{
"name": "/actions/main/out/haptic",
"type": "vibration",
"requirement": "optional"
}
Vectors contain analogue data; be that the fractional pulling of a trigger, or the XY movement of a hat control
{
"name": "/actions/main/in/trigger",
"type": "vector1",
"requirement": "mandatory"
}
Boolean actions are simple digital button presses, they are either on or off
{
"name" : "/actions/main/in/turnLeft",
"type" : "boolean"
}
The pose is the hands bulk location and rotation
{
"name" : "/actions/main/in/HandPoseLeft",
"type" : "pose"
}
The skeleton is the fine movement of the hands; indicating individual bone position. The provided bones will always be the same 31 bones that are detailed at https://github.com/ValveSoftware/openvr/wiki/Hand-Skeleton (note that the left and right hands have different bone names!)
{
"name" : "/actions/main/in/HandSkeletonLeft",
"skeleton" : "/skeleton/hand/left",
"type" : "skeleton"
}
This section allows for translated strings, used within the streamVr or other openVr utilities for users to edit bindings.
References to other files (paths relative to the action manifest) can be provided. These files will map actions to controller buttons etc for a particular VR vender
"default_bindings": [
{
"controller_type": "oculus_touch",
"binding_url": "oculusTouchDefaults.json"
}
]
Default bindings files should be included that map actions for a particular controller.
{
"action_manifest_version" : 0,
"bindings": {
"/actions/main": {
"haptics" : [
{
"output" : "/actions/main/out/haptic",
"path" : "/user/hand/left/output/haptic"
},
{
"output" : "/actions/main/out/haptic",
"path" : "/user/hand/right/output/haptic"
}
],
"poses" : [
{
"output" : "/actions/main/in/HandPoseLeft",
"path" : "/user/hand/left/pose/raw"
},
{
"output" : "/actions/main/in/HandPoseRight",
"path" : "/user/hand/right/pose/raw"
}
],
"skeleton": [
{
"output" : "/actions/main/in/HandSkeletonLeft",
"path" : "/user/hand/left/input/skeleton/left"
},
{
"output" : "/actions/main/in/HandSkeletonRight",
"path" : "/user/hand/right/input/skeleton/right"
}
],
"sources" : [
{
"inputs" : {
"click" : {
"output" : "/actions/main/in/grip"
}
},
"mode" : "trigger",
"path" : "/user/hand/left/input/grip"
},
{
"inputs" : {
"click" : {
"output" : "/actions/main/in/grip"
}
},
"mode" : "trigger",
"path" : "/user/hand/right/input/grip"
},
{
"inputs" : {
"click" : {
"output" : "/actions/main/in/trigger"
}
},
"mode" : "trigger",
"path" : "/user/hand/right/input/trigger"
},
{
"inputs" : {
"click" : {
"output" : "/actions/main/in/trigger"
}
},
"mode" : "trigger",
"path" : "/user/hand/left/input/trigger"
},
{
"inputs" : {
"east" : {
"output" : "/actions/main/in/turnRight"
},
"north" : {
"output" : "/actions/main/in/teleport"
},
"west" : {
"output" : "/actions/main/in/turnLeft"
}
},
"mode" : "dpad",
"parameters" : {
"deadzone_pct" : "75",
"overlap_pct" : "0",
"sticky" : "true",
"sub_mode" : "touch"
},
"path" : "/user/hand/left/input/joystick"
},
{
"inputs" : {
"position" : {
"output" : "/actions/main/in/walk"
}
},
"mode" : "joystick",
"path" : "/user/hand/right/input/joystick"
}
]
}
},
"category" : "steamvr_input",
"controller_type" : "oculus_touch",
"description" : "Bindings for the for a oculusTouch controller",
"name" : "Example application bindings for a oculusTouch controller",
"options" : {},
"simulated_actions" : []
}
Controller specific binding examples are documented in a separate page
Vibrations can be triggered as follows
getStateManager().getState(ActionBasedOpenVrState.class).triggerHapticAction("/actions/main/out/haptic", duration, frequency, amplitude)
It is also possible to restrict the output to a particular hand by passing either "/user/hand/right" or "/user/hand/left" as an additional parameter
Vectors are mapped to analogue actions within java and can be accessed as follows
getStateManager().getState(ActionBasedOpenVrState.class).getAnalogActionState("/actions/main/in/trigger")
Note that Vector1, Vector2 and Vector3 type actions are all returned as Vector3 but only applicable coordinates will be non zero (i.e. a Vector1 will have y == 0 and z == 0).
It is also possible to restrict the input to a particular hand by passing either "/user/hand/right" or "/user/hand/left" as an additional parameter
Booleans are mapped to digital actions within java and can be accessed as follows
getStateManager().getState(ActionBasedOpenVrState.class).getDigitalActionState("/actions/main/in/turnLeft")
It is also possible to restrict the input to a particular hand by passing either "/user/hand/right" or "/user/hand/left" as an additional parameter
Poses (hand position and rotation) can be obtained as follows
getStateManager().getState(ActionBasedOpenVrState.class).getPose("/actions/main/in/HandPoseLeft")
Note that the position and rotation may seen "surprising", it may be easier to bind hands and use the utility methods in there which will be less surprising.
Skeletons are used for both animating the hand model and getting reference points for user finger positions (e.g. pushing buttons).
Tamarin provides mechanisms to get the raw bone positions (relative to the Pose's position)
getStateManager().getState(ActionBasedOpenVrState.class).getModelRelativeSkeletonPositions("/actions/main/in/HandSkeletonLeft")
Or to mutate an armature (set of bones that controls a mesh's deformation
getStateManager().getState(ActionBasedOpenVrState.class).updateHandSkeletonPositions("/actions/main/in/HandSkeletonLeft", armature, HandMode.WITHOUT_CONTROLLER);
Note the HandMode options, this controls weather the skeleton positions track where the fingers really are (i.e. holding a controller) or where the user probably wants them to be (i.e without controller)
It may be easier to bind hands and use the utility methods in there.