Desktop Simulation

During development it is often inconvenient to be constantly putting the VR headset on and taking it off again. For this reason Tamarin provides a Desktop Simulation mode. In this mode the application behaves as close as possible to a VR application but with all behaviour driven by the mouse and keyboard. This is not intended for end users it is intended for development workflow (Tamarin tries to make making mixed VR/Desktop applications easy but for a good user experience some customised user code is likely to be required)

How to make it work

Refer to abstract XrActionBaseAppState and XrBaseAppState throughout your application

The XrActionAppState and XrAppState have base classes XrActionBaseAppState and XrBaseAppState. These should be referred to whenever you are getting a state

    XrBaseAppState xrAppState = application.getStateManager().getState(XrBaseAppState.ID, XrBaseAppState.class);
    XrActionBaseAppState openXrActionState = application.getStateManager().getState(XrActionBaseAppState.ID, XrActionBaseAppState.class);

[Note that VRHandsAppState does not directly touch OpenXR calls so there is no need to use a base state for it]

Include keys to simulate handset button presses

In the Action.builder() desktop simulation keys can be specified

These take a hand side that is generating the action, the keyboard button to simulate it and if it is a toggle. Toggle meaning if the keyboard button controls if the action flickers on then off (good for a trigger) or toggles between on and off states (good for a grip, or button intended to be held in VR)

Example:

.withDesktopSimulationKeyTrigger(HandSide.LEFT, new KeyTrigger(KeyInput.KEY_F1), true)

The pose does not need to be selected here (it is separately simulated), haptics cannot be simulated, and 2D actions are currently not supported.

Full Manifest:

public static ActionManifest manifest(){
    Action grip = Action.builder()
            .actionHandle(ActionHandles.GRIP)
            .translatedName("Grip an item")
            .actionType(ActionType.FLOAT)
            .withSuggestedBinding(GoogleDaydreamController.PROFILE, GoogleDaydreamController.pathBuilder().leftHand().trackpadClick())
            //other normal VR bindings 
            .withSuggestedBinding(ValveIndexController.PROFILE, ValveIndexController.pathBuilder().rightHand().squeezeValue())
            .withDesktopSimulationKeyTrigger(HandSide.LEFT, new KeyTrigger(KeyInput.KEY_F1), true)
            .withDesktopSimulationKeyTrigger(HandSide.RIGHT, new KeyTrigger(KeyInput.KEY_F2), true)
            .build();

    Action trigger = Action.builder()
            .actionHandle(ActionHandles.TRIGGER)
            .translatedName("Trigger action")
            .actionType(ActionType.FLOAT)
            .withSuggestedBinding(GoogleDaydreamController.PROFILE, GoogleDaydreamController.pathBuilder().leftHand().selectClick())
             //other normal VR bindings
            .withSuggestedBinding(ValveIndexController.PROFILE, ValveIndexController.pathBuilder().rightHand().triggerValue())
            .withDesktopSimulationKeyTrigger(HandSide.LEFT, new KeyTrigger(KeyInput.KEY_F3), false)
            .withDesktopSimulationKeyTrigger(HandSide.RIGHT, new KeyTrigger(KeyInput.KEY_F4), false)
            .build();

    Action openHandMenu = Action.builder()
            .actionHandle(ActionHandles.OPEN_HAND_MENU)
            .translatedName("Open Hand Menu")
            .actionType(ActionType.BOOLEAN)
            .withSuggestedBinding(GoogleDaydreamController.PROFILE, GoogleDaydreamController.pathBuilder().leftHand().trackpadClick())
             //other normal VR bindings
            .withSuggestedBinding(ValveIndexController.PROFILE, ValveIndexController.pathBuilder().rightHand().thumbStickClick())
            .withDesktopSimulationKeyTrigger(HandSide.LEFT, new KeyTrigger(KeyInput.KEY_F5), true)
            .withDesktopSimulationKeyTrigger(HandSide.RIGHT, new KeyTrigger(KeyInput.KEY_F6), true)
            .build();

    return ActionManifest.builder()
            .withActionSet(ActionSet
                    .builder()
                    .name("main")
                    .translatedName("Main Actions")
                    .priority(1)
                    .withAction(grip)
                    .withAction(trigger)
                    //other actions
                    .withAction(openHandMenu)
                    .build()
            ).build();
}

Use the Simulated App states

DesktopSimulatingXrAppState and DesktopSimulatingXrActionAppState can be used to instantiate VR-like behaviour where XrAppState and XrActionAppStat would be used in VR.

    AppSettings settings = new AppSettings(true);
    settings.setTitle("Tamarin OpenXR Example");
    settings.setSamples(4);
    settings.setWindowSize(1280, 720);
    Main app = new Main(
            new DesktopSimulatingXrAppState(),
            new DesktopSimulatingXrActionAppState(Main.manifest(), ActionHandles.HAND_POSE, ActionSets.MAIN),
            new VRHandsAppState(Main.handSpec()),
            //these are just the default JME states (that we have to explicitly select because of using the constructor that takes states)
            new FlyCamAppState(),
            new StatsAppState(),
            new ConstantVerifierState(),
            new DebugKeysAppState());
    app.setLostFocusBehavior(LostFocusBehavior.Disabled);
    app.setSettings(settings);
    app.setShowSettings(false);
    app.start();

#Full example

A full example can be found in the Tamarin Test Bed