index.bs

<pre class="metadata">
Shortname: webxr-gamepads-module
Title: WebXR Gamepads Module - Level 1
Group: immersivewebwg
Status: ED
TR: https://www.w3.org/TR/webxr-gamepads-module-1/
ED: https://immersive-web.github.io/webxr-gamepads-module/
Repository: immersive-web/webxr-gamepads-module
Level: 1
Mailing List Archives: https://lists.w3.org/Archives/Public/public-immersive-web-wg/

!Participate: <a href="https://github.com/immersive-web/webxr-gamepads/issues/new">File an issue</a> (<a href="https://github.com/immersive-web/webxr-gamepads-module/issues">open issues</a>)
!Participate: <a href="https://lists.w3.org/Archives/Public/public-immersive-web-wg/">Mailing list archive</a>
!Participate: <a href="irc://irc.w3.org:6665/">W3C's #immersive-web IRC</a>

Editor: Brandon Jones 87824, Google https://google.com/, bajones@google.com
Former Editor: Nell Waliczek 93109, Amazon [Microsoft until 2018] https://amazon.com/, nhw@amazon.com
Editor: Manish Goregaokar 109489, Google [Mozilla until 2020], manishearth@google.com
Editor: Rik Cabanier 106988, Meta https://facebook.com, cabanier@fb.com

Abstract: This specification module describes support for accessing button, trigger, thumbstick, and touchpad data associated with virtual reality (VR) and augmented reality (AR) devices on the Web.

Status Text: This WebXR Gamepads Module is designed as a module to be implemented in addition to <a href="https://www.w3.org/TR/webxr/">WebXR Device API</a>, and was originally included in WebXR Device API which was divided into core and modules.
</pre>

<pre class="link-defaults">
spec:infra;
    type:dfn; text:string
spec:webxr-1;
    type: dfn; text: change input source; for: XRSession
    type: dfn; text: list of active xr input sources
    type: dfn; text: viewer
    type: dfn; text: xr device; for: XRSession
    type: dfn; text: sensitive information
    type: dfn; text: primary action
    type: dfn; text: xr animation frame
    type: dfn; text: primary squeeze action
    type: dfn; text: trusted ui
</pre>

<pre class="anchors">
spec: WebXR Device API - Level 1; urlPrefix: https://www.w3.org/TR/webxr/#
    type: dfn; text: list of frame updates; for: XRSession; url: xrsession-list-of-frame-updates
    type: dfn; text: time; for: XRFrame; url: xrframe-time
spec: Gamepad; urlPrefix: https://www.w3.org/TR/gamepad/#
    type: interface; text: Gamepad; url: dom-gamepad
    type: interface; text: GamepadButton; url: dom-gamepadbutton
    type: enum; text: GamepadMappingType; url: dom-gamepadmappingtype
    type: enum-value; text: "xr-standard"; for: GamepadMappingType; url: dom-gamepadmappingtype-xr-standard
    type: method; text: navigator.getGamepads(); url: dom-navigator-getgamepads
    type: attribute; text: id; for: Gamepad; url: dom-gamepad-id
    type: attribute; text: index; for: Gamepad; url: dom-gamepad-index
    type: attribute; text: mapping; for: Gamepad; url: dom-gamepad-mapping
    type: attribute; text: connected; for: Gamepad; url: dom-gamepad-connected
    type: attribute; text: buttons; for: Gamepad; url: dom-gamepad-buttons
    type: attribute; text: axes; for: Gamepad; url: dom-gamepad-axes
    type: attribute; text: touched; for: GamepadButton; url: dom-gamepadbutton-touched
    type: attribute; text: pressed; for: GamepadButton; url: dom-gamepadbutton-pressed
    type: attribute; text: value; for: GamepadButton; url: dom-gamepadbutton-value
    type: dfn; text: gamepadconnected; for: Window; url: dfn-gamepadconnected
    type: dfn; text: gamepaddisconnected; for: Window; url: dfn-gamepaddisconnected
    type: dfn; text: select an unused gamepad index; url: dfn-selecting-an-unused-gamepad-index
</pre>

<link rel="icon" type="image/png" sizes="32x32" href="favicon-32x32.png">
<link rel="icon" type="image/png" sizes="96x96" href="favicon-96x96.png">

<style>
  .unstable::before {
    content: "This section is not stable";
    display: block;
    font-weight: bold;
    text-align: right;
    color: red;
  }
  .unstable {
    border: thin solid pink;
    border-radius: .5em;
    padding: .5em;
    margin: .5em calc(-0.5em - 1px);
    background-image: url("data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' width='300' height='290'><text transform='rotate(-45)' text-anchor='middle' font-family='sans-serif' font-weight='bold' font-size='70' y='210' opacity='.1'>Unstable</text></svg>");
    background-repeat: repeat;
    background-color: #FFF4F4;
  }
  .unstable h3:first-of-type {
    margin-top: 0.5rem;
  }

  .unstable.example:not(.no-marker)::before {
    content: "Example " counter(example) " (Unstable)";
    float: none;
  }

  .non-normative::before {
    content: "This section is non-normative.";
    font-style: italic;
  }
  .tg {
    border-collapse: collapse;
    border-spacing: 0;
  }
  .tg th {
    border-style: solid;
    border-width: 1px;
    background: var(--def-bg);
    font-family: sans-serif;
    font-weight: bold;
    border-color: var(--def-border);
  }
  .tg td {
    padding: 4px 5px;
    background-color: var(--def-bg);
    font-family: monospace;
    border-style: solid;
    border-width: 1px;
    border-color: var(--def-border);
    overflow: hidden;
    word-break: normal;
  }
</style>

Introduction {#intro}
============

<section class="non-normative">

Hardware that enables Virtual Reality (VR) and Augmented Reality (AR) applications are now broadly available to consumers, offering an immersive computing platform with both new opportunities and challenges. The ability to interact directly with immersive hardware is critical to ensuring that the web is well equipped to operate as a first-class citizen in this environment. The WebXR Gamepads module adds interfaces and behaviors to the WebXR Device API and the Gamepads API to allow for querying the state of buttons, triggers, thumbsticks, and touchpads available as input sources on many WebXR compatible devices.

</section>

This module is an addition to the <a href="https://www.w3.org/TR/webxr/">WebXR Device API</a>.

Terminology {#terminology}
-----------

This document uses the acronym <b>XR</b> throughout to refer to the spectrum of hardware, applications, and techniques used for Virtual Reality, Augmented Reality, and other related technologies. Examples include, but are not limited to:

 * Head mounted displays, whether they are opaque, transparent, or utilize video passthrough
 * Mobile devices with positional tracking
 * Fixed displays with head tracking capabilities

The important commonality between them being that they offer some degree of spatial tracking with which to simulate a view of virtual content.

Terms like "XR Device", "XR Application", etc. are generally understood to apply to any of the above. Portions of this document that only apply to a subset of these devices will indicate so as appropriate.

XR Devices often have additional controller hardware that allows users to interact with immersive experiences with button, trigger, thumbstick, or touchpad inputs. Frequently these devices are spatially tracked as well, and referred to as "motion controllers", "handheld controllers", or "tracked controllers".

WebXR Device API Integration {#webxr-device-api-integration}
============
As defined in the <a href="https://www.w3.org/TR/webxr/">WebXR Device API API</a>, an {{XRInputSource}} represents an <dfn>XR input source</dfn>, which is any input mechanism which allows the user to perform targeted actions in the same virtual space as the [=viewer=]. Example [=XR input source=]s include, but are not limited to, handheld controllers, optically tracked hands, and gaze-based input methods that operate on the [=viewer=]'s pose.

This document outlines the behavior of an {{XRInputSource}} when it has button, trigger, thumbstick, or touchpad data to report. This is commonly a motion controller, but may also be headset with buttons, triggers, thumbsticks, or touchpads on the [=XRSession/XR device=].  As stated in the WebXR device API, input mechanisms which are not explicitly associated with the [=XRSession/XR Device=], such as traditional gamepads, MUST NOT be considered [=XR input source=]s.

XRInputSource {#xrinputsource-interface}
-------------

Button, trigger, thumbstick, and touchpad data is reported though a {{Gamepad}} object exposed on the {{XRInputSource}} it is associated with.

<pre class="idl">
partial interface XRInputSource {
  [SameObject] readonly attribute Gamepad? gamepad;
};
</pre>

The <dfn attribute for="XRInputSource">gamepad</dfn> attribute is a {{Gamepad}} that describes the state of any buttons and axes on the [=XR input source=]. If the [=XR input source=] does not have at least one of the following properties, the {{gamepad}} attribute MUST be <code>null</code>:

 - A single button and a gripSpace
 - More than one button
 - One or more axes

XRSession {#xrsession-interface}
------------

{{XRInputSource}}s are reported in the {{XRSession/inputSources}} array as they are connected and disconnected. When the presence of a {{XRInputSource/gamepad}} changes for any entry in the {{XRSession/inputSources}} array, the user agent MUST invoke the WebXR Device API's algorithm for [=change input source|responding to input source attribute changes=].

The [=XRSession/list of frame updates=] is updated to include [=apply gamepad frame updates=].

To <dfn for=XRFrame>apply gamepad frame updates</dfn> for an {{XRFrame}} |frame|, the user agent MUST run the following steps:

 1. For each {{XRInputSource}} with a {{XRInputSource/gamepad}} |gamepad| associated with |frame|'s {{XRFrame/session}}, perform the following steps:
    1. Update |gamepad| to reflect the gamepad data at |frame|'s [=XRFrame/time=].

</div>

NOTE: This means that the {{XRInputSource/gamepad}} object is "live", and any internal state is to be updated in-place every frame. Furthermore, it doesn't work to save a reference to an {{XRInputSource}}'s {{gamepad}} on one frame and compare it to the same {{XRInputSource}}'s {{gamepad}} from a subsequent frame to test for state changes, because they will be the same object. Therefore developers that wish to compare input state from frame to frame should cache the state in question.


Gamepad API Integration {#gamepad-api-integration}
========================

{{Gamepad}} instances returned by an {{XRInputSource}}'s {{XRInputSource/gamepad}} attribute behave as described by the <a href="https://www.w3.org/TR/gamepad/">Gamepad API</a>, with several additional behavioral restrictions.

An {{XRInputSource}}'s associated {{Gamepad}} MAY be exposed via {{navigator.getGamepads()}}, even if there is no active XR session by user agent choice. This allows XR devices to be used as "regular" gamepads if the user agent wishes to allow this.

Note: Hand tracking {{XRInputSource}}s as described in [[WEBXR-HAND-INPUT-1]] may contain a {{Gamepad}} if the user agent wishes to allow hand-based input sources to be used with gamepad-based content.

Navigator {#navigator-differences}
-----------------------

The <a href="https://www.w3.org/TR/gamepad/">Gamepad API</a> states a snapshot of {{Gamepad}} data can be retrieved by calling the {{navigator.getGamepads()}} function. However, {{Gamepad}} instances returned by an {{XRInputSource}}'s {{XRInputSource/gamepad}} attribute MUST NOT be included in the array returned by {{navigator.getGamepads()}}.

Gamepad {#gamepad-differences}
------------------------

The following {{Gamepad}} attributes MUST exhibit the following behavioral restrictions when the {{Gamepad}} has been returned by an {{XRInputSource}}'s {{XRInputSource/gamepad}} attribute.

  - {{XRInputSource/gamepad}}'s {{Gamepad/id}} attribute MUST be an empty string (<code>""</code>).
  - {{XRInputSource/gamepad}}'s {{Gamepad/index}} attribute MUST be <code>-1</code> if it is not exposed via {{navigator.getGamepads()}}, otherwise it should be assigned as specified by [=select an unused gamepad index|selecting an unused gamepad index=].
  - {{XRInputSource/gamepad}}'s {{Gamepad/connected}} attribute MUST be <code>true</code> until the {{XRInputSource}} is removed from the [=list of active XR input sources=] or the {{XRSession}} is ended.
  - If an axis reported by the {{XRInputSource/gamepad}}'s {{Gamepad/axes}} array represents an axis of a touchpad, the value MUST be <code>0</code> when the associated {{GamepadButton}}'s {{GamepadButton/touched}} is <code>false</code>.

<dfn export id="xr-standard-gamepad-mapping">"xr-standard" Gamepad Mapping</dfn> {#xr-standard-heading}
-----------------------------

The {{GamepadMappingType/"xr-standard"}} mapping is defined in the <a href="https://www.w3.org/TR/gamepad/#dom-gamepadmappingtype">Gamepad API</a> and reserved for use by this spec. It indicates that the layout of the buttons and axes of the {{XRInputSource/gamepad}} corresponds as closely as possible to the tables below.

In order to report a {{Gamepad/mapping}} of {{GamepadMappingType/"xr-standard"}} the device MUST report a {{XRInputSource/targetRayMode}} of {{XRTargetRayMode/"tracked-pointer"}} and MUST have a non-<code>null</code> {{XRInputSource/gripSpace}}. It MUST have at least one <dfn>primary trigger</dfn>, separate from any touchpads or thumbsticks. The [=primary trigger=] MUST trigger the [=primary action=] for the input source. The device MAY have a <dfn>primary squeeze button</dfn>, which, if present, MUST trigger the [=primary squeeze action=] for the input source. If a device does not meet the requirements for the {{GamepadMappingType/"xr-standard"}} mapping it may still expose a {{XRInputSource/gamepad}} with a {{Gamepad/mapping}} of <code>""</code> (empty string). The {{GamepadMappingType/"xr-standard"}} mapping MUST only be used by {{Gamepad}} instances reported by an {{XRInputSource}}.

<table class="tg">
  <thead>
    <tr>
      <th>Buttons</th>
      <th><code>xr-standard</code> Mapping</th>
      <th>Required</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>buttons[0]</td>
      <td>[=Primary trigger=]</td>
      <td>Yes</td>
    </tr>
    <tr>
      <td>buttons[1]</td>
      <td>[=Primary squeeze button=]</td>
      <td>No</td>
    </tr>
    <tr>
      <td>buttons[2]</td>
      <td>Primary touchpad</td>
      <td>No</td>
    </tr>
    <tr>
      <td>buttons[3]</td>
      <td>Primary thumbstick</td>
      <td>No</td>
    </tr>
  </tbody>
</table>
<br/>
<table class="tg">
  <thead>
    <tr>
      <th>Axes</th>
      <th><code>xr-standard</code> Mapping</th>
      <th>Required</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>axes[0]</td>
      <td>Primary touchpad X</td>
      <td>No</td>
    </tr>
    <tr>
      <td>axes[1]</td>
      <td>Primary touchpad Y</td>
      <td>No</td>
    </tr>
    <tr>
      <td>axes[2]</td>
      <td>Primary thumbstick X</td>
      <td>No</td>
    </tr>
    <tr>
      <td>axes[3]</td>
      <td>Primary thumbstick Y</td>
      <td>No</td>
    </tr>
  </tbody>
</table>

Devices that lack one of the optional inputs listed in the tables above MUST preserve their place in the {{Gamepad/buttons}} or {{Gamepad/axes}} array, reporting a [=placeholder button=] or [=placeholder axis=], respectively. A <dfn>placeholder button</dfn> MUST report <code>0</code> for {{GamepadButton/value}}, <code>false</code> for {{GamepadButton/pressed}}, and <code>false</code> for {{GamepadButton/touched}}. A <dfn>placeholder axis</dfn> MUST report <code>0</code>. [=Placeholder buttons=] and [=placeholder axis|axes=] MUST be omitted if they are the last element in the array or all following elements are also [=placeholder buttons=] or [=placeholder axis|axes=].

Additional buttons or axes may be exposed after these reserved indices, and SHOULD appear in order of decreasing importance. Related axes (such as both axes of a thumbstick) MUST be grouped and, if applicable, MUST appear in X, Y, Z order.

UA/Platform reserved buttons {#reserved-buttons}
-----------------------------
User Agents SHOULD reserve at least one physical button, when possible, for performing unspoofable actions as part of a [=trusted UI=]. For example, the User Agent could designate the "menu" or "app" button present on many controllers as a dedicated button for exiting immersive presentation.

Additionally, many XR platforms also reserve a button for platform specific actions, such as returning to a home environment.

Buttons reserved by the UA or platform MUST NOT be exposed on the {{Gamepad}}. Additionally, User Agents SHOULD make an effort to coordinate which buttons are reserved for a given XR platform. The <a href="https://github.com/immersive-web/webxr-input-profiles/tree/master/packages/registry">WebXR Input Profiles Registry</a> is the recommended location for coordinating button reservations.

Example Mappings {#example-mappings}
-----------------------------

<section class="note">
This diagram demonstrates how two example controllers would be exposed with the {{GamepadMappingType/"xr-standard"}} mapping. Images are not intended to represent any particular device and are used for reference purposes only. <img src="images/xr-standard-mapping.svg" width="2100" height="2530" style="width: 750px; height: auto;" alt="Simple 'xr-standard' controller and Advanced 'xr-standard' controller">
</section>


Security and Privacy {#security}
=============================================

The WebXR Device API provides powerful new features which bring with them several unique privacy, security, and comfort risks that user agents must take steps to mitigate. This topic is covered in detail as part of the <a href="https://www.w3.org/TR/webxr/">WebXR Device API</a>. This module adds additional considerations, but does not change the fundamental [[WEBXR-1#security|WebXR security and privacy]] principles.

Fingerprinting {#fingerprinting-security}
--------------

Given that the API describes hardware available to the user and its capabilities it will inevitably provide additional surface area for fingerprinting. While it's impossible to completely avoid this, user agents should take steps to mitigate the issue. As defined in the WebXR Device API, {{XRInputSource}}s are only reported after an {{XRSession}} has been created, which requires additional protections when [=sensitive information=] will be exposed. In addition, this module requires {{XRInputSource}}'s {{XRInputSource/gamepad}}.{{Gamepad/id}} to not report a string identifiers.


<h2 id="changes" class="no-num">
Changes</h2>

<h3 id="changes-from-20191010" class="no-num">
Changes from the <a href="https://www.w3.org/TR/2019/WD-webxr-gamepads-module-1-20191010/">First Public Working Draft 10 October 2019</a></h3>

- Explicitly leave Gamepad API Integration up to the user agent (<a href="https://github.com/immersive-web/webxr-gamepads-module/pull/49">GitHub #49</a>)
- Added spec text specifying that reserved buttons are not exposed (<a href="https://github.com/immersive-web/webxr-gamepads-module/pull/48">GitHub #48</a>)
- Relaxed definitions of gamepad.index (<a href="https://github.com/immersive-web/webxr-gamepads-module/pull/46">GitHub #46</a>, <a href="https://github.com/immersive-web/webxr-gamepads-module/pull/47">GitHub #47</a>)
- Defined primary trigger and linked buttons (<a href="https://github.com/immersive-web/webxr-gamepads-module/pull/24">GitHub #24</a>)
- Clarified operations when the gamepad state is updated (<a href="https://github.com/immersive-web/webxr-gamepads-module/pull/22">GitHub #22</a>)


Acknowledgements {#ack}
================

The following individuals have contributed to the design of the WebXR Device API specification:

  * <a href="mailto:cvan@mozilla.com">Chris Van Wiemeersch</a> (<a href="https://mozilla.org/">Mozilla</a>)
  * <a href="mailto:kgilbert@mozilla.com">Kearwood Gilbert</a> (<a href="https://mozilla.org/">Mozilla</a>)
  * <a href="mailto:rafael.cintron@microsoft.com">Rafael Cintron</a> (<a href="https://microsoft.com/">Microsoft</a>)
  * <a href="mailto:sebastian.sylvan@gmail.com">Sebastian Sylvan</a> (Formerly <a href="https://microsoft.com/">Microsoft</a>)

And a special thanks to <a href="mailto:vladv@unity3d.com">Vladimir Vukicevic</a> (<a href="https://unity3d.com/">Unity</a>) for kick-starting this whole adventure!

</section>