Define how input is handled

[cabanier]

Only the UA really knows where layers are placed in 3D space.
The author (or JS framework) can guess where they are but this might be brittle.

What is the best way to provide input events? Should non-projection layers become event targets?

[Manishearth]

I feel like we could trigger select events with xy coordinates?

Having xy coordinates would be useful for screen-space select events as well.

[cabanier]

I feel like we could trigger select events with xy coordinates?

Would it return a x/y coordinate + layer?

[Manishearth]

Oh, I was suggesting that we trigger select events on the layer, and we extend select events to also contain optional xy parameters when found on a layer. Or have an XRInputSourceLayerEvent: XRInputSourceEvent with xy coordinates. I like the former solution because we can use the same thing with screen space input.

[avaer]

What about projection layers? Would that be a special UA case, or left for the page to handle?

[cabanier]

Oh, I was suggesting that we trigger select events on the layer, and we extend select events to also contain optional xy parameters when found on a layer. Or have an XRInputSourceLayerEvent: XRInputSourceEvent with xy coordinates. I like the former solution because we can use the same thing with screen space input.

I agree. I think this means that we should make layers event targets.

[cabanier]

What about projection layers? Would that be a special UA case, or left for the page to handle?

I think they could handle events like in regular WebXR,

[Manishearth]

Perhaps we should open an issue on the core spec about adding xy coordinates to input source events, and perhaps extending the targetraymode to include something new for layers (we could also just use screen again).

[Manishearth]

So while discussing this with @asajeffrey we realized this might be trickier than "slap an onselect event on XRLayer and call it a day".

Basically, WebXR doesn't really strongly define a pointing ray. targetRayMode is the origin of the "preferred pointing ray", but applications are free to add an offset to it (for whatever reason). So we don't have enough information to actually run a hit test, unless we provide the user a way to specify an offset for certain.

Perhaps the shape of this API needs to be similar to the hit test API, using an XRRay.

Some ideas:

• You can register an XRRay on the input source, and it's used to trigger select events on the layer.
• You can create LayerHitTestSources with XRRays and input sources, and events fire on that instead.
• The input source has an attribute XRRay pointerRay, initialized to XRRay(), that we can modify.

I like this last one the best because it still preserves the original API we were discussing here for the simple case, with the option to tweak it if you need to (which is easily done by constructing an XRRay with the origin offset used on the target ray space).

[Manishearth]

It also occurs to me that we may not just want select events, but also pointerOver, pointerOut, pointerMove events so that applications can render a reticle. Though for a reticle the x,y coordinates in immersive-web/webxr#978 aren't enough, if you also want to truncate the pointer you need world space coordinates of the intersection point, or a ray distance.

This may be a separate issue, just wanted to mention it.

[cabanier]

It also occurs to me that we may not just want select events, but also pointerOver, pointerOut, pointerMove events so that applications can render a reticle.

Yes.

Though for a reticle the x,y coordinates in immersive-web/webxr#978 aren't enough, if you also want to truncate the pointer you need world space coordinates of the intersection point, or a ray distance.

I think that depends where you want to render the reticle. If it's in the layer that is hit, x/y would be enough. If you want to do it in a projectionlayer on top, yes, you'd also need the position in 3D.
Maybe we need to provide both...

[cabanier]

• You can register an XRRay on the input source, and it's used to trigger select events on the layer.
• You can create LayerHitTestSources with XRRays and input sources, and events fire on that instead.
• The input source has an attribute XRRay pointerRay, initialized to XRRay(), that we can modify.

I like this last one the best because it still preserves the original API we were discussing here for the simple case, with the option to tweak it if you need to (which is easily done by constructing an XRRay with the origin offset used on the target ray space).

This XRRay would have the pointing ray as preferred by the author?
If so, yes I agree.

[Manishearth]

If you want to do it in a projectionlayer on top, yes, you'd also need the position in 3D.
Maybe we need to provide both...

Well, this is more so that you can truncate a pointing ray. so that it's not poking through the layer.

We can subclass the event to add a rayDistance, though.

This XRRay would have the pointing ray as preferred by the author?

Yep. Initialized to the default XRRay, but can be set to anything else.

[Manishearth]

Supporting pointer move events might have a problem: that seems more of a performance pitfall (since you're hit testing every frame) compared to select events. I wonder if it should be opt in somehow.

(UAs could theoretically run it only when there are events attached, but idk if that's easy to practically implement)