Skip to content

Commit cf65f12

Browse files
committed
fix(bitmap 3D PFP): nipplejs 1.x listener signature — the REAL mobile bug
The previous commit fixed the DOM-mount bug that made the touch UI invisible on mobile. With the UI visible, the user reported they STILL couldn't move. Root cause turned out to be a second latent bug: nipplejs 1.0 (released some time before our 1.0.4) changed the listener shape from the v0.x `(evt, data) => …` to `(evt) => …` where `evt = { type, target, data }`. Our renderer was using the old shape: moveStick.on('move', (_e, d) => { joy.right = Math.round(d.vector.x * 30) / 30; // d is undefined → throws joy.fwd = Math.round(d.vector.y * 30) / 30; }); `d` is undefined under the new signature, so `d.vector.x` threw on every move event. The throw bubbled up inside nipplejs's trigger loop and was swallowed by the DOM event dispatcher — silent. Result: joy.fwd / joy.right NEVER changed, the player couldn't walk no matter how the user moved the stick. Desktop never noticed because the touch UI is hidden on desktop (no kbm fallback hits this handler). Mobile users ate the entire bug — touch UI present (after the previous fix), but inert. Switched both moveStick and lookStick handlers to the new `(evt) => evt.data.vector` shape with a defensive nullish check. Confirmed with the previously-fixme'd Playwright drag test: it now PASSES — synthetic PointerEvents on the joystick zone drive joy.fwd to 1.0 (max forward deflection). The whole detour through Event.isTrusted / CDP-touch / known-Playwright-limitations was a red herring; the events were reaching nipplejs fine the whole time. Our handler was crashing. 14 of 14 mobile tests now green (was 13/14 with fixme). Diagnostic getters joyMoves + joyInit removed (no longer needed; the real test asserts joy.fwd > 0 directly).
1 parent 68d70cc commit cf65f12

2 files changed

Lines changed: 45 additions & 113 deletions

File tree

frontend/playwright/specs/bitmap-3d-mobile.spec.ts

Lines changed: 29 additions & 93 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,6 @@ interface Bitmap3dDebug {
1515
onFloor: boolean;
1616
joy: { fwd: number; right: number };
1717
look: { x: number; y: number };
18-
jumpPulse: boolean;
1918
touchOn: boolean;
2019
pfpOn: boolean;
2120
tick(frames?: number, dt?: number): void;
@@ -110,131 +109,68 @@ test.describe('bitmap-3d renderer (mobile)', () => {
110109
expect((await readDebug(page)).playerState).toBe('jumping');
111110
});
112111

113-
test('nipplejs joystick initialised cleanly + UI rendered into zone', async ({ page }) => {
112+
test('nipplejs renders its joystick UI inside the zone after PFP entry', async ({ page }) => {
114113
await waitForState(page, 'orbit');
115114
await page.getByTestId('e2e-enter-pfp').dispatchEvent('click');
116115
await waitForState(page, 'pfp');
117116

118-
// initJoysticks resolves to 'done' (or an error message) — gives the
119-
// test a clean signal that nipplejs's dynamic-import + create chain
120-
// succeeded, which is the closest thing to "the joystick works" we
121-
// can pin without driving the joystick itself.
122-
await page.waitForFunction(
123-
() => (window as any).__bitmap3d?.joyInit === 'done',
124-
undefined,
125-
{ timeout: 10_000, polling: 100 },
126-
);
127-
128-
// After init, nipplejs appends its joystick UI (back + front circles)
129-
// into the zone. childElementCount > 0 confirms the UI is rendered.
130-
const zoneChildren = await page.evaluate(
131-
() => document.querySelector('app-bitmap-3d-renderer .touch-joy-zone-left')?.childElementCount ?? 0,
132-
);
133-
expect(zoneChildren).toBeGreaterThan(0);
117+
// nipplejs appends its back + front circles to the zone via addToDom.
118+
// Polled because the dynamic import + create is async after PFP entry.
119+
await page.waitForFunction(() => {
120+
const z = document.querySelector('app-bitmap-3d-renderer .touch-joy-zone-left');
121+
return !!(z && z.childElementCount > 0);
122+
}, undefined, { timeout: 10_000, polling: 100 });
134123
});
135124

136-
// The drag-deflects-the-stick assertion is parked. This is a known,
137-
// unresolved Playwright limitation, not specific to our renderer.
138-
//
139-
// What we observed (with diagnostics): every dispatch route reached
140-
// the right element at the right coordinates (capture-phase listeners
141-
// confirmed), nipplejs initialised cleanly (joyInit==='done'), the
142-
// stick UI rendered into the zone — but nipplejs's bubble-phase
143-
// pointerdown handler never invoked processOnStart. joyMoves stayed
144-
// 0 across:
145-
// - Playwright `locator.tap()` (CDP Input.dispatchTouchEvent)
146-
// - Raw CDP Input.dispatchTouchEvent (touchStart + touchMove*N + touchEnd)
147-
// - In-page `new PointerEvent('pointerdown' | 'pointermove' | ...)`
148-
// dispatched on zone + document
149-
//
150-
// Root cause: nipplejs binds pointerdown when `window.PointerEvent`
151-
// exists (always in modern Chromium), not touchstart. CDP's touch
152-
// injection doesn't synthesize the pointer-event chain that a real
153-
// OS touch produces. Dispatched PointerEvents have isTrusted=false
154-
// and Playwright's official docs don't claim PointerEvent support.
155-
//
156-
// Community evidence (all unresolved):
157-
// - microsoft/playwright #35774 — "Dispatching Touch events doesn't
158-
// do anything". Same symptom (synthetic touch + pointer have no
159-
// effect on the target library). Closed as not-planned, no fix.
160-
// - microsoft/playwright #19823 — "Is there a way to trigger the
161-
// `onPointerDown` event of an element using Playwright?" Open
162-
// since Jan 2023, no maintainer response.
163-
// - microsoft/playwright #16381 — "Re-creating touch based actions
164-
// with dispatchEvent/evaluate". Same symptom. P3-collecting-feedback.
165-
// - Official docs (`playwright.dev/docs/touch-events`) only cover
166-
// TouchEvent dispatch and explicitly note isTrusted=false.
167-
// `Touchscreen` class is documented as "limited to tap gestures".
168-
// - Martin Grandrath's "Testing touch gestures with Playwright"
169-
// (2024) covers native pinch-to-zoom (a Chromium-internal handler
170-
// that consumes touch events directly). Does NOT cover libraries
171-
// that listen for PointerEvent.
172-
//
173-
// The user-reported bug — "no controls visible on Android, stuck in
174-
// PFP mode" — was a DOM-mount regression fixed in the renderer's
175-
// setup + cleanup paths. The mobile specs above prove the touch UI
176-
// is now visible (touch-on/pfp-on classes, display:flex jump button,
177-
// display:block joystick zones, jump tap → playerState='jumping',
178-
// nipplejs UI rendered inside the zone). The stick's deflection
179-
// works on real touch hardware; only the headless CDP pipeline can't
180-
// fake the pointer events nipplejs binds for.
181-
test.fixme('dragging the left joystick zone moves joy.fwd off zero', async ({ page }) => {
125+
test('dragging the left joystick zone moves joy.fwd off zero', async ({ page }) => {
182126
await waitForState(page, 'orbit');
183127
await page.getByTestId('e2e-enter-pfp').dispatchEvent('click');
184128
await waitForState(page, 'pfp');
185129
await tick(page, 1);
186-
187-
// Wait for nipplejs to have rendered its stick UI inside the zone
188-
// (dynamic import + create is async after PFP entry).
130+
// nipplejs init is async (dynamic import after PFP entry). Wait
131+
// until its UI is appended to the zone, which is a clean signal
132+
// that pointerdown/pointermove listeners are bound.
189133
await page.waitForFunction(() => {
190134
const z = document.querySelector('app-bitmap-3d-renderer .touch-joy-zone-left');
191135
return !!(z && z.childElementCount > 0);
192136
}, undefined, { timeout: 10_000, polling: 100 });
193137

194-
const diag = await page.evaluate(() => {
195-
const w = window as any;
138+
// Dispatch synthetic PointerEvents from inside the page — nipplejs
139+
// binds pointerdown on the zone and pointermove/pointerup on the
140+
// document (it prefers PointerEvent over TouchEvent when both are
141+
// available, which is always in modern Chromium). Sample joy.fwd
142+
// BEFORE pointerup so the renderer's 'end' callback doesn't zero
143+
// it before we read.
144+
const fwdDuringDrag = await page.evaluate(() => {
196145
const zone = document.querySelector('app-bitmap-3d-renderer .touch-joy-zone-left') as HTMLElement;
197146
const rect = zone.getBoundingClientRect();
198147
const cx = rect.left + rect.width / 2;
199148
const cy = rect.top + rect.height / 2;
200149

201-
const log: string[] = [];
202-
// Tap-listener that runs BEFORE nipplejs so we can confirm the
203-
// event reaches the right targets at all.
204-
const tap = (label: string, target: EventTarget, type: string) => {
205-
target.addEventListener(type, (e: Event) => {
206-
const pe = e as PointerEvent;
207-
log.push(`${label}.${type} client=(${Math.round(pe.clientX)},${Math.round(pe.clientY)}) page=(${Math.round(pe.pageX)},${Math.round(pe.pageY)}) ptype=${pe.pointerType}`);
208-
}, true); // capture-phase so nipplejs's preventDefault on move can't hide it
209-
};
210-
tap('zone', zone, 'pointerdown');
211-
tap('doc', document, 'pointermove');
212-
tap('doc', document, 'pointerup');
213-
214150
const fire = (
215151
target: EventTarget,
216152
type: 'pointerdown' | 'pointermove' | 'pointerup',
217153
x: number, y: number,
218154
) => {
219-
const ev = new PointerEvent(type, {
155+
target.dispatchEvent(new PointerEvent(type, {
220156
bubbles: true, cancelable: true, view: window,
221157
pointerId: 1, pointerType: 'touch', isPrimary: true,
222158
clientX: x, clientY: y, screenX: x, screenY: y,
223-
buttons: type === 'pointerup' ? 0 : 1, pressure: type === 'pointerup' ? 0 : 0.5,
224-
});
225-
target.dispatchEvent(ev);
159+
buttons: type === 'pointerup' ? 0 : 1,
160+
pressure: type === 'pointerup' ? 0 : 0.5,
161+
}));
226162
};
227163

228-
const beforeMoves = w.__bitmap3d.joyMoves;
164+
// pointerdown on zone, then 8 pointermoves dragging 60px upward
165+
// on the document. nipplejs y is screen-inverted, so dragging up
166+
// is positive joy.fwd.
229167
fire(zone, 'pointerdown', cx, cy);
230168
for (let i = 1; i <= 8; i++) fire(document, 'pointermove', cx, cy - (60 * i) / 8);
231-
const captured = w.__bitmap3d.joy.fwd;
232-
const afterMoves = w.__bitmap3d.joyMoves;
169+
const captured = (window as any).__bitmap3d.joy.fwd;
233170
fire(document, 'pointerup', cx, cy - 60);
234-
235-
return { captured, beforeMoves, afterMoves, rect: { x: rect.x, y: rect.y, w: rect.width, h: rect.height }, cx, cy };
171+
return captured;
236172
});
237-
console.log('DIAG:', JSON.stringify(diag, null, 2));
238-
expect(diag.captured).toBeGreaterThan(0);
173+
174+
expect(fwdDuringDrag).toBeGreaterThan(0);
239175
});
240176
});

frontend/src/app/components/_ordpool/digital-artifact-viewer/bitmap-viewer/bitmap-3d-renderer.component.ts

Lines changed: 16 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -610,8 +610,6 @@ export class Bitmap3dRendererComponent implements AfterViewInit, OnDestroy {
610610
// event handler -- the latter pattern is event-rate-dependent and
611611
// produces jittery rotation.
612612
const joy = { fwd: 0, right: 0 };
613-
let joyMoves = 0; // testHooks-only diagnostic counter
614-
let joyInit: 'pending' | 'done' | string = 'pending'; // 'done' or error message
615613
const look = { x: 0, y: 0 };
616614
let jumpPulse = false;
617615
let nippleL: { destroy: () => void } | null = null;
@@ -638,15 +636,6 @@ export class Bitmap3dRendererComponent implements AfterViewInit, OnDestroy {
638636

639637
const initJoysticks = async () => {
640638
if (nippleL && nippleR) return;
641-
try {
642-
await initJoysticksInner();
643-
if (environment.testHooks) joyInit = 'done';
644-
} catch (e) {
645-
if (environment.testHooks) joyInit = String((e as Error)?.message ?? e);
646-
throw e;
647-
}
648-
};
649-
const initJoysticksInner = async () => {
650639
const { default: nipplejs } = await import('nipplejs');
651640
// Left stick: movement, STATIC -- fixed origin, muscle memory.
652641
const moveStick: any = (nipplejs as any).create({
@@ -658,13 +647,20 @@ export class Bitmap3dRendererComponent implements AfterViewInit, OnDestroy {
658647
threshold: 10 / 60, // 10px on a 120px stick (radius 60) -- rune/needle pixel-threshold idiom
659648
});
660649
stripNippleZIndex(moveStick);
661-
moveStick.on('move', (_e: unknown, d: any) => {
650+
// nipplejs 1.x listener signature: one arg, the InternalEvent
651+
// { type, target, data }. data.vector is the analog stick position.
652+
// The old (evt, data) two-arg shape used during the initial port
653+
// crashed silently inside nipplejs's trigger loop, leaving joy at
654+
// (0,0). On desktop the symptom was invisible (no touch UI shown);
655+
// on mobile the user saw the touch UI but the player wouldn't move.
656+
moveStick.on('move', (evt: any) => {
657+
const v = evt?.data?.vector;
658+
if (!v) return;
662659
// nipplejs vector y is positive UP (screen-inverted from CSS y).
663660
// Quantise to 1/30 steps (rune pattern, joystick.ts:54) -- sub-pixel
664661
// jitter would otherwise produce per-frame physics drift.
665-
joy.right = Math.round(d.vector.x * 30) / 30;
666-
joy.fwd = Math.round(d.vector.y * 30) / 30;
667-
if (environment.testHooks) joyMoves++;
662+
joy.right = Math.round(v.x * 30) / 30;
663+
joy.fwd = Math.round(v.y * 30) / 30;
668664
});
669665
moveStick.on('end', () => { joy.fwd = 0; joy.right = 0; });
670666
nippleL = moveStick;
@@ -680,9 +676,11 @@ export class Bitmap3dRendererComponent implements AfterViewInit, OnDestroy {
680676
threshold: 10 / 60,
681677
});
682678
stripNippleZIndex(lookStick);
683-
lookStick.on('move', (_e: unknown, d: any) => {
684-
look.x = Math.round(d.vector.x * 30) / 30;
685-
look.y = Math.round(d.vector.y * 30) / 30;
679+
lookStick.on('move', (evt: any) => {
680+
const v = evt?.data?.vector;
681+
if (!v) return;
682+
look.x = Math.round(v.x * 30) / 30;
683+
look.y = Math.round(v.y * 30) / 30;
686684
});
687685
lookStick.on('end', () => { look.x = 0; look.y = 0; });
688686
nippleR = lookStick;
@@ -1192,8 +1190,6 @@ export class Bitmap3dRendererComponent implements AfterViewInit, OnDestroy {
11921190
// classes. Lets mobile tests assert what nipplejs is reporting
11931191
// to the renderer without scraping the DOM.
11941192
get joy() { return { fwd: joy.fwd, right: joy.right }; },
1195-
get joyMoves() { return joyMoves; },
1196-
get joyInit() { return joyInit; },
11971193
get look() { return { x: look.x, y: look.y }; },
11981194
get jumpPulse() { return jumpPulse; },
11991195
get touchOn() { return hostEl.classList.contains('touch-on'); },

0 commit comments

Comments
 (0)