feat(🤖): correct per-buffer YUV conversion for Android camera frames

[wcandillon]

Three improvements to the Android camera-frame path, found while reviewing the VisionCamera integration. Companion PR on the producer side: mrousavy/react-native-vision-camera#4023 (GPU-sampleable YUV buffers).

1. GPUExternalTexture.yuvToRgbMatrix (non-spec extension): per-buffer correct color conversion

Dawn samples Android external-format (opaque YCbCr) buffers through a Vulkan SamplerYcbcrConversion hard-coded to RGB_IDENTITY + full range (SamplerVk.cpp::GetYCbCrForTextureView, crbug.com/497675620), so textureSampleBaseClampToEdge returns raw [Y, Cb, Cr] and the shader must convert. The example hard-coded BT.709 narrow-range coefficients, but Android camera streams are usually BT.601, and range varies by device/driver, so colors were subtly wrong (hue shift + washed-out or crushed levels depending on the device).

The driver reports the truth per buffer (suggestedYcbcrModel / suggestedYcbcrRange); Dawn captures it at import time. We now read it back through SharedTextureMemoryAHardwareBufferProperties.yCbCrInfo and derive the exact 3x4 [Y, Cb, Cr, 1] -> R'G'B' matrix (601/709/2020, full/narrow). On iOS (Dawn converts NV12 in the sampler transform) and for RGBA surfaces the property is the identity passthrough, so shaders can apply it unconditionally:

const tex = device.importExternalTexture({ source: videoFrame, ... });
queue.writeBuffer(uniforms, OFFSET, new Float32Array(tex.yuvToRgbMatrix));

The VisionCamera example now uploads the matrix as three vec4f uniform rows instead of hard-coding coefficients in WGSL.

2. Fail fast on CPU-only AHardwareBuffers

Dawn derives WebGPU usages from the AHB usage bits (AHBFunctions.cpp) and only grants TextureBinding for AHARDWAREBUFFER_USAGE_GPU_SAMPLED_IMAGE. CameraX's default ImageReaders allocate CPU-only buffers, and the resulting failure was an opaque validation error deep inside Dawn. wrapNativeBuffer now checks the usage bits and throws an actionable error pointing at vision-camera's pixelFormat: 'native'.

3. Remove the unreachable biplanar branch on Android

Dawn maps every YUV AHB format (Y8Cb8Cr8_420, P010, vendor formats) to OpaqueYCbCrAndroid (AHBFunctions.cpp::FormatFromAHardwareBufferFormat), so the R8BG8Biplanar420Unorm plane-splitting path and its kBT709LimitedToRgb matrix could never run. Deleted, with the reasoning documented at the call site.

Bonus: example cleanup

The Android both-axes flip in the example (CAMERA_FLIP_X/CAMERA_FLIP_Y) is exactly a 180° rotation, now folded into the rotation passed to importExternalTexture; the WGSL prelude loses its per-fragment flip branches and platform conditionals.

Testing

• yarn tsc and yarn lint clean in packages/webgpu and apps/example.
• GPUExternalTexture.cpp syntax-checked for both branches (Apple clang and NDK r27 aarch64-linux-android29).
• Matrix derivation verified against the previous hard-coded BT.709 narrow values (matches to 4 decimal places for model=709/range=narrow).
• Needs an on-device pass (iOS + a BT.601 Android device) before merge; colors on Android should noticeably improve on devices where the driver reports 601.

Longer term, the cleaner fix is upstream: Dawn honoring suggestedYcbcrModel/Range in its static sampler would let us delete cameraDecode entirely and also fix chroma siting/filtering. We already carry a Dawn patch, so that's a viable follow-up.

🤖 Generated with Claude Code