MPMask documentation issues

[marcusx2]

Description of issue (what needs changing)

MPMask has confusing documentation

Clear description

The MPMask class has 3 methods to get the output: hasFloat32Array, hasUint8Array, and hasWebGLTexture.

Configuration Options seem to indicate that by using outputCategoryMask true and outputConfidenceMasks true, then hasUint8Array will return true and hasfloat32array will return true, respectively. The quote is

If set to True, the output includes a segmentation mask as a uint8 image, where each pixel value indicates the winning category value.

However, if you check this example, the imageSegmenter is created with outputCategoryMask set to true, yet in the callback function, if you add the following logs

function callback(result: ImageSegmenterResult) {
  const cxt = canvasClick.getContext("2d");
  const { width, height } = result.categoryMask;
  let imageData = cxt.getImageData(0, 0, width, height).data;
  canvasClick.width = width;
  canvasClick.height = height;
  let category: String = "";
  console.log(result.categoryMask.hasFloat32Array());
  console.log(result.categoryMask.hasUint8Array());
  console.log(result.categoryMask.hasWebGLTexture());

Only hasWebGLTexture returns true. This is either a bug or a documentation issue. According to the documentation, hasUint8Array should return true because outputCategoryMask was set to true.

The documentation also doesn't say that the mask will always be available at least as a WebGLTexture, but that's what seems to be the case. I'd like to ask for confirmaton.

Finally, the MPMask has a close method, but the documentation doesn't specify when exactly it should be used. Instead, it seems to indicate that the user never has to call this method, and the example itself doesn't use it.

[marcusx2]

With further testing, it seems that the MPMask has a WebGLTexture if the ImageSegmenter is created with a GPU delegate, and has a uint8array if it's created with a CPU delegate.

The example uses uint8array and float32array going against the docs itself that recommends using the underlying type available, which in this case is the WebGLTexture.

[HarishPermal]

Hi @marcusx2,

Thanks for this detailed breakdown!, on MPMask behavior shifts based on the chosen delegate, As you see when the GPU delegate is chosen the data stays on the WebGL to maintain speed for CPU the data is in system memory as Uint8/Float32 arrays for direct access.

This is intentional to avoid slow data transfers between the CPU and GPU. And your feedback on updating the documentation, improving code examples will be very helpful for the community ☺️, i'll tag this under docs-bug and i'll discuss with the team and will the document.

Thanks Again.