Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ImageData Float16Array support #11143

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
+103 −37
Open

Add ImageData Float16Array support #11143

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
08076f0
Add ImageData Float16Array support
ccameron-chromium Mar 17, 2025
cb43615
Incorporate feedback from kaiido review
ccameron-chromium Mar 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
140 changes: 103 additions & 37 deletions source
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2907,6 +2907,7 @@ a.setAttribute('href', 'https://example.com/'); // change the content attribute
<li><dfn data-x="idl-DOMString" data-x-href="https://webidl.spec.whatwg.org/#idl-DOMString"><code>DOMString</code></dfn></li>
<li><dfn data-x="idl-double" data-x-href="https://webidl.spec.whatwg.org/#idl-double"><code>double</code></dfn></li>
<li><dfn data-x="IDL enumeration" data-x-href="https://webidl.spec.whatwg.org/#idl-enums">enumeration</dfn></li>
<li><dfn data-x="idl-Float16Array" data-x-href="https://webidl.spec.whatwg.org/#idl-Float16Array"><code>Float16Array</code></dfn></li>
<li><dfn data-x="idl-Function" data-x-href="https://webidl.spec.whatwg.org/#common-Function"><code>Function</code></dfn></li>
<li><dfn data-x="idl-long" data-x-href="https://webidl.spec.whatwg.org/#idl-long"><code>long</code></dfn></li>
<li><dfn data-x="idl-object" data-x-href="https://webidl.spec.whatwg.org/#idl-object"><code>object</code></dfn></li>
Expand Down Expand Up @@ -66035,19 +66036,25 @@ interface <dfn interface>TextMetrics</dfn> {
readonly attribute double <span data-x="dom-textmetrics-ideographicBaseline">ideographicBaseline</span>;
};

typedef (<span data-x="idl-Uint8ClampedArray">Uint8ClampedArray</span> or <span data-x="idl-Float16Array">Float16Array</span>) <dfn typedef>ImageDataArray</dfn>;

enum <dfn enum>ImageDataPixelFormat</dfn> { "<span data-x="dom-ImageDataPixelFormat-rgba-unorm8">rgba-unorm8</span>", "<span data-x="dom-ImageDataPixelFormat-rgba-float16">rgba-float16</span>" };

dictionary <dfn dictionary>ImageDataSettings</dfn> {
<span>PredefinedColorSpace</span> <span data-x="dom-PredefinedColorSpace-srgb">colorSpace</span>;
<span>ImageDataPixelFormat</span> <span data-x="dom-ImageDataSettings-pixelFormat">pixelFormat</span> = "<span data-x="dom-ImageDataPixelFormat-rgba-unorm8">rgba-unorm8</span>";
};

[Exposed=(Window,Worker),
<span>Serializable</span>]
interface <dfn interface>ImageData</dfn> {
<span data-x="dom-imagedata">constructor</span>(unsigned long sw, unsigned long sh, optional <span>ImageDataSettings</span> settings = {});
<span data-x="dom-imagedata-with-data">constructor</span>(<span data-x="idl-Uint8ClampedArray">Uint8ClampedArray</span> data, unsigned long sw, optional unsigned long sh, optional <span>ImageDataSettings</span> settings = {});
<span data-x="dom-imagedata-with-data">constructor</span>(<span>ImageDataArray</span> data, unsigned long sw, optional unsigned long sh, optional <span>ImageDataSettings</span> settings = {});

readonly attribute unsigned long <span data-x="dom-imagedata-width">width</span>;
readonly attribute unsigned long <span data-x="dom-imagedata-height">height</span>;
readonly attribute <span data-x="idl-Uint8ClampedArray">Uint8ClampedArray</span> <span data-x="dom-imagedata-data">data</span>;
readonly attribute <span>ImageDataArray</span> <span data-x="dom-imagedata-data">data</span>;
readonly attribute <span>ImageDataPixelFormat</span> <span data-x="dom-imagedata-pixelFormat">pixelFormat</span>;
readonly attribute <span>PredefinedColorSpace</span> <span data-x="dom-imagedata-colorSpace">colorSpace</span>;
};

Expand Down Expand Up @@ -70400,9 +70407,8 @@ try {

<dt><code data-x=""><var>imageData</var> = new <span subdfn data-x="dom-imagedata-with-data">ImageData</span>(<var>data</var>, <var>sw</var> [, <var>sh</var> [, <var>settings</var> ] ])</code></dt>
<dd>
<p>Returns an <code>ImageData</code> object using the data provided in the <code
data-x="idl-Uint8ClampedArray">Uint8ClampedArray</code> argument, interpreted using the given
dimensions and the color space indicated by <var>settings</var>.</p>
<p>Returns an <code>ImageData</code> object using the data provided in the <code>ImageDataArray</code>
argument, interpreted using the given dimensions and the color space indicated by <var>settings</var>.</p>

<p>As each pixel in the data is represented by four numbers, the length of the data needs to be
a multiple of four times the given width. If the height is provided as well, then the length
Expand Down Expand Up @@ -70491,7 +70497,7 @@ try {
constructor steps are:</p>

<ol>
<li><p>Let <var>length</var> be the number of bytes in <var>data</var>.</p></li>
<li><p>Let <var>length</var> be the length of <var>data</var>.</p></li>

<li><p>If <var>length</var> is not a nonzero integral multiple of four, then throw an
<span>"<code>InvalidStateError</code>"</span> <code>DOMException</code>.</p></li>
Expand Down Expand Up @@ -70519,8 +70525,7 @@ try {
<i><span data-x="initialize-imagedata-source">source</span></i> set to <var>data</var>.</p>

<p class="note">This step does not set <span>this</span>'s data to a copy of <var>data</var>.
It sets it to the actual <code data-x="idl-Uint8ClampedArray">Uint8ClampedArray</code> object
passed as <var>data</var>.</p>
It sets it to the actual <code>ImageDataArray</code> object passed as <var>data</var>.</p>
</li>
</ol>

Expand Down Expand Up @@ -70607,32 +70612,63 @@ try {


<p>To <dfn>initialize an <code>ImageData</code> object</dfn> <var>imageData</var>, given a
positive integer number of rows <var>rows</var>, a positive integer number of pixels per row
<var>pixelsPerRow</var>, an optional <code>ImageDataSettings</code> <dfn
data-x="initialize-imagedata-settings"><var>settings</var></dfn>, an optional <code
data-x="idl-Uint8ClampedArray">Uint8ClampedArray</code> <dfn
data-x="initialize-imagedata-source"><var>source</var></dfn>, and an optional
<code>PredefinedColorSpace</code> <dfn
positive integer number of pixels per row <var>pixelsPerRow</var>, a positive integer number
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't new, but aren't the arguments inverted here?
All the call-sites do Initialize imageData given sw, sh, settings set to [...], but here the algorithm expects number of rows as the first argument and then pixels per row.
Though maybe this should be fixed separately and I'm sure there is some modernization that could get in there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I swapped the ordering of the parameters, since that seems fairly urgent to not have wrong (when we do a modernization pass, we should also change the names to width and height).

of rows <var>rows</var>, an optional <code>ImageDataSettings</code> <dfn
data-x="initialize-imagedata-settings"><var>settings</var></dfn>, an optional
<code>ImageDataArray</code> <dfn data-x="initialize-imagedata-source"><var>source</var></dfn>,
and an optional <code>PredefinedColorSpace</code> <dfn
data-x="initialize-imagedata-defaultcolorspace"><var>defaultColorSpace</var></dfn>:</p>

<ol>
<li><p>If <var>source</var> was given, then initialize the <dfn attribute for="ImageData"><code
data-x="dom-imagedata-data">data</code></dfn> attribute of <var>imageData</var> to
<var>source</var>.</p></li>
<li><p>If <var>source</var> was given then:</p>
<ol>
<li><p>If <var>settings</var> was given and
<var>settings</var>["<code data-x="dom-ImageDataSettings-pixelFormat">pixelFormat</code>"]
equals "<span data-x="dom-ImageDataPixelFormat-rgba-unorm8">rgba-unorm8</span>" and
<var>source</var> is not of type <code data-x="idl-Uint8ClampedArray">Uint8ClampedArray</code>
then throw an <span>"<code>InvalidStateError</code>"</span> <code>DOMException</code>.</p></li>

<li>
<p>Otherwise (<var>source</var> was not given), initialize the <code
data-x="dom-imagedata-data">data</code> attribute of <var>imageData</var> to a new <code
data-x="idl-Uint8ClampedArray">Uint8ClampedArray</code> object. The <code
data-x="idl-Uint8ClampedArray">Uint8ClampedArray</code> object must use a new <span>Canvas
Pixel <code data-x="idl-ArrayBuffer">ArrayBuffer</code></span> for its storage, and must have a
zero start offset and a length equal to the length of its storage, in bytes. The <span>Canvas
Pixel <code data-x="idl-ArrayBuffer">ArrayBuffer</code></span> must have the correct size to
store <var>rows</var> × <var>pixelsPerRow</var> pixels.</p>
<li><p>If <var>settings</var> was given and
<var>settings</var>["<code data-x="dom-ImageDataSettings-pixelFormat">pixelFormat</code>"]
equals "<span data-x="dom-ImageDataPixelFormat-rgba-float16">rgba-float16</span>" and
<var>source</var> is not of type <code data-x="idl-Float16Array">Float16Array</code>
then throw an <span>"<code>InvalidStateError</code>"</span> <code>DOMException</code>.</p></li>

<p>If the <span>Canvas Pixel <code data-x="idl-ArrayBuffer">ArrayBuffer</code></span> cannot be
allocated, then rethrow the <code data-x="js-RangeError">RangeError</code> thrown by JavaScript,
and return.</p>
<li><p>Initialize the <dfn attribute for="ImageData"><code
data-x="dom-imagedata-data">data</code></dfn> attribute of <var>imageData</var> to
<var>source</var>.</p></li>
</ol>
</li>

<li><p>Otherwise (<var>source</var> was not given):</p>
<ol>
<li><p>If <var>settings</var> was given and
<var>settings</var>["<code data-x="dom-ImageDataSettings-pixelFormat">pixelFormat</code>"]
equals "<span data-x="dom-ImageDataPixelFormat-rgba-unorm8">rgba-unorm8</span>", or if
<var>settings</var> was not given, then
initialize the <code data-x="dom-imagedata-data">data</code> attribute of <var>imageData</var> to a new
<code data-x="idl-Uint8ClampedArray">Uint8ClampedArray</code> object.
The <code data-x="idl-Uint8ClampedArray">Uint8ClampedArray</code> object
must have a length of 4 × <var>rows</var> × <var>pixelsPerRow</var> elements,
must use a new <code data-x="idl-ArrayBuffer">ArrayBuffer</code> for its storage,
must have a byte offset of zero bytes,
and must have a byte length of 4 × <var>rows</var> × <var>pixelsPerRow</var> bytes.
If the <code data-x="idl-ArrayBuffer">ArrayBuffer</code> could not be allocated, then rethrow the
<code data-x="js-RangeError">RangeError</code> thrown by JavaScript, and return.</p></li>

<li><p>If <var>settings</var> was given and
<var>settings</var>["<code data-x="dom-ImageDataSettings-pixelFormat">pixelFormat</code>"]
equals "<span data-x="dom-ImageDataPixelFormat-rgba-float16">rgba-float16</span>", then
initialize the <code data-x="dom-imagedata-data">data</code> attribute of <var>imageData</var> to a new
<code data-x="idl-Float16Array">Float16Array</code> object.
The <code data-x="idl-Float16Array">Float16Array</code> object
must have a length of 4 × <var>rows</var> × <var>pixelsPerRow</var> elements,
must use a new <code data-x="idl-ArrayBuffer">ArrayBuffer</code> for its storage,
must have a byte offset of zero bytes,
and must have a byte length of 8 × <var>rows</var> × <var>pixelsPerRow</var> bytes.
If the <code data-x="idl-ArrayBuffer">ArrayBuffer</code> could not be allocated, then rethrow the
<code data-x="js-RangeError">RangeError</code> thrown by JavaScript, and return.</p></li>
</ol>
</li>

<li><p>Initialize the <dfn attribute for="ImageData"><code
Expand All @@ -70643,6 +70679,16 @@ try {
data-x="dom-imagedata-height">height</code></dfn> attribute of <var>imageData</var> to
<var>rows</var>.</p></li>

<li><p>If <var>settings</var> was given, then initialize the
<dfn attribute for="ImageData"><code data-x="dom-imagedata-pixelFormat">pixelFormat</code></dfn>
attribute of <var>imageData</var> to
<var>settings</var>["<dfn dict-member for="ImageDataSettings"><code
data-x="dom-ImageDataSettings-pixelFormat">pixelFormat</code></dfn>"].</p></li>

<li><p>Otherwise, initialize the <code data-x="dom-imagedata-pixelFormat">pixelFormat</code>
attribute of <var>imageData</var> to
"<span data-x="dom-ImageDataPixelFormat-rgba-unorm8">rgba-unorm8</span>".</p></li>

<li><p>If <var>settings</var> was given and <var>settings</var>["<code
data-x="dom-ImageDataSettings-colorSpace">colorSpace</code>"] <span data-x="map
exists">exists</span>, then initialize the <dfn attribute for="ImageData"><code
Expand Down Expand Up @@ -70693,13 +70739,33 @@ try {
attribute to <var>serialized</var>.[[ColorSpace]].</p></li>
</ol>

<p>A <dfn>Canvas Pixel <code data-x="idl-ArrayBuffer">ArrayBuffer</code></dfn> is an <code
data-x="idl-ArrayBuffer">ArrayBuffer</code> whose data is represented in left-to-right order, row
by row top to bottom, starting with the top left, with each pixel's red, green, blue, and alpha
components being given in that order for each pixel. Each component of each pixel represented in
this array must be in the range 0..255, representing the 8 bit value for that component. The
components must be assigned consecutive indices starting with 0 for the top left pixel's red
component.</p>
<p>The <code>ImageDataPixelFormat</code> enumeration is used to specify type of the
<code data-x="dom-imagedata-data">data</code> attribute of an <code>ImageData</code>
and the arrangement and numerical representation of the color components for each pixel.</p>

<p>The "<dfn enum-value for="ImageDataPixelFormat"><code
data-x="dom-ImageDataPixelFormat-rgba-unorm8">rgba-unorm8</code></dfn>" value indicates that the
<code data-x="dom-imagedata-data">data</code> attribute of an <code>ImageData</code> must be of type
<span data-x="idl-Uint8ClampedArray">Uint8ClampedArray</span>.
The color components of each pixel must be stored in four sequential elements in the order of red, green, blue, and then alpha.
Each element represents the 8-bit unsigned normalized value for that component.</p>

<p>The "<dfn enum-value for="ImageDataPixelFormat"><code
data-x="dom-ImageDataPixelFormat-rgba-float16">rgba-float16</code></dfn>" value indicates that the
<code data-x="dom-imagedata-data">data</code> attribute of an <code>ImageData</code> must be of type
<span data-x="idl-Float16Array">Float16Array</span>.
The color components of each pixel must be stored in four sequential elements in the order of red, green, blue, and then alpha.
Each element represents the value for that component.</p>

<p>An <code>ImageData</code> object <dfn data-x="concept-imagedata-bitmap-representation">represents a rectanglar bitmap</dfn>
with width equal to the <code data-x="dom-imagedata-width">width</code> attribute
and height equal to the <code data-x="dom-imagedata-height">height</code> attribute.
The pixel values of this bitmap are stored in the <code data-x="dom-imagedata-data">data</code> attribute
in left-to-right order, row by row from top to bottom, starting with 0 for the top left pixel,
with the order and numerical representation of the color components of each
pixel determined by the <code data-x="dom-imagedata-pixelFormat">pixelFormat</code> attribute.
The color space of the pixel values of the bitmap is determined by the
<code data-x="dom-imagedata-colorSpace">colorSpace</code> attribute.</p>

<p>The <dfn method for="CanvasImageData"><code
data-x="dom-context-2d-putImageData-short">putImageData(<var>imageData</var>,
Expand Down Expand Up @@ -70777,7 +70843,7 @@ try {
set the pixel with coordinate (<span data-x=""><var>dx</var>+<var>x</var></span>,
<var>dy</var>+<var>y</var>) in <var>bitmap</var> to the color of the pixel at coordinate
(<var>x</var>, <var>y</var>) in the <var>imageData</var> data structure's
<span>Canvas Pixel <code data-x="idl-ArrayBuffer">ArrayBuffer</code></span>,
<span data-x="concept-imagedata-bitmap-representation">bitmap</span>,
converted from <var>imageData</var>'s <code data-x="dom-imagedata-colorSpace">colorSpace</code>
to the <span data-x="concept-canvas-color-space">color space</span> of <var>bitmap</var>
using <span>'relative-colorimetric'</span> rendering intent.</p></li>
Expand Down